Training machine-learned models for perceptual tasks using biometric data

Abstract

Generally, the present disclosure is directed to systems and methods that train machine-learned models (e.g., artificial neural networks) to perform perceptual or cognitive task(s) based on biometric data (e.g., brain wave recordings) collected from living organism(s) while the living organism(s) are performing the perceptual or cognitive task(s). In particular, aspects of the present disclosure are directed to a new supervision paradigm, by which machine-learned feature extraction models are trained using example stimuli paired with companion biometric data such as neural activity recordings (e.g. electroencephalogram data, electrocorticography data, functional near-infrared spectroscopy, and / or magnetoencephalography data) collected from a living organism (e.g., human being) while the organism perceived those examples (e.g., viewing the image, listening to the speech, etc.).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS [0002] This disclosure claims priority to US Provisional Patent Application No. 62 / 801,831, filed February 6, 2019, the entire contents of which are incorporated herein by reference. technical field [0003] The present disclosure relates generally to machine learning. More specifically, the present disclosure relates to the use of biometric data, such as neural activity recordings (eg, brain wave recordings) to train machine learning models to perform perceptual tasks, eg, via multimodal learning techniques. Background technique [0004] Humans are good at a variety of perceptual and cognitive tasks, including visual object recognition, acoustic event recognition, and speech recognition. A typical way to train a machine learning model to perform these tasks is to train the model on training data, which includes training examples that have been explicitly labeled by human labelers. For example, to generate such training data...

AI Technical Summary

Problems solved by technology

While these unsupervised methods can reduce the amount of labeled data required to build new models, the performance of the resulting models consistently falls short of that made possible with only a moderate amount of human-annotated training examples

Images