Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A transfer learning method from macro expression to micro expression

A transfer learning and micro-expression technology, applied in the field of computer vision and pattern recognition, can solve problems such as limited binding force, weak strength, and unsatisfactory recognition effect, and achieve the effect of improving the recognition rate and overcoming the limited number of samples

Active Publication Date: 2019-03-01
SHANDONG UNIV
View PDF7 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The duration of micro-expressions is only 0.04s-0.2s, and the intensity is extremely weak. Due to the influence of lighting, noise and other factors, the recognition effect of micro-expression feature representation methods is still not ideal.
The current linear subspace learning method has enabled micro-expression recognition to have a better classification ability through sparse constraints, but this constraint is limited and is still largely affected by human facial features, light, and noise. Difficulties in feature representation and extraction of micro-expressions
At the same time, there are still limitations in the limited number of samples in the micro-expression database and the difficulties in sample data collection and labeling.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A transfer learning method from macro expression to micro expression
  • A transfer learning method from macro expression to micro expression
  • A transfer learning method from macro expression to micro expression

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0078] A transfer learning method from macro-expression to micro-expression, such as figure 1 shown, including the following steps:

[0079] (1) Extract macro-expression features and micro-expression features respectively: extract LBP features from macro-expression to form macro-expression feature representation matrix X a , micro-expression extraction LBP-TOP features to form a micro-expression feature representation matrix X b , and take the average of the three orthogonal planes of the LBP-TOP feature. The original LBP-TOP feature is the LBP feature extracted from the three orthogonal planes of XY, XT, and YT, forming a 3*59-dimensional feature. Here, for two The dimensions of expression are unified, and the LBP features on the three orthogonal planes are accumulated and averaged to form a 59-dimensional LBP-TOP feature, forming a micro-expression representation feature that is unified with the macro-expression dimension;

[0080] (2) Project the micro-expression feature ...

Embodiment 2

[0085] According to a method of transfer learning from macro-expressions to micro-expressions described in Embodiment 1, the difference is that,

[0086] In step (2), the objective function is as shown in formula (I):

[0087]

[0088] In formula (I), α, β, γ are balance coefficients, α>0, β>0, γ>0;

[0089] x a is the macro-expression feature matrix, n a Represents the number of macro expression samples, d represents the feature dimension;

[0090] x b is the micro-expression feature matrix, n b Represents the number of dimension expression samples, and d represents the feature dimension;

[0091] P∈R d×np , P is the projection matrix, np is the subspace dimension;

[0092] V is the reconstruction coefficient matrix (equivalent to the secondary projection matrix);

[0093] E is an error matrix, which stores the respective characteristic components in the macro-expression and micro-expression, and the characteristic components include their unique characteri...

Embodiment 3

[0112] According to a method of transfer learning from macro-expression to micro-expression described in embodiment 1 or 2, the difference is that

[0113] In step (3), the projection matrix P is initialized by NMF, and the target projection matrix is ​​iteratively optimized by solving the objective function, and the target projection matrix P is output when the convergence condition is met. 1 , including the following steps:

[0114] E. Convert the objective function into an approximate form for solution: such as figure 2 said;

[0115] The objective function is shown in formula (Ⅵ):

[0116]

[0117] Considering various constraints, the objective function is non-convex, and the objective function is transformed into an approximate form, as shown in formula (VII):

[0118]

[0119] In formula (VII), Y is by the eigenvector that WY=ΛDY problem solves as the matrix that row vector forms, Λ is to form diagonal matrix as diagonal element by corresponding eigenvalue;

...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a transfer learning method from macro expression to micro expression, in the process of training, the model adopts subspace projection, the micro-expression feature matrix andmacro-expression feature matrix are projected into a common subspace, and the macro-expression feature matrix is used to reconstruct the micro-expression feature matrix. The macro-expression and micro-expression features and structural differences in the common subspace are minimized by imposing constraints to maximize the correlation between macro-expression and micro-expression in the subspace,and the target projection matrix is iteratively optimized. Finally, the test set data are projected to the common subspace using the target projection matrix and classified by KNN. The useful information of the existing macro-expression domain samples is transferred to the micro-expression domain, which is equivalent to expanding the number of labeled micro-expression samples, thus overcoming thedisadvantage that the micro-expression samples are difficult to label. This method not only reduces the waste of label manpower, but also greatly improves the recognition performance and provides another strategy for micro-expression recognition.

Description

technical field [0001] The invention belongs to the field of computer vision and pattern recognition, and relates to a transfer learning method from macro-expression to micro-expression, in particular to a non-negative matrix factorization (NMF) for initialization, which can maintain the common latent characteristics of the two domains before and after projection and promote A method for transfer learning from macro-expression to micro-expression based on subspace projection of two-domain distance. technical background [0002] As early as 1966, Haggard and others discovered the existence of micro-expressions, see Haggard E A, Isaacs KS. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy[M] / / Methods of research in psychotherapy. Springer, Boston, MA, 1966 :154-165. They think that micro-expression is one of the manifestations of the human body's self-protection mechanism, which can express the thoughts or emotions that people suppress and don'...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N99/00
CPCG06V40/174G06F18/211
Inventor 贲晛烨肖瑞雪王德强朱雪娜巩力铜赵耕达
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com