Федотов Дмитрий Валерьевич Контекстно-зависимое распознавание эмоций на основе многомодальных данных

brief description

Федотов Дмитрий Валерьевич Контекстно-зависимое распознавание эмоций на основе многомодальных данных
ОГЛАВЛЕНИЕ ДИССЕРТАЦИИ
кандидат наук Федотов Дмитрий Валерьевич
1.1 Emotion Recognition

1.2 Contextual Information

1.3 Smart Environments

1.4 Dialogue Systems

1.5 Motivation

1.6 Thesis Contributions

1.7 Outline

2 Background and Related Research

2.1 Approaches to Emotion Recognition

2.2 Contextual Emotion Recognition

2.2.1 Speaker Context

2.2.2 Dialogue Context

2.2.3 Environmental Context and User State Recognition in Smart Environments

2.3 Organized Challenges on Emotion Recognition

2.4 Background in Machine Learning Algorithms

2.4.1 Neural Networks

2.4.2 Ridge Regression

2.4.3 Support Vector Machines

2.4.4 XGBoost

2.5 Summary

3 Data and Tools

3.1 Corpora

3.1.1 RECOLA

3.1.2 SEMAINE

3.1.3 SEWA

3.1.4 IEMOCAP

3.1.5 UUDB

3.1.6 Summary

3.2 Data Preprocessing

3.2.1 Data Cleaning

3.2.2 Feature Extraction

3.2.3 Gold Standard and Annotations Shifting - Concept and General Approaches

3.2.4 Gold Standard and Annotations Shifting - Combination of Approaches

3.3 Evaluation Metrics

3.4 Summary

4 Modeling Speaker Context in Time-continuous Emotion Recognition

4.1 Straightforward Approach

4.1.1 Feature Based Time-Dependent Models

4.1.2 Raw Data Based Time-Dependent Models

4.1.3 Feature Based Time-Independent Models

4.2 DataSparsing

4.2.1 General Concept

4.2.2 Data Sparsing for Feature Based Time-Dependent Models

4.2.3 Data Sparsing with Varying Feature Window

4.3 Transferability to Cross-corpus Setting

4.4 Analysis and Discussion

4.5 Summary

5 Utilizing Contextual Information in Dyadic Interactions

5.1 Discovering Mutual Effects in Emotional Dynamics of Interaction

5.2 Dependent Dyadic Context Modeling

5.2.1 Feature-level fusion

5.2.2 Decision-level fusion

5.3 Independent Dyadic Context Modeling

5.4 Analysis and Discussion

5.5 Summary

6 Towards Contextual Emotion Recognition in Smart Environments

6.1 Smart Tourism

6.2 EmoTourDB

6.2.1 Data collection

6.2.2 Features

6.2.3 Labels

6.2.4 Additional information

6.2.5 Synchronisation and Calibration

6.2.6 Missing Data

6.3 Modeling

6.4 Discussions and Limitations

6.5 Summary

7 Conclusion and Future Directions

7.1 Overall Summary

7.2 Thesis Contributions

7.2.1 Theoretical

7.2.2 Practical

7.2.3 Experimental

7.3 Future Directions

A Heat maps representation of performance graphs

B Additional results for speaker context modeling in cross-corpus scenario

C Additional results for sparsing analysis in speaker context modeling

References

Acronyms

List of Figures

List of Tables

Реферат