Ле Тхе Ань Deep Neural Network Models for Sequence Labeling and Coreference Tasks/ Глубокие нейросетевые модели для задач разметки последовательности и разрешения кореференции

Короткий опис

Ле Тхе Ань Deep Neural Network Models for Sequence Labeling and Coreference Tasks/ Глубокие нейросетевые модели для задач разметки последовательности и разрешения кореференции
ОГЛАВЛЕНИЕ ДИССЕРТАЦИИ
кандидат наук Ле Тхе Ань
Contents

Abstract

Acknowledgments

Abbreviations

List of Figures

List of Tables

1 Introduction

1.1 Overview of Deep Learning

1.1.1 Artificial Intelligence, Machine Learning, and Deep Learning

1.1.2 Milestones in Deep Learning History

1.1.3 Types of Machine Learning Models

1.2 Brief Overview of Natural Language Processing

1.3 Dissertation Overview

1.3.1 Scientific Actuality of the Research

1.3.2 The Goal and Task of the Dissertation

1.3.3 Scientific Novelty

1.3.4 Theoretical and Practical Value of the Work in the Dissertation

1.3.5 Statements to be Defended

1.3.6 Presentations and Validation of the Research Results

1.3.7 Publications

1.3.8 Dissertation Structure

2 Deep Neural Network Models for NLP Tasks

2.1 Word Representation Models

2.1.1 Word Representation

2.1.2 Prediction-based Models

2.1.3 Count-based Models

2.2 Deep Neural Network Models

2.2.1 Convolutional Neural Network

2.2.2 Recurrent Neural Network

2.2.3 Long Short-Term Memory Cells

2.2.4 LSTM Networks

2.3 Pre-trained Language Models

2.3.1 ELMo

2.3.2 Transformer

2.3.3 OpenAI's GPT

2.3.4 Google's BERT

2.4 Summary

3 Sequence Labeling with Character-aware Deep Neural Networks and Language Models

3.1 Introduction to the Sequence Labeling Tasks

3.2 Related Work

3.2.1 Rule-based Models

3.2.2 Feature-based Models

3.2.3 Deep Learning-based Models

3.2.4 Related Work on Vietnamese Named Entity Recognition

3.2.5 Related Work on Russian Named Entity Recognition

3.3 Tagging Schemes

3.4 Evaluation Metrics

3.5 WCC-NN-CRF Models for Sequence Labeling Tasks

3.5.1 Backbone WCC-NN-CRF Architecture

3.5.2 Language Model-based Architecture

3.6 Application of WCC-NN-CRF Models for Named Entity Recognition

3.6.1 Overview of Named Entity Recognition Task

3.6.2 Datasets and Pre-trained Word Embeddings

3.6.3 Evaluation of backbone WCC-NN-CRF Model

3.6.4 Evaluation of ELMo-based WCC-NN-CRF model

3.6.5 Evaluation of BERT-based Multilingual WCC-NN-CRF Model

3.7 Application of WCC-NN-CRF Model for Sentence Boundary Detection

3.7.1 Introduction to the Sentence Boundary Detection Task

3.7.2 Sentence Boundary Detection as a Sequence Labeling Task

3.7.3 Evaluation of WCC-NN-CRF SBD Model

3.8 Conclusions

4 Coreference Resolution with Sentence-level Coreferential Scoring

4.1 The Coreference Resolution Task

4.2 Related Work

4.2.1 Rule-based Models

4.2.2 Deep Learning Models

4.3 Coreference Resolution Evaluation Metrics

4.4 Baseline Model Description

4.5 Sentence-level Coreferential Relation-based Model

4.6 BERT-based Coreference Model

4.7 Experiments and Results

4.7.1 Datasets

4.7.2 Evaluation of Proposed Models

4.8 Conclusions

5 Conclusions

5.1 Conclusions for Sequence Labeling Task

5.2 Conclusions for Coreference Resolution Task

5.3 Summary of the Main Contributions of the Dissertation

Bibliography