Reading notes from Monday 11/4.
Low-resource Cross-lingual Event Type Detection via Distant Supervision with Minimal Effort
What is the problem the paper identifies?
Training models in low-resource languages typically depends on using labeled data from a high-resource language for reference. However, even high-resource languages may lack sufficient labeled data.
What is the contribution the paper presents?
The authors propose two solutions. The first is to create supervised data through the use of a bilingual dictionary and minimal outside annotation. The second is to use machine learning techniques. When tested, these two methods perform about equally well in a classification task. The paper then explores how to achieve maximum efficient performance.
Give more detail about the specific problem the paper is addressing.
The current established way of applying NLP methods to low-resource languages its to transfer the learning process from a high-resource language using a shared word representation (bilingual dictionary), which models are then trained on. But this approach assumes that there is a large amount of labeled data in the high-resource language.
The paper introduces two methods of solution:
These two are found to work equally well in the experiment the paper utilizes: a situation frame (SF) detection task. An SF is either an issue or a need from a news article; situation frame detection is the task of classifying SFs (e.g., in an article about a food shortage, classification as a food need).
Lastly, the paper proposes methods for combining the two methods outlined above to maximize performance, and analyzes their results.
What contributions does the work provide?
Keyword-based models:
Heuristic and simple, yet able to complete tasks such as text classification and information retrieval. Also able to use bilingual dictionaries to perform well cross-lingually. Possible downside is that they only perform well in the specific task contained in this study (situation frames), because event detection is usually based upon a noun or keyword that is easier to translate.
Cross-lingual text classification (CLTC):
Very similar to cross-lingual event type detection (which is what we're using in our SF tasks). Aims to classify text using data from an auxiliary language. There are multiple sub-methods within CLTC, such as the use of bilingual dictionaries to translate models and automatic machine translation.
Bilingual Word Embedding
This method shares embedding space between two languages; with this we are able to project vectors onto the shared space.
Models
adversarial neural network
To use an adversarial neural network, the authors trained a bilingual word embedding (which then had shared feature representation across the auxiliary and target languages). Based on this representation, they then used a convolutional neural network (CNN) which is a deep neural classifier. There were two benefits to using a deep neural classifier (as opposed to a shallow one): better performance in previous classification tasks, and the ability to incorporate bilingual word embeddings.
Because bilingual word embeddings do not capture the ground-truth mapping between two languages (as they would ideally), the authors closed the resulting domain gap using adversarial training techniques. In summary, this makes the CNN capture event information while being language-invariant.
keyword-based model
These are used by first building keywords for SF types in English and then directly translating them to the target language with the use of a bilingual dictionary. Manual checks were imposed, such as the help of native speakers to refine translation. Keyword selection in English was also refined using similarity metrics.
Methods
Training data was moderated in two ways.
Bootstrapping language-specific keywords
Keyword matching can result in low coverage (due to higher variability within a subject in the target language relative to the variability of the auxiliary language and missing data in bilingual dictionaries). To address this, the authors develop an iterative bootstrapping algorithm that generates additional language-specific keywords. They do this in a two-step process.
Clustering takes text from each language and clusters them, then identifies non-keyword terms that appear over a certain threshold similarity metric th_2. This is meant to find words that were previously missed.
Labeling takes the now-updated set of text documents and relabels them. Now having generated a new set of text data, the process is repeated until coverage is satisfactory.
Manual annotation of the target language
The authors obtained annotated keywords by enlisting the aid of native Tigrinya speakers, as well as non-speaker linguists.
Experiments
The datasets consisted of sentences from Tigrinya and Oromo (both Ethiopian languages).
The authors used both sentence-level prediction and then document-level aggregation. The former provides more training data and made the training easier. These results were then combined, and each SF was assigned the maximum confidence score across all sentences present in a document (with the lower-confidence predictions filtered out).