Yunsu Kim

I am a Research Scientist at Lilt. I have worked on cross-lingual, semi-supervised, and unsupervised learning for neural machine translation. Currently, I’m focusing on the personalized adaptation of translation models and an efficient computer-assisted workflow for human translators.

I have a long-lasting dream of building an agent that understands and speaks languages of any country or planet in any style or tone. I want the agent to be compassionate and thoughtful with a good understanding of the situation, like Gerty.

news

Nov 6, 2021	Check out this blog post on my work and life at Lilt!
Jul 19, 2021	The new homepage is launched!

selected publications

When and Why is Unsupervised Neural Machine Translation Useless?

Yunsu Kim, Miguel Graça, and Hermann Ney

Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT 2020)

Abs PDF Slides Video

This paper studies the practicality of the current state-of-the-art unsupervised methods in neural machine translation (NMT). In ten translation tasks with various data settings, we analyze the conditions under which the unsupervised methods fail to produce reasonable translations. We show that their performance is severely affected by linguistic dissimilarity and domain mismatch between source and target monolingual data. Such conditions are common for low-resource language pairs, where unsupervised learning works poorly. In all of our experiments, supervised and semi-supervised baselines with 50k-sentence bilingual data outperform the best unsupervised results. Our analyses pinpoint the limits of the current unsupervised NMT and also suggest immediate research directions.
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

Yunsu Kim, Petre Petrov, Pavel Petrushkov, Shahram Khadivi, and Hermann Ney

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019)

Abs PDF Poster

We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods greatly outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. We show that our improvements are valid also in zero-shot/zero-resource scenarios.
When and Why is Document-level Context Useful in Neural Machine Translation?

Yunsu Kim, Duc Thanh Tran, and Hermann Ney

Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019)

Abs PDF Code Poster Slides

Document-level context has received lots of attention for compensating neural machine translation (NMT) of isolated sentences. However, recent advances in document-level NMT focus on sophisticated integration of the context, explaining its improvement with only a few selected examples or targeted test sets. We extensively quantify the causes of improvements by a document-level model in general test sets, clarifying the limit of the usefulness of document-level context in NMT. We show that most of the improvements are not interpretable as utilizing the context. We also show that a minimal encoding is sufficient for the context modeling and very long context is not helpful for NMT.
Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies

Yunsu Kim, Yingbo Gao, and Hermann Ney

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)

Abs PDF Code Poster

Transfer learning or multilingual model is essential for low-resource neural machine translation (NMT), but the applicability is limited to cognate languages by sharing their vocabularies. This paper shows effective techniques to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies. We relieve the vocabulary mismatch by using cross-lingual word embedding, train a more language-agnostic encoder by injecting artificial noises, and generate synthetic data easily from the pretraining data without back-translation. Our methods do not require restructuring the vocabulary or retraining the model. We improve plain NMT transfer by up to +5.1% BLEU in five low-resource translation tasks, outperforming multilingual joint training by a large margin. We also provide extensive ablation studies on pretrained embedding, synthetic data, vocabulary size, and parameter freezing for a better understanding of NMT transfer.