SentAlign: Accurate and Scalable Sentence Alignment
Stein{\th}\'or Steingr\'imssonHrafn LoftssonAndy Way
Stein{\th}\'or Steingr\'imssonHrafn LoftssonAndy Way
Nov 2023
0被引用
1笔记
摘要原文
We present SentAlign, an accurate sentence alignment tool designed to handle very large parallel document pairs. Given user-defined parameters, the alignment algorithm evaluates all possible alignment paths in fairly large documents of thousands of sentences and uses a divide-and-conquer approach to align documents containing tens of thousands of sentences. The scoring function is based on LaBSE bilingual sentence representations. SentAlign outperforms five other sentence alignment tools when evaluated on two different evaluation sets, German-French and English-Icelandic, and on a downstream machine translation task.
