Check an English translation of an Arabic book for gaps

A translation of an Arabic book can read fluently and still be missing things. A paragraph gets skipped, a long quotation is summarized in one line, a hadith number changes from 224 to 244, or the same scholar is spelled three ways across the chapters. None of this shows when you read the English on its own, because fluent English is exactly what a good machine or human translation produces.

This guide is for anyone who has to sign off on an English translation of an Arabic text before using it: a researcher citing a classical work, an editor or publisher preparing a translation for print, or a translator revising a first draft (their own, a colleague's or a machine's). It gives a check you can run in minutes on two Word files, what each flag means, and the reading a bilingual reviewer still has to do.

What goes wrong, and why fluency hides it

The Linguistic Data Consortium's Arabic-to-English guidelines list the error categories its reviewers grade: missed translation, added translation, wrong translation, and "overlooked file or section", which carries by far the heaviest penalty in its scoring table. The same guidelines say numbers in a translation should follow how they appear in the source, and that names with a conventional English form should use it.

Omissions are the hardest of these to catch by reading. In a paper on detecting them, Graham Russell quotes Melamed's rule of thumb that intended omissions "are seldom longer than a few words, while accidental omissions are often on the order of a sentence or more", and then warns that it is "far from wholly reliable": a translator may drop a paragraph on purpose, and a single missing word can change the meaning. So a tool can point you to likely gaps, but a person has to decide each one.

For a classical Arabic book, the things most worth protecting are:

• Whole passages: a quotation, a chain of narrators, a list of opinions.
• Numbers: Hijri years, ages, quantities, and the reference numbers of hadith, verses and pages.
• Names: scholars, places and book titles, which readers will search for.
• Structure: chapter and section headings, which readers use to cite the work.

Step 1: put both files in Word with matching headings

The check below compares the two files section by section, so both need the same sections marked as headings (Word's Heading 1, Heading 2 and so on).

1. Open the Arabic source and the English translation as .docx files.
2. Make sure each chapter or section title uses a Heading style in both files. If the translation came back with headings as plain bold text, apply the styles in the translation first; it is quicker than fixing them in the Arabic.
3. Keep footnotes or notes in the same place in both files (both as Word footnotes, or both as paragraphs). Mixing the two makes the paragraph counts disagree for no real reason.

If the Arabic came from a scanned book, check it first: a converted file with merged lines or missing pages will make a good translation look wrong. Our pre-delivery check for OCR'd Arabic Word files covers that.

Step 2: run the section-by-section comparison

This script uses only the Python standard library. It reads both .docx files, splits them at headings, and for each section reports:

• a different number of paragraphs;
• a section whose English is much shorter, relative to its Arabic, than the book's typical section (below 0.6 times the median ratio, a cut-off you can change);
• numbers that appear in one file but not the other, after converting Arabic-Indic (٠-٩, U+0660 to U+0669) and Persian (۰-۹, U+06F0 to U+06F9) digits to 0-9.

Optionally, it reads a list of names and flags variant spellings in the English.

"""Compare an Arabic .docx with its English translation, section by section.

Usage: python3 compare_translation.py source.docx translation.docx [variants.txt]
Sections start at paragraphs whose style name begins with "Heading".
variants.txt (optional): one line per name, preferred form first, then
variants to flag, separated by " | ".
"""
import re
import sys
import zipfile
import xml.etree.ElementTree as ET
from collections import Counter
from statistics import median

W = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
DIGITS = str.maketrans("٠١٢٣٤٥٦٧٨٩۰۱۲۳۴۵۶۷۸۹", "01234567890123456789")


def paragraphs(path):
    with zipfile.ZipFile(path) as z:
        root = ET.fromstring(z.read("word/document.xml"))
    for p in root.iter(W + "p"):
        style = p.find(f"{W}pPr/{W}pStyle")
        name = style.get(W + "val") if style is not None else ""
        text = "".join(t.text or "" for t in p.iter(W + "t")).strip()
        if text:
            yield name, text


def sections(path):
    out = [{"title": "(before first heading)", "paras": []}]
    for style, text in paragraphs(path):
        if style.lower().startswith("heading"):
            out.append({"title": text, "paras": []})
        else:
            out[-1]["paras"].append(text)
    return [s for s in out if s["paras"] or s["title"] != "(before first heading)"]


def numbers(texts):
    joined = " ".join(texts).translate(DIGITS)
    joined = re.sub(r"(?<=\d)[,٬](?=\d{3})", "", joined)
    return Counter(re.findall(r"\d+", joined))


src, tgt = sections(sys.argv[1]), sections(sys.argv[2])
print(f"sections: source {len(src)}, translation {len(tgt)}")
if len(src) != len(tgt):
    print("  MISMATCH: compare the headings before trusting anything below")

ratios = []
for s, t in zip(src, tgt):
    a = sum(len(p) for p in s["paras"])
    b = sum(len(p) for p in t["paras"])
    ratios.append(b / a if a else 0)
typical = median(ratios) if ratios else 0

for i, (s, t) in enumerate(zip(src, tgt), 1):
    flags = []
    if len(s["paras"]) != len(t["paras"]):
        flags.append(f"paragraphs {len(s['paras'])} vs {len(t['paras'])}")
    if typical and ratios[i - 1] < 0.6 * typical:
        flags.append(f"short: length ratio {ratios[i-1]:.2f}, typical {typical:.2f}")
    missing = numbers(s["paras"]) - numbers(t["paras"])
    extra = numbers(t["paras"]) - numbers(s["paras"])
    if missing:
        flags.append("numbers missing: " + " ".join(sorted(missing.elements())))
    if extra:
        flags.append("numbers added: " + " ".join(sorted(extra.elements())))
    print(f"{i:>3} {t['title'][:40]:<40} " + ("; ".join(flags) if flags else "ok"))

if len(sys.argv) > 3:
    text = " ".join(p for sec in tgt for p in sec["paras"])
    for line in open(sys.argv[3], encoding="utf-8"):
        forms = [f.strip() for f in line.split("|") if f.strip()]
        for v in forms[1:]:
            n = len(re.findall(r"(?<![\w-])" + re.escape(v) + r"(?!\w)", text))
            if n:
                print(f"variant '{v}' x{n}: preferred '{forms[0]}'")

Run it with python3 compare_translation.py source.docx translation.docx names.txt. The names file looks like this:

al-Ghazali | al-Ghazzali | Ghazzali
Ibn Majah | Ibn Maja

Illustrative example. While writing this guide we built a short four-chapter Arabic file and an English version with four deliberate faults: a long quotation in chapter two cut to one sentence, a hadith number changed from ٢٢٤ to 244 in chapter three, two paragraphs merged into one in chapter four, and al-Ghazali spelled al-Ghazzali once. The script (Python 3.11) printed:

sections: source 4, translation 4
  1 Chapter One                              ok
  2 Chapter Two                              short: length ratio 0.67, typical 1.35
  3 Chapter Three                            numbers missing: 224; numbers added: 244
  4 Chapter Four                             paragraphs 2 vs 1
variant 'al-Ghazzali' x1: preferred 'al-Ghazali'

When we deleted the chapter three heading from the English, it reported "sections: source 4, translation 3" and a mismatch warning, and the later flags shifted to the wrong chapters. That is why the heading count comes first: fix it before reading anything else.

Step 3: read what the flags mean

Flag	Usual causes	What to do
Section count differs	A heading not styled, two chapters merged, a chapter missing	Line up the headings by hand, then rerun
Paragraph count differs	Paragraphs merged or split (harmless), or one dropped	Open both at that section and match paragraph by paragraph
Short section	A quotation summarized, a list cut, a passage skipped	Read the Arabic of that section against the English in full
Number missing or added	A typo, a misread digit, a Hijri date converted to Gregorian, a number written out in words	Check against the Arabic; if a date was converted, the source year should still appear
Name variant	Different romanization in different chapters, or a different translator per chapter	Choose one form and replace throughout

Not every flag is an error. English often merges short Arabic paragraphs, and a translator may write "three hundred" where the Arabic has ٣٠٠. The flags tell you where to look, which on a book-length file is most of the work.

Step 4: settle the names

Readers search a translation by name, so one person spelled three ways is three people to a search box. Decide the convention once and write it down:

• Scholarly romanization: the Library of Congress's ALA-LC Arabic table (2012 version) is widely used in libraries and academic publishing. Its examples write the article with a hyphen (al-Miṣrī) and mark long vowels and emphatic letters (Raḍī al-Dīn).
• Simplified spelling: many trade translations drop the dots and macrons (Radi al-Din). That is a valid choice if it is consistent.
• Established English forms: the LDC guidelines say a name with a conventional English form should use it, even when a strict romanization would differ.

Put every name, place and book title into the names file as you go, with the variants you have seen. Running the script after each revision catches a variant the moment it comes back.

Step 5: the bilingual read

The script cannot tell you whether a sentence is translated correctly, only whether something is likely missing or changed. A reader who knows both languages still has to:

1. Read every flagged section in full against the Arabic.
2. Read a sample of unflagged sections too. Pick them on purpose: the densest legal or doctrinal passage, a page of poetry, and a page with many names.
3. Check every Qur'an verse and hadith reference against the source edition, not just the number.
4. Check that the translator's notes and brackets are marked as additions, so readers can tell them from the author's text.

The LDC guidelines grade at least 1,200 words drawn from several documents in each delivery. That is a reasonable floor for a sample, but on a book a flagged section should always be read in full whatever the sample size.

Checklist before you rely on the translation

• Both files have the same headings, styled as headings
• Section count matches
• Every "short" section read in full against the Arabic
• Every paragraph-count difference explained (merged, split or missing)
• Every missing or added number checked, including Hijri dates and reference numbers
• One romanization convention chosen and written down
• Names file built and no variants left
• Qur'an and hadith references checked against the source edition
• Translator's additions marked as such
• A bilingual reader has signed off on flagged sections and a deliberate sample

Limitations

• The script compares lengths, counts and digits. It cannot judge meaning, so an accurate- looking section can still contain a wrong translation.
• Numbers written as words (three hundred, ثلاثمائة) are not compared. Neither are dates converted from Hijri to Gregorian, which will appear as a missing and an added number.
• The length cut-off is a heuristic. A section of poetry or dense terminology can be flagged short or long for legitimate reasons.
• It needs both files as .docx with headings styled. It does not read PDF, and it ignores footnotes, text boxes and tables.
• We tested it on small files built for this guide, not on a full book, and not on output from any particular translation tool, including ScribeTools.
• None of this makes a translation certified or suitable for legal use. It only helps a reviewer find what to read first.

If you are starting from a scanned Arabic book, see digitizing Arabic books for getting an editable source first, and handing an OCR'd Word file to a translator for preparing it for translation tools. If the book's notes matter, rebuild them as real Word footnotes in the Arabic before you compare.