Rebuild headings and a table of contents in a converted Arabic book

A scanned Arabic book or thesis turned into a Word file usually looks right: «الفصل الأول» is centred and bold, «المبحث الثاني» sits above its section in a larger size. But to Word, most of those lines are ordinary paragraphs. Open the Navigation pane and it is empty or patchy. Insert a table of contents and Word says it has nothing to build from, or it lists half the chapters.

Microsoft's own help puts it plainly: "Word uses the headings in your document to build an automatic table of contents", and "missing entries often happen because headings aren't formatted as headings." This guide is for the editor, muhaqqiq, translator or publisher who has a converted book and needs a real outline: for a contents page that updates, for navigation while proofreading, for splitting the book among translators, or for an EPUB later. It goes deeper than the one line about heading styles in the QA checklist before delivery.

Why a bold line is not a heading

Word decides what belongs in the outline from each paragraph's outline level, not its appearance. In the file format this is the w:outlineLvl element. ISO/IEC 29500 says it "shall not affect the appearance of the text in the document" but is used to calculate a table-of-contents field, with values 0 to 9, where 9 (or no value at all) means "no level". The built-in Heading 1, Heading 2 and Heading 3 styles carry levels 1, 2 and 3. A paragraph in Normal style with bold, centred, 18-point text has no level, however much it looks like a chapter title.

Conversion tools rebuild what a page looks like, and they can't always tell a chapter title from an emphasised line. So whatever produced your file, ScribeTools included, check the outline before you rely on it.

Step 1: map the book's divisions to levels

Classical and modern Arabic books divide content with named units, and the hierarchy differs from book to book. A common pattern in fiqh works and theses is كتاب › باب › فصل, or باب › فصل › مبحث › مطلب › فرع. Some books put فصل under مبحث. Don't guess. Read the printed contents list (فهرس الموضوعات, often at the back of Arabic books) and write a short table before you touch a style:

Printed unit	Word style	Notes
المقدمة، التمهيد، الخاتمة	Heading 1	Same level as a chapter
الفصل	Heading 1
المبحث	Heading 2
المطلب	Heading 3
Unnamed bold sub-titles	none, or Heading 4	Decide once, for the whole book

Illustrative example: a thesis on the chapter of purification uses فصل › مبحث › مطلب, so فصل is Heading 1. A different book in the same series uses باب › فصل; there فصل is Heading 2. The same word can sit at different levels in different books, but never in the same book.

Step 2: apply the heading styles

1. Keep the scan or the printed contents list open beside the Word file.
2. Click in a chapter title, then go to Home > Styles and choose Heading 1. Do the same for each level in your table.
3. If the converter used its own style (for example "Chapter Title") consistently, you don't need to restyle every line. Either give that style an outline level in its paragraph settings, or select all text using it and apply the Heading style in one go.
4. Change the look of a level by modifying the Heading style itself (font, size, alignment, right-to-left), not by formatting each title by hand. Every heading at that level then stays identical.
5. Leave headings out of tables and text boxes. Microsoft notes that "the Navigation pane doesn't display headings that are in tables, text boxes, or headers or footers." Converters sometimes put a centred title in a one-cell table. Move it out into the body.

Open the Navigation pane (View > Navigation Pane, or Ctrl+F) and compare it with the printed contents list as you go. You can right-click a heading in the pane to change its level.

Step 3: run an outline check

Styling hundreds of headings by eye leaves gaps. The script below (Python 3, standard library only) reads the .docx, works out each paragraph's outline level from direct formatting or its style, and reports:

• lines that start with a division word (الفصل، المبحث، المطلب…) but have no heading level
• short, fully bold lines with no level, to check against the scan
• typed contents lines: a title, a dot or tatweel leader and a page number
• jumps of more than one level, such as Heading 1 straight to Heading 3
• the same division word used at two different levels
• headings inside tables

It ends with the outline as Word will see it. Edit KEYWORDS to fit your book.

"""Report heading problems in a converted Arabic .docx before building a table of contents.
Usage: python3 check_headings.py book.docx"""
import re, sys, zipfile
import xml.etree.ElementTree as ET

W = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
KEYWORDS = [  # words that open a heading in many Arabic books; edit for yours
    "الكتاب", "الباب", "الفصل", "المبحث", "المطلب", "الفرع", "المسألة",
    "تمهيد", "التمهيد", "مقدمة", "المقدمة", "خاتمة", "الخاتمة"]
KW = re.compile(r"^\s*(" + "|".join(KEYWORDS) + r")(\s|:|$)")
LEADER = re.compile(r"(\.{4,}|…{2,}|_{4,}|ـ{4,})\s*[\d٠-٩۰-۹]+\s*$")

def val(el, tag):
    x = el.find(tag) if el is not None else None
    return x.get(W + "val") if x is not None else None

def load_styles(z):
    root = ET.fromstring(z.read("word/styles.xml"))
    styles = {}
    for s in root.iter(W + "style"):
        sid = s.get(W + "styleId")
        lvl = val(s.find(W + "pPr"), W + "outlineLvl")
        styles[sid] = {"name": val(s, W + "name") or sid,
                       "based": val(s, W + "basedOn"),
                       "lvl": int(lvl) if lvl is not None else None,
                       "bold": s.find(f"{W}rPr/{W}b") is not None}
    return styles

def style_level(styles, sid, seen=()):
    s = styles.get(sid)
    if s is None or sid in seen:
        return None
    if s["lvl"] is not None:
        return s["lvl"]
    m = re.fullmatch(r"heading (\d)", s["name"].lower())
    if m:  # built-in Heading 1-9 carry level n-1 in Word even if styles.xml omits it
        return int(m.group(1)) - 1
    return style_level(styles, s["based"], seen + (sid,))

def paragraphs(z):
    root = ET.fromstring(z.read("word/document.xml"))
    body = root.find(W + "body")
    in_table = {id(p) for t in body.iter(W + "tbl") for p in t.iter(W + "p")}
    for p in body.iter(W + "p"):
        text = "".join(t.text or "" for t in p.iter(W + "t")).strip()
        if text:
            yield p, text, id(p) in in_table

def main(path):
    z = zipfile.ZipFile(path)
    styles = load_styles(z)
    findings, outline, kw_levels, prev = [], [], {}, -1
    for n, (p, text, in_table) in enumerate(paragraphs(z), 1):
        ppr = p.find(W + "pPr")
        sid = val(ppr, W + "pStyle") or "Normal"
        direct = val(ppr, W + "outlineLvl")
        lvl = int(direct) if direct is not None else style_level(styles, sid)
        if lvl == 9:
            lvl = None
        runs = [r for r in p.iter(W + "r") if "".join(t.text or "" for t in r.iter(W + "t")).strip()]
        bold = bool(runs) and all(r.find(f"{W}rPr/{W}b") is not None for r in runs)
        short = len(text.split()) <= 8 and not text.endswith((".", "،", ":", "؛"))
        kw = KW.match(text)
        short_text = text[:40]
        if LEADER.search(text):
            findings.append(f"para {n}: typed contents line (dot leader + page number): «{short_text}»")
            continue
        if lvl is None:
            if kw:
                findings.append(f"para {n}: starts with «{kw.group(1)}» but has no heading level (style {styles.get(sid, {}).get('name', sid)}): «{short_text}»")
            elif bold and short:
                findings.append(f"para {n}: short bold line with no heading level, check against the scan: «{short_text}»")
            continue
        if in_table:
            findings.append(f"para {n}: heading inside a table; the Navigation pane will not show it: «{short_text}»")
        if lvl > prev + 1:
            findings.append(f"para {n}: jumps from level {prev + 1} to level {lvl + 1}: «{short_text}»")
        if kw:
            kw_levels.setdefault(kw.group(1), set()).add(lvl + 1)
        outline.append("  " * lvl + f"{lvl + 1} {text[:50]}")
        prev = lvl
    for word, levels in kw_levels.items():
        if len(levels) > 1:
            findings.append(f"«{word}» headings use more than one level: {sorted(levels)}")
    print(f"{len(findings)} finding(s)")
    for f in findings:
        print(" -", f)
    print("Outline Word will use:")
    print("\n".join(outline) if outline else "  (none)")

if __name__ == "__main__":
    main(sys.argv[1])

Illustrative example: we ran it in this session on a small test file built by hand to contain each problem (two typed contents lines, an unstyled «المقدمة», a فصل followed straight by a Heading 3 مطلب, a converter style with no level, an unstyled bold sub-title, a second فصل styled as Heading 2 and a heading inside a table). Paragraph numbers count non-empty paragraphs only. It printed:

8 finding(s)
 - para 1: typed contents line (dot leader + page number): «المقدمة ........................ ٥»
 - para 2: typed contents line (dot leader + page number): «الفصل الأول: في الطهارة ........ ١٢»
 - para 3: starts with «المقدمة» but has no heading level (style Normal): «المقدمة»
 - para 7: jumps from level 1 to level 3: «المطلب الأول: في المياه»
 - para 9: starts with «المبحث» but has no heading level (style Chapter Title): «المبحث الثاني: في الآنية»
 - para 11: short bold line with no heading level, check against the scan: «حكم الماء المستعمل»
 - para 14: heading inside a table; the Navigation pane will not show it: «المبحث الأول: في المواقيت»
 - «الفصل» headings use more than one level: [1, 2]
Outline Word will use:
1 الفصل الأول: في الطهارة
    3 المطلب الأول: في المياه
  2 الفصل الثاني: في الصلاة
  2 المبحث الأول: في المواقيت

On a matching file with every title styled from the table in Step 1, it printed 0 finding(s) and a clean six-line outline. Short bold lines are only candidates. Plenty of them, such as emphasised opening words or poetry, are not headings, so decide each one against the scan.

Step 4: replace the typed contents with an automatic one

OCR copies the printed contents page as text: each title, a row of dots and a printed page number. Those numbers point at the printed edition and will never match Word's pages.

1. Keep the typed list in a separate file. You'll check against it in Step 5.
2. Delete it from the book, then put the cursor where the contents should go.
3. Go to References > Table of Contents and choose an automatic style. A manual style does not use your headings and won't update.
4. After any edit to titles or page flow, right-click the contents and choose Update Field, then update the entire table.

If readers must cite the printed edition, the automatic contents' page numbers are Word's pages, not the print's. Keep printed page markers in the text as described in keeping printed page numbers in a converted Arabic book, and say in the front matter which numbering the contents use.

Step 5: check the result against the printed contents

• Every entry in the printed فهرس appears in the Navigation pane, in the same order
• No extra entries: emphasised lines, verse or running heads styled as headings by mistake
• Each division word uses one level throughout the book
• No level is skipped (Heading 1 → Heading 3)
• Title wording matches the print, including the ordinal (الأول، الثاني) and any colon
• No heading sits in a table, text box, header or footer
• The outline check prints 0 finding(s), or you've decided each remaining finding
• The contents were updated after the last edit, and the page-numbering note is in the front matter

Errors to watch for

• Running heads styled as headings. If the book's running head repeats the chapter title on every page and the conversion kept it in the body, you get the same heading dozens of times. Delete running heads before styling; the outline check's output makes repeats easy to spot.
• One title split into two headings. A long title wrapped over two printed lines may arrive as two paragraphs. Join them, or the contents show two half-titles.
• Misread ordinals. OCR errors in titles (الثانى/الثاني, a dropped dot) go straight into the contents. Proofread titles against the scan as a separate pass.
• Numbering typed into the title. If the print numbers sections (١-٢-٣), keep the numbers as typed text unless you plan to link Word's multilevel numbering to the heading styles. Don't mix the two.

Limitations

• The script reads paragraph and style properties only. It doesn't look inside text boxes, headers, footers or footnotes, and it doesn't follow every way a heading level can be inherited (for example through linked list styles).
• Its keyword and bold checks are heuristics. They miss headings that don't start with a division word and aren't fully bold, and they flag bold lines that aren't headings.
• We tested it on hand-built files, not on a real converted book or on a ScribeTools export, and didn't test the Word steps in this session. The Word steps follow Microsoft's pages for Word for Windows; Word for Mac and Word for the web differ in places.
• An outline built from Heading styles carries into some other formats and tools, but how each one treats it varies. Check the target before you promise a client navigation in the final format.

If you're still converting the book, start with digitizing Arabic books or Arabic PDF to Word, then come back to the outline once the text has been proofread.