Does any OCR preserve Arabic diacritics (tashkeel)?

Why tashkeel is worth this much attention

In classical Arabic text, vocalization is not decoration — it fixes meaning. A single mark distinguishes the subject from the object of a verse, separates homographs, and disambiguates transmitted readings in hadith. A scholar who receives OCR output stripped of its tashkeel has not lost polish; they have lost part of the editorial work someone already did, and will now redo it by hand, letter by letter. That is why “does it keep the diacritics?” is the first question researchers ask of any OCR tool — and the first thing most tools fail.

Why diacritics disappear in most OCR

Mainstream OCR engines were trained mostly on Latin scripts or on unvocalized Arabic — newspapers, modern books, administrative documents. Tashkeel marks (fatha, damma, kasra, sukun, shadda) are small strokes above and below the letters, so engines classify them as speckle and delete them, or misread them as independent characters and corrupt the word.

The failure shows up in three common forms:

• Silent stripping: the text comes out complete but with every diacritic gone — often discovered only after a whole dīwān has been converted.
• Corruption: a shadda becomes a letter, or a mark attaches to the wrong letter, changing the meaning.
• Broken encoding: the text looks right on screen but collapses when pasted into Word or searched, because the marks were stored in the wrong Unicode order.

For quranic commentary, hadith, classical poetry, and critical editions, the diacritics are not decoration — they are the ḍabṭ (vocalization) a scholar’s reader depends on. Losing them means redoing that work by hand.

Test any tool in two minutes

1. Take one page of fully vocalized text — a poem, a page of muṣḥaf-style print, or a critical edition.
2. Run it through the tool, then paste the output into Word.
3. Check three things: are the marks present? Are they on the correct letters? And does search find a word when you type it without diacritics?

A tool that passes all three is safe for your book. Anything less, and you will pay the difference in manual review.

How the usual free tools do

Google Docs OCR handles clean, unvocalized modern print well, but vocalized text usually comes out stripped or with fragmentary marks. Tesseract can in principle be trained on vocalized Arabic, but its stock Arabic models do not preserve tashkeel dependably. These tools are excellent in their lane — vocalized classical text is not their lane.

What if the book is only partially vocalized?

Most critical editions vocalize only where ambiguity demands it: a Quranic quotation, a verse of poetry, a proper name that could be misread. This case is more treacherous than fully vocalized text, because stripped marks do not jump out during a quick review — and the one vocalized word on a page is usually the word the editor marked deliberately to prevent a misreading. When you evaluate a tool on such a book, hunt down the vocalized passages specifically and check them one by one rather than trusting a general impression of the page.

How ScribeTools handles tashkeel

ScribeTools is built for vocalized classical text from the ground up: each page runs through multiple OCR engines and vision LLMs together, the best result is selected, and the output is emitted as correctly ordered Unicode — so the marks survive export to Word, and search works both with and without diacritics. Footnotes stay out of the main text, and Arabic-Indic numerals and hijri dates are left untouched.

One honest caveat: dense vocalization is the hardest terrain in all of Arabic OCR, so review the output before your name goes on it, whatever tool you use. You can test your own vocalized pages free — 20 credits monthly, no card. Start with your hardest page, not your easiest.