evidoria

← Back to browse

Good practice Imported

NDLOCR — Japan's National Diet Library Doubles Historical-Text Recognition Accuracy With AI

Japan · Tokyo · See the Japan profile · See the Tokyo profile

Evidence: Observational / pre–post Top 14% 80/100 · Ask Evidence Copilot about this practice

Japan's National Diet Library open-sourced an AI OCR engine that lifted recognition accuracy on pre-1945 texts from about 40% to over 90%, then used it to convert 2.47 million digitized books and periodicals — 223 million page images — into searchable text.

0.9686
Mean F-score across 33 material categories
~40% to >90%
Pre-1945 text recognition accuracy
2.47 million items / 223 million page images
Digitized items converted to text, FY2021

Details

Maturity
Established
Promoter
National Diet Library, with Morpho AI Solutions, Toppan Inc. and LINE Corporation
Period
2021–2022 (NDLOCR development); FY2021 conversion of 2.47 million items
Keywords
libraries & archives, digital government, public administration

Context

Nearly half of the National Diet Library's digitized holdings predate 1945, and commercial Japanese OCR performed poorly on their historical kanji, mixed layouts and phonetic scripts.

Activities

NDL commissioned Morpho AI Solutions, with Toppan Inc. on training data, to build NDLOCR, a deep-learning OCR pipeline trained on roughly 13 million characters, released open-source in April 2022.

Results

NDL Lab's evaluation found a mean F-score of 0.9686 against an explicit 0.86 criterion, exceeding it in 32 of 33 material-type categories. The library then contracted LINE Corporation to apply the pipeline at scale, converting about 2.47 million digitized items (223 million page images) to text in FY2021, lifting pre-1945 recognition accuracy from about 40% to over 90%.

Conclusions

Gains are real but bounded: one material category underperformed its target and the headline figures come from NDL's own internal evaluation, not an independent external audit.

Implementation
Indicative cost
Medium (€50k–€500k)
Time to results
Medium (1–3 years)
Staffing & skills
Morpho AI Solutions (model development), Toppan Inc. (training-data construction), LINE Corporation (scaled application)

Conditions for success

  • large annotated training corpus (~13M characters)
  • explicit accuracy criteria per material type

Common failure modes

  • one of 33 categories (1970s periodicals) fell short of the accuracy criterion

Commonly funded by

National / regional programmes

Indicative funding routes for practices of this type — always check each programme's current calls and eligibility rules.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Similar practices you may find useful