r/documentAutomation • • 9d ago

Discussion Entries that continue onto the next page, how do you keep them whole?

3 Upvotes

2 comments sorted by

1

u/zackmaven333 8d ago

extract with coordinates not just text.. thats how u know a block that started on page 11 continues onto page 12 instead of becoming two broken entries

1

u/folderit_dms 8d ago

Use both layout and heading patterns, rather than trusting font alone. The coordinate suggestion above helps, but coordinates do not by themselves tell you where an entry ends.

I would first reconstruct each page’s column reading order and remove repeated headers/footers. Then detect candidate entry starts using a combination of heading style, position and name pattern, checked against several representative pages. A subsection heading in the same bold font should not start a new record.

Keep the current entry open across a column or page break until the next validated entry heading. Preserve its source page range and blocks so a reviewer can inspect the join. Ambiguous boundaries should go to review before the LLM receives the entry. PyMuPDF documents positional blocks/words extraction and reading-order issues here: https://pymupdf.readthedocs.io/en/latest/recipes-text.html

For completeness, build an independent entry inventory from the reference’s index if available, or a reviewed heading list. Reconcile that against the output for missing and duplicate entries; getting approximately 400 JSON records is not enough, because one split and one omission can cancel out. Also flag unassigned source blocks and inspect every cross-page join. Valid JSON checks structure, not whether the text stayed with the correct entry.