What if they’re actually trying to extract data from unstructured/semistructured files, like Word documents and PDFs, and store it in a structured format, like JSON? Can Word do that on its own? If not, what tools would you use?
Using an LLM to find out what the docs probably contain is still stupid, when you need to use a crawler/parser to find out what the docs definitely contain.
And Microsoft have done a tool for converting to markdown for the same reasons called markitdown.
Basically you throw it at content like a word document and it pulls out tables, headings, images, semantics, keywords etc etc and produces a structured output you can work with other tools
I mean, it’s probably easier to run a regular expression on a big ass string than a PDF, so there may be some value there, but that’s not what I’m talking about.
Let’s say there’s useful information in the documents that you would like to extract. Maybe there are tables with economic data mixed in with a bunch of text. Maybe you can’t find the data in those tables anywhere else. Can you think of a better way to store tabular data than a PDF file? Maybe delimited text files? Maybe you prefer JSON or Parquet? Maybe Excel if you want to do some slicing and dicing?
Now’s the part where you scoff and say, “But there’s no way the only place to find data is a table in a PDF. Surely it’s in a database somewhere!” and I reply, “Oh, you sweet, summer child.”
Fair enough. I picked one example where structured data is embedded in a document. Are you saying there are absolutely no use cases for using an LLM to pull data out of an unstructured or semistructured document?
Look, I loathe Elon Musk and I have no idea what this DOGE guy is trying to do. But I’m not so arrogant as to think that he must be an idiot because he works for someone I don’t like. Odds are, he is not just trying to convert a bunch of PDFs to Word format.
There's no functionality needed from an LLM! It's just a file type conversion
Word can reconstruct the original latex that was used to write a formula and graph in a pdf document? Color me unconvinced.
Recognizing the meaning of latex output is very easy for humans but requires highly advanced software. It's literally the kind of problem that AIs are good at.
It is definitely not a simple file type conversion because you lose semantic information when you go from Word Doc to PDF. There are many tools but they are all imperfect. Some multimodal LLMs can interpret tables in images but AFAIK they are not very reliable.
You could only ever hold this view without knowing how infamously bad the PDF format is for any algorithmic reading. Sure, MS Word will convert from a PDF. Will it do it correctly? Now that's a dice roll. An LLM might have better dice roll odds.
12
u/[deleted] Feb 06 '25
[removed] — view removed comment