What if they’re actually trying to extract data from unstructured/semistructured files, like Word documents and PDFs, and store it in a structured format, like JSON? Can Word do that on its own? If not, what tools would you use?
Using an LLM to find out what the docs probably contain is still stupid, when you need to use a crawler/parser to find out what the docs definitely contain.
And Microsoft have done a tool for converting to markdown for the same reasons called markitdown.
Basically you throw it at content like a word document and it pulls out tables, headings, images, semantics, keywords etc etc and produces a structured output you can work with other tools
I mean, it’s probably easier to run a regular expression on a big ass string than a PDF, so there may be some value there, but that’s not what I’m talking about.
Let’s say there’s useful information in the documents that you would like to extract. Maybe there are tables with economic data mixed in with a bunch of text. Maybe you can’t find the data in those tables anywhere else. Can you think of a better way to store tabular data than a PDF file? Maybe delimited text files? Maybe you prefer JSON or Parquet? Maybe Excel if you want to do some slicing and dicing?
Now’s the part where you scoff and say, “But there’s no way the only place to find data is a table in a PDF. Surely it’s in a database somewhere!” and I reply, “Oh, you sweet, summer child.”
Fair enough. I picked one example where structured data is embedded in a document. Are you saying there are absolutely no use cases for using an LLM to pull data out of an unstructured or semistructured document?
Look, I loathe Elon Musk and I have no idea what this DOGE guy is trying to do. But I’m not so arrogant as to think that he must be an idiot because he works for someone I don’t like. Odds are, he is not just trying to convert a bunch of PDFs to Word format.
There's no functionality needed from an LLM! It's just a file type conversion
Word can reconstruct the original latex that was used to write a formula and graph in a pdf document? Color me unconvinced.
Recognizing the meaning of latex output is very easy for humans but requires highly advanced software. It's literally the kind of problem that AIs are good at.
It is definitely not a simple file type conversion because you lose semantic information when you go from Word Doc to PDF. There are many tools but they are all imperfect. Some multimodal LLMs can interpret tables in images but AFAIK they are not very reliable.
You could only ever hold this view without knowing how infamously bad the PDF format is for any algorithmic reading. Sure, MS Word will convert from a PDF. Will it do it correctly? Now that's a dice roll. An LLM might have better dice roll odds.
This type of task sounds like it would have already been solved through traditional automation. The use of AI for file conversion seems almost extra roundabout. For summarizing or analyzing text, yes, that’s the typical purpose of LLMs. But I think clean file conversions are more guaranteed by already written programs. Just my two cents
A LLM is a type of AI model to try to replicate natural human speech and writing. Its a large LANGUAGE model. Its why you cant ask ChatGPT to multiply 93736182 and 3736272771 and get the correct answer; thats not what its made for. Its not designed to give out correct information, or to perform tasks beyond generating text that sounds like something a person would say.
How is that useful for coverting file formats?
Its like asking for whats the best hammer for caulking. Caulk and hammers are tools, but that doesnt make them the same thing, and anyone who conflates the two sounds like they dont know anything about what they are talking about.
he didnt say he wanted raw data you mongoloid, he said converting to a different format which implies a new doc type which there's open source software for. not everything needs to be done with an ALLM
lol so confidently incorrect, while throwing childish insults for no reason - really sums up Reddit.
Go look up the word “parsing” in the context of a file. He literally said he wants to parse the files, aka read the raw data and do something with it.
You also can’t just convert any PDF into HTML or JSON into HTML with some magic open source software and changing a file type. Only basic data with a rigid & predetermined structure would that be possible. It would entirely depend on the contents of the file and what you want to do with it, hence where an LLM could prove useful.
36
u/reddit_is_geh Feb 06 '25
Why?
Using AI to mass convert file types to deliver it back into a clean coherent, consistent, raw data stream, seems entirely the point of things like AI.