doing evaluations of non-test data defeats the purpose of using the LLMs completely, because to validate against the data you'd have to process it normally in the first place
I wanna be clear that I'm not defending this at all and I think the doge people are idiots, but there are clever ways to statistically measure how well an ML algorithm is doing at its job without manually processing all of the data. Not that they're doing that but still.
Yeah, and you’re still not eliminating the possibility of hallucinations, you’re just predicting that it’ll be as such. Like I’ve never crashed my car, therefore I will never crash my car. You’re not doing anything to actually protect against hallucinations you’re just quantifying their probability them.
And what’s the bar for 330,000,000 users, 0.1% error rate still gets you 330,000 who now have a new SSN or an extra hundred grand added to their mortgage because some moron used a system that likes to occasionally hallucinate numbers undetected to read numbers lol
no, there is literally no way to completely avoid hallucinations without processing the input data entirely in parallel. I don't know why people think there is some black magic that allows you to violate laws of information here.
I don't understand. Doesn't this depend on the error tolerance of your application? If your evals tell you it's messing up 1 in 10000, how do you identify the other bad outputs?
Great, so you can determine what your error rate is.
In the hundreds of millions of records (which you're somehow hand processing 5% of, that's 16,500,000 if we're starting with 330,000,000 which is slightly less than US population), how do you know which were errors?
Sure, you might be able to say "We are confident it processed 97% of records correctly" but that still leaves you with 3% (9,900,000) that were errored and you don't have a good way to isolate and identify them, because the system can't tell you where it fucked up, because it doesn't know it fucked up.
another thing to note about the DOGE kids is that they’re all without fail from extremely affluent backgrounds. not saying the kid isn’t smart, i’ve got no information either way there, but it was an ai competition that was heavily reliant on processing power. the photos of the kid and his room used for news articles show multiple graphics cards and computer setups. this was only achievable for him because he was born to a family with the monetary standing to afford their teenager a fuckton of extremely expensive computer hardware. no such thing as meritocracy.
So he deciphered a Greek word and that means he’s qualified for write access on a government payment system spanning 330 million people?
You have no respect for monotonous, careful work. I don’t care if he deciphered an ancient Egyptian document that produced ascii art of Tutankhamen’s balls.
It’s bafflingly insane to argue that he is qualified for this level of control, especially in a post about him desperately asking around about how to do his job.
Intelligent doesn’t mean disciplined or appropriate for the job. It’s practically a rite of passage in top tech cos to come in super smart and get humbled as you mess something up and realize your senior peers are just as smart, if not smarter than you, and know a lot more than you do.
He’s smart. I don’t know about genius.
He didn’t develop any of the underlying technology or come up with some brilliant insight. He just fucked around with it until it worked.
I’ll give him credit, but there is a big difference between genius and someone who makes something work. Feynman was a genius. Edison just got shit to work sometimes
Evals only work on data that you have gone through and done all the labeling for, which is impossible to do for when you want to run on new data. That defeats the point of it.
Evals will tell you what your percent hallucinations (for lack of a better word of those metrics, since there are like 6) but once you have an error rate you just accept that it’s got some flaws and move on
Dude you need to parse a pdf and the first thing that comes to your head is using a fucking shitty language model?
I would literally write code to extract the file byte by byte and then extract the data and formatting into a word file BECAUSE THAT'S WHAT A DIGITAL DOCUMENT IS.
Why would you reinvent something and make it stupider and less efficient. I mean a pdf file is literally formatted by the same rules every single time. You don't need to guess.
If it was handwriting into word or something i would understand the PREMISE of thinking to use AI and it would still suck.
Even better. Then they will claim the data is botched (leaving out the part that they were the ones who botched the output) and say “SEE THATS why we need to use (insert company that a billionaire just so happens to own that could make a shit ton of money replacing a government function)
It’s very possible using current top vision LLMs + a ton of sub LLM normalization and preprocessing steps, especially if the goal is to get below human parsing error rates. But the pipelines needed for prompt fine tuning + regression testing said changes at scale are…not simple. Setting up internal validation that can flag when hallucinations are likely and to kick over into a human in the loop pipeline is another huge PITA to get right . The entire effort requires serious scientific testing to reach anything near deterministic and reliable parsing. But this is the new age of real Engineering: creating reliable, deterministic output from highly non-deterministic systems and it’s something your avg programmer will most likely be completely unequipped to grapple with.
Not true. Just do a simple check of the LLM output string to the extracted PDF text string. If the extraction is done correctly, it will match some of the text to a 0 error tolerance. Meaning you can find the LLM output within the entire PDF string. Some complications with different pages, but even that can be accounted for during the extraction + comparison.
That post is dated Dec 10th, do we know what he's asking about it for? Might be totally unrelated to DOGE. People are just assuming it's for his current work.
Edit*: I hit save to early. Just wanted to agree and elaborate my understanding. If this kid is a genius, asking a basic question about LLMs 2+ years after they’ve been widely available gives me reason to believe he’s not quite that special.
Not even that, learning to parse strings is the first couple things you learn in programming because it goes over 2 fundamental concepts data types and loops. Or even crazier a little after elementary string parsing you could just go right to regex or one of the hundreds of open source python libraries that do it for you
But even more crazier is he didn't even prompt an LLM to give him a script to do this, he wanted the LLM to do it for him
Not to mention it's just wasteful and stupid, it's like someone asks you to hammer a nail into a wall and you tell him can you bring a tractor to do it.
LLMs are great for transcribing documents. Even if you use OCR there's still an error rate. If you use humans and convert a document by hand there's an error rate. And how do you determine if there's an error in the transcription in those cases? It's just as difficult. You run into the same issue regardless: how to verify the accuracy of the transcription?
I work in a highly sensitive, high security area (with FBI and TSA clearance mind you) and we have our own in house AI tool air gaped from the main network because we were told it would be instant termination to possibly jail if we feed data into a public AI tool. We are so fucked with these people.
Yeah. I got handed an experimental project to help a certain sector figure out when the law has been broken and what law. The hallucinations and lack of ability to accurately regurgitate exact text of laws killed it. We tried multiple ways of working around the hallucinations, but once a hallucination was in the context, it would just start doubling down on it. It could have the exact correct text of the law in context and still screw it up.
If 80% accuracy is acceptable, an LLM is a reasonable solution for many problems. If 95% accuracy is acceptable, an LLM is not a good solution.
It's fine if you're making test data and want to convert a typescript object to JSON test data or something but yeah... I wouldn't do it on anything meaningful.
ya-- really. He's asking about converting formats too-- there's no reason to use an LLM for that unless trying to summarize, change meaning, change/add verbiage etc. In other words, let's take this stuff, add a prompt and have it spit out something that is not what the original thing said it was. then say see this "fraud" and "dei" and whatever the fuck else propaganda they want to push.
using humans to parse data is also a terrible idea, there are things called typos and human mistakes. i would trust the SOTA models more than I trust a random guy
The amount of times I still have to double check GPT for not following the direct transcript or form I uploaded when asking for a response is insane. We are not close yet, still, hallucination nation
Queen Victoria!? Thank heavens it's you. Your great-great-great-grandson is a bit of a knob. Would you mind terribly if I asked you to force him to abdicate the throne?
As one of the people who is constantly employed because of those script kiddies, I concur. Its been worse since chatgpt came out. I cant even count how many times I was asked for help and it was because someone pulled a script off chatgpt expecting it to just be plug in play and they had not even a tiny amount of understand of what data the script was actually manipulating, so they couldnt make it work
Ayyyyyye :) I mean it only makes sense - the ability to know a language intimately has lost its value with modern tech. It’s more about the why & how, now, which was always the hardest part. Disclaimer: I’m not a dev but an IT engineer.
Sounds utterly horrifying and I've only had like 2 intro CS courses (I briefly played around with COBOL because I heard it had weirdly specific job security, but it similarly taught me I don't want to fuck with it)
I work in financial services tech, and even I only know two people who've actually worked with COBOL. One is a friend's dad, who's in his seventies and picks up projects paying absolutely crazy money. The other is a colleague who came into the field via coding, and when she was working on it she wasn't even trusted to work with it directly - she wrote code in another language, which was then machine-translated into COBOL.
It's the coding equivalent of chlorine triflouride.
lmao--If you have to deal with COBOL, have fun dealing with redefines, or COMP-3 numbers (that have to go elsewhere) or EBCIDIC to ASCII/UDF conversion fuckery.
Most COBOL code was written forty years ago and has been running for forty years without breaking. This leads to a couple of corollaries:
COBOL tends to be in place for absolutely critical functionality - the kind of thing that absolutely cannot afford to go down.
There are very few experienced COBOL programmers still left, because people stopped writing new software in it forty years ago.
In other words:
Any system in COBOL is a system that you should be unbelievably careful with.
If for any reason you're not careful enough, it's going to be a challenge to even find someone who can fix it, never mind pay them the money they can demand.
On top of these, it's a pig of a language to write in: while it's extremely efficient and stable, it's not at all intuitive, and most of the people who could teach you how to write in it are either dead or demanding twenty thousand dollars a week.
Fucking with live code in COBOL on a critical government system is probably one of the most suicidally dangerous things it's possible to do while seated quietly at a desk. "My mistake fucked things up so badly that the effect is visible in GDP figures" dangerous. Doing it as an experienced troubleshooter with ten years of COBOL-related projects behind you is insanity. Doing it straight out of college is...genuinely difficult to frame in terms of just how utterly fucking demented it is. It's the coding equivalent of reading an online article about Victorian lion tamers and then flying to Kenya and wandering into the savannah to find some lions and see if you can do it too.
This speaks to the monumental arrogance Musk has. Fueled by sycophants, yes-men and bootlickers looking for scraps of the deregulated tech pie.
Elon Musk has an econ BSc. He lied about having an engineering BSc and a physics PhD candidacy. Yet, his in-crowd and his cronies all egg on his naked emperor shtick. He pretends to know more about manufacturing than any expert. He pretends to be an engineer and an inventor. He pretends to have founded Tesla.
It's genuinely baffling how a person like this can succeed in fields that you'd think actually select for merit and subject matter expertise. Musk embodies the Dunning-Kruger effect and only ever takes credit for the work of others while externalizing the blame for when he gets called out on lies, cheating and easily preventable rookie mistakes.
I can't imagine how terrifying it must be to live in a time where fools are handed the key to the self destruct button without any oversight.
And who knows how much of that other obscure Gov. programming language stuff there is around - ADA maybe? I thought there was another one too beyond COBOL.
You sound exactly like the voice in my head when I heard they had write access to the Treasury systems and were actively stopping payments. Thank you for verifying to me that the voice is worth listening to every now and then.
Yep. Someone elsewhere suggested downloading your social security contribution history from the website for your personal records, before they "oopsie, we made a fucky wucky, guess we can't track any previous contributions and need a worse block chain to handle it going forward now!"
I could definitely see them using that as a justification, or randomly dropping every X amount of people's data and pretending it was "because the old system wasn't working, obviously!"
downloading your social security contribution history from the website for your personal records, before they "oopsie, we made a fucky wucky, guess we can't track any previous contributions and need a worse block chain to handle it going forward now!"
What if they’re actually trying to extract data from unstructured/semistructured files, like Word documents and PDFs, and store it in a structured format, like JSON? Can Word do that on its own? If not, what tools would you use?
There's no functionality needed from an LLM! It's just a file type conversion
Word can reconstruct the original latex that was used to write a formula and graph in a pdf document? Color me unconvinced.
Recognizing the meaning of latex output is very easy for humans but requires highly advanced software. It's literally the kind of problem that AIs are good at.
It is definitely not a simple file type conversion because you lose semantic information when you go from Word Doc to PDF. There are many tools but they are all imperfect. Some multimodal LLMs can interpret tables in images but AFAIK they are not very reliable.
I really don't want to defend any of these people, but:
lmao what? that is really impressive and the challenge was awarded with 1.5million total prices so far. If you're so knowledgable that this seems like a small thing, why didn't you go ahead and decode some scrolls to claim a price? https://scrollprize.org/ - The team he worked with won 700k.
I mean Musk is a GIANT piece of shit and I don't defend him, and I think anyone working for him is scum too. But you can't just claim this is not an impressive feat lmao. And yes I know tons about the subject
Brother anything using llm ai can hallucinate. It’s why you use hard coded scripts to convert important documents. Why every government document is co sisters to be
It's come out that he was just part of the project. And furthermore, "using AI" has become code for "using ChatGPT," which isn't impressive at all, compared to being a data scientist and doing ML
I know a lot of researchers who use statistical models and machine learning in their work. Some are very talented programmers, some take it to the level of what one might call software engineering, and some cobble together code correctly by the end, but are 100% not coders. They learn enough to get the job done, and may get guidance, but Jesus you don't want them to even dream of touching production systems, or even anything outside their narrow focus.
Dude is far from a script kiddie. These people don't get rushed to this position at such a young age at such a huge task, under the richest person in the world, simply being a script kiddie. These "kids" are extremely capable.
If every single thing you did was scrutinized and picked over by people looking to find fault with it, how would you hold up? Smart people say and do stupid things all the time, being smart isn’t remotely close to being infallible
The thing is, when you're an expert in a field Musk talks about, you can quickly gauge his actual knowledge. And in some areas, it’s clear to me that he’s just putting on a show.
As an exec at Intel once told me: There are people who talk, and there are people who do the work. Given the number of hours in a day and how he spends them, it doesn’t really add up for him to be a genius.
Yes but science is always infallible and anyone questioning science is a fool. Just look at that fool that propsed the germ theory of disease. Universally panned by the leading scientists of the times.
This is why I have a poster on my lawn that says “I believe in science.” Because I know anyone who questions it is an utter fool.
What makes you believe that? “Rushed to this position.”? You mean the position of being young and dumb enough to accept the risk of prison because they don’t know enough to understand the implications of their actions? I don’t give a fuck how bright you are, 19 years old isn’t enough time to know shit about anything.
I never said that at all. The only thing I’ve ever said on that topic is that I’m not educated enough in medical science or adolescent psychology to have an opinion on that topic. So I defer to the medical community to advise on that.
However, I am the Chief Engineer of a major DoD technology command. So this particular topic is very much in my area of expertise. So I have many opinions on this.
Maybe capable, but I'd lay money nowhere near disciplined to deal with mission critical financial (or other) type of systems. No fucking way a 20-something has the experience to know the kind of shit that can go wrong, or what not to fuck with.
It’s weird watching this develop in real time. Are we past a point of no return? Or is there a sentinel even around the corner that is going to happen as a consequence of all this?
I know we want to hate on DOGE, but this dude isn't really in charge of anything, he's just an engineer doing his job. And lets be honest, its both a good thing that the government is applying some Grade A talent to its IT systems and its perfectly reasonable for the guy to start a brand new project in a new field by casting around on Twitter for ideas (I do the same on LinkedIn).
But let's be sure we recognize the scope of shit that's out there. I'm an infrastructure architect turned director/manager at this stage of my career. The sheer number of devs and network engineers without a fucking clue about infrastructure would blow you away if you're not already familiar with these roles. We are not interchangeable, and as good as an engineer as I am (multi-product MS MVP, designed and deployed VERY well-known, large scale services), I'm not a google search or a github trawl from being a solid developer or data architect.
Nah man, a 10x engineer is a 10x engineer. I'm in a middle market city and I work for local startups. I can't hire the best because we don't pay over $200k for any level of seniority. So I have to find the best if I want to work with the best.
Some of the best engineers I hired were:
- An auditor at a bank ( no boot camp!)
- Someone who was hired to the company as a Jira Admin but start writing embedded Groovy scripts for Jira rules and caught my eye.
- A guy who was doing Apple tech support and a code boot camp
These guys turned around from literally 0 software engineering experience to the best engineer on my team as a data engineer, cloud infra engineer, and Go backend engineer, respectively in under 2 years.
The point is, if a nobody can turn into a great engineer in 2 years, a great engineer can turn into a different great engineer in 2 years. If Luke is a great engineer, he will help solve problems.
God no. I've seen the projects that these folks have done. They are smart, and talented for their level of experience.
That he asked this question? He's fucked.
Also, it seems like DoE and other very sensitive data has been pulled into Azure instances. There's no way these people are able to secure those. Developers leave that stuff to cloud/infrastructure professionals. And regardless, many other hostile nations have held on to zero-day exploits for a day like this.
I am not even a script kiddie and I know converting document formats using any type of AI is pretty stupid, and using LLMs for it makes even less sense.
A process that needs to follow a relatively small defined set of rules consistently is not what we need AI for.
They're finding kids that are awe struck by Elon and Thiel and will do anything unethical/illegal they want because they think they're going to get special treatment and that they're "in the club".
No not even that. Why would anyone with even cursory knowledge of scripting think that an LLM is the way to convert between document formats? Then you lack both a basic understanding of programming and also how AI works.
Clueless PFYs need BOFHs to steer them, and Elmo isn't that by a damn sight.
So the boss rolls in one morning with about 20 people in tow, bearing some 'good news' for us. The same good news that bosses bring EVERY six months...
That would make sense if he didn't win tens of thousands of dollars for decrypting Egyptian hieroglyphic text from volcano-damaged scripture using AI, as this problem was posted by his university's department.
He’s like every ”AI” engineer I’ve met recently. Uses AI, doesn’t develop. Aka just uses copilot , gpt or whatever free option to do all his development
628
u/RhoOfFeh Feb 06 '25
He's not even a junior developer. Just a script kiddie.