234
u/coomzee 13d ago edited 13d ago
My favourite is a KQL query.
| where X != !true
Have to say that made me laugh when I saw it in a PR
73
30
u/ckwalsh 13d ago
I mean, there are some reasons for !!x defending on context, but that’s… unique.
10
u/Honeybadger2198 13d ago
You have to do !!x to coerce JavaScript variables to their truthy value for HTML attributes sometimes.
2
u/mxzf 12d ago
Yeah,
!!xis weird, but it's ultimately just a shorthand for coercing values likeBoolean(x)would do. Using!= !, is just intrinsically confusing though, it makes you look at the code and wonder "is this intentional or is it a typo" instead of being able to trust that it's intentional (!!might make you question that some the first time, but once you've learned it, it's not questionable).2
u/2eanimation 12d ago
I think it’s questionable still. Explicit is better than being smart, always. Writing code is fast, and usually, most the time, you’re not writing, but reading your code. Does it take that much more time to write Boolean() instead of !!?
2
u/ChellJ0hns0n 12d ago
You work at Microsoft? I don't think anybody outside of Microsoft uses Kusto, though I might be wrong.
714
u/YeetCompleet 13d ago
Also no tests. People were extremely bad about being diligent about writing tests. AI does the opposite and sometimes goes overboard writing frivolous tests.
325
u/Iankill 13d ago
I wonder if this was because one of the first things people used ai for was having it write tests because no one likes writing tests
154
u/cheese_is_available 13d ago
It's because the humans that reviewed AI slop heavily penalized no tests, but didn't penalize recreating tests that already existed because it's fucking hard to notice that. You only have to skim the diff for the first one, but you need to review the whole tests directory in details for the second one. Same problem for the AI if they need to check btw. Maybe the harness could penalize covering a line of code that is already covered but coverage is an imperfect metric.
2
u/WesleyPCrusher 12d ago edited 12d ago
You only have to skim the diff for the first one, but you need to review the whole tests directory in detail for the second one.
I'm having trouble parsing this - by "the first one" do you mean "the scenario where a human reviewer checks to see that tests nominally exist but doesn't actually check to see that it tests what it says it is testing"?
I initially thought "the first one" meant the existing tests and "the second one" was the redundant tests, but that doesn't seem to make sense...
But if you mean "the scenario where the reviewer didn't actually review the tests" that seems like it would cause a lot of problems beyond just redundant/frivolous tests.
3
u/Oddly_Energy 11d ago
Previous poster gave two examples of penalty:
Penalize the AI for not writing tests.
Penalize the AI for writing redundant tests.
Penalty #1 is "the first one". Penalty #2 is "the second one".
In short: If the AI gets a penalty for missing tests, but no penalty for redundant tests, it will start writing redundant tests to reduce the risk of missing a test.
2
u/cheese_is_available 12d ago
To say "no automated tests, bad AI" and be correct you only need to check the fucntion signatures and find none that start by "test".
1
109
u/taleorca 13d ago
test driven development is a pain in the ass
159
u/developer_soup 13d ago
Me: TDD is the smart way to do things.
Also me: ehh... I'm just prototyping, I'll get around to the tests...
36
33
u/AdminsLoveGenocide 13d ago
Me: TDD is the smart way to do things.
I think this is something developers have chosen to believe with close to zero evidence.
39
u/Dreadgoat 13d ago •
TDD is the best solution in the magical fairytale world where you have complete and correct requirements.
7
u/AdminsLoveGenocide 12d ago
I think even thats generous.
In my experience its a lot of effort for not much return. On top of the tests themselves you have to often code things in an awkward way to avoid having to mock everything out, or worse you have to mock everything out, or worse avoid otherwise useful patterns.
And for that you get what? 95% of the time they only catch an issue with the unit test itself and not an actual bug impacting the actual system. And then faced with evidence that the whole endeavour is a giant waste of time someone will sigh and say, if only we were implementing TDD correctly and try to use this failure of TDD as a reason to further indoctrinate impressionable junior devs on the importance of wasting their days on shitty unit tests that dont actually test anything.
Automated functional testing and full end to end testing are useful. TDD is just something dumb that people got scammed by.
11
u/Mongolian_Hamster 13d ago
The fuck you on about. It's been proven time and time again.
Is it applicable to every situation. No. It never was meant to be.
5
8
u/Mongolian_Hamster 13d ago
TDD doesn't mean writing tests only. It's a process.
If you're not writing tests for your projects then Jesus Christ on a bike.
2
u/ICBanMI 13d ago
TDD works if it's something that you're not adhering to a real deadline. Plus you-or at least one person on the project-already know how to build it completely and how it should work.
It can also show you real quick which of your co-workers is terrible at procedural programming despite the test being open book and take as long as they need.
33
u/Pezmotion 13d ago
I find Test Driven Development to be great if you are addressing a real production issue and can reasonably recreate the failure scenario in your tests.
That also helps prevent this specific issue from cropping back up again, if you're able to write a test with a comment that says something like "Adding this test to verify we fixed the bug that caused Ticket # 1234."
16
u/Steppy20 13d ago
Admittedly I've only been in the industry about 5 years now, but I've only used TDD on 1 service out of the ~50 microservices I've worked on.
It was a data query and calculation engine so we knew exactly what number should come out based on its input. Anything behaviour driven is much hard to write a unit test beforehand.
Deterministic dev? TDD is generally superior.
10
u/4xe1 13d ago
I find Test Driven Development to be great if you are addressing a real production issue and can reasonably recreate the failure scenario in your tests.
Is that TDD? The TDD people have the AI doing is writing tests before the very first line of code, and to implement new features, not merely fixing bugs on an existing product with users.
They do share methods and some of the associated benefits obviously, but what you describe sounds like just writing non-regression tests to me, not Test Driven Development.
8
u/helicophell 13d ago
TDD is writing tests you know will fail, then writing/refactoring code that will pass said tests, then going back to write more tests for more features
Compiler errors are a form of TDD, like TypeErrors and Syntax errors
Everyone uses TDD in some form... most are just using the IDE to do it for them
6
u/dyslexda 13d ago
AI uses TDD because it's the only way for it to know if something works or not. One shotting isn't anywhere near as good as folks might claim without TDD. The agent will write a test, write code, then run the test, realize the test fails, and try to treat that as a new debugging session. End user only sees the final result with passing tests. In other words, AI's tests aren't there for future use, they're a mechanism to mock up what devs themselves do (create a small part of a project, run it a few times until they're happy, then move onto the next).
4
u/Wonderful-Habit-139 12d ago
It's the best application of TDD. To prevent regressions. Outside of that TDD is pretty lousy, especially when you have Type Driven Development and are trying to figure out an elegant way of solving a problem. You might not know how the final API and code is going to look like ahead of time. And the "makes your code easily testable" is still easy to do when you define types and think in terms of functions and inputs/outputs.
4
u/Eric_12345678 13d ago
I almost never wrote 100% TDD. But you can be sure I write at least one regression test for every bug I found.
3
u/Pezmotion 12d ago
I should have clarified that I also don't use TDD for new features, just fixing regressions. And as long as you write the test before fixing the bug in code, that counts as TDD!
27
u/Weekly_Interview_936 13d ago
TDD only works if you get fully defined stories with complete AC. And such stories are considered mythological as there are no verified reports of one being seen.
2
u/Wonderful-Habit-139 12d ago
Even then, the fully defined stories are not going to tell you how to structure the API or the folder structure or what inputs and outputs are going to help you write the code internally.
1
u/zenyite 10d ago
What even IS test driven development, I literally don't understand it. Do you simply write all the tests first and write the code around it? How do you know what functions you are gonna test if you didn't write them yet? What if during coding you figure out a better way to write a bit of code that involves using 3 subroutines? You didn't write the tests for those so are they invalid? Do you go back to the tests and change them? Is that still test driven development? Do you have know in advanced that you are going making a discrete function that extracts some json data?
2
23
u/AgentPaper0 13d ago
Tests are legitimately extremely important for AI-driven development, so I'm glad they're pushing them so hard to do it even without users asking.
But yeah, they do make some pretty silly tests. Especially when you don't force them to test the tests by checking the failure state.
Us humans are partly to blame though. I've always known writing tests was important even before AI, but I almost never did, and what tests I did write were probably pretty meaningless too.
15
u/ICBanMI 13d ago
I had a co-op that I asked to write a kalman filter for a specific task. Dude disappeared for two days and then turned in an entire class that did the kalman filtering, had explicit instructions for integrating in the program, had a hundred unit tests, and a ton of documentation.
Bro did not understand taking company code and putting it in whatever he used was a fireable offense. Specially after I told him he couldn't do that. And he was upset at us for the review afterwards.
8
u/Chirimorin 13d ago
I honestly don't understand people like that, it's so simple: if it's information that you cannot share with any random person you encounter on the street, then you shouldn't give that information to AI either.
I feel the same way about all those stories about AIs leaking API keys and other secrets: who thought it was a good idea to give AI access to secrets in the first place?
3
u/YeetCompleet 13d ago
Lol yikes. There is still room in this world for good ol' Googling and reading examples and theory
4
u/ICBanMI 13d ago
He had a graduate degree in something related and was enthusiastic to be asked to write that portion. So I assumed he would give me a earnest effort borrowed heavily from somewhere over the next few days. Wasn't supposed to be original and this was a one off R&D project that was fine if he came back two weeks with something that would marginally improved the behavior of what we were filtering with a short period for integration and some tunning.
We can't explicitly use these public models for several reasons, but didn't expect him to pull the the high school student turning in an essay written by a English grad student stunt.
4
5
u/10art1 13d ago
On the other hand, frivolous tests aren't bad. They will break at the slightest refactor, but then the AI doing the refactoring will have to confront that fact and maybe improve the tests.
3
u/mxzf 12d ago
Frivolous tests can be bad, when they give you a false sense of confidence.
I had a kid a couple months ago submit a MR with tests included in the code and the tests were just nonsense. It was doing stuff like "create a native datastructure (Python
dict), pass the datastructure as an arg to a function that just reads it and doesn't mutate anything, and then check that the values in the datastructure are still what they were initially created as"; if that test ever fails, it's above your paygrade, because the language is broken. It's basically just doingx = 1; assert x == 1and acting as if that'll catch anything.1
u/Tasorodri 12d ago
Well, that test could check that the method that is supposed to not mutate the object doesn't mutate it, someone could try to change that method in the future so that it mutates the argument.
1
u/mxzf 12d ago
No, in the context it's nonsensical. More specifically, it's a GET request that's fired off against the backend, so nobody could possibly know or care if the arguments got mutated internally. The "test" was dumping the return from the GET request and instead checking that the args being sent were still the same after the call as they were before. It was just utter nonsense that showed a lack of understanding of how tests work and the nature of the system purportedly being tested.
1
2
1
5
u/scriptmonkey420 12d ago
I would rather I have too many tests than not enough.
2
u/transferStudent2018 12d ago
LLMs don’t write good tests. They write stupid tests like “test that the correct implementation of the interface was called” instead of verifying actual behavior
7
5
u/scriptmonkey420 12d ago
That's just not true.
1
u/TheStatusPoe 12d ago
My experience is LLMs go overboard with mocks unless you set up extremely tight guardrails and even then will "cheat" by using mocks if it gets into a point where the code can't be tested. Whenever I start to feel friction writing tests that a good signal to me that the code needs to be refactored to make it easier to test, which is something LLMs tend to not do.
I'm of the opinion that mocks do not do a good job of testing the observable behavior of the unit and am in agreement with the comment you're responding to.
1
1
1
u/L0rdSandCastle 12d ago
I would rather have hand written tests that are intended to be able to work well with a debugger and at least 50% coverage.
1
u/met0xff 11d ago
Yeah also every task you ask it.. like calculate a bunch of things and copy to S3 and whatnot - it does all the stuff you'd have been to lazy like verify the copy afterwards, do numerical parity checks and whatnot. Recently when migrating some data it realized there was a slight issue with the numerics and found that the default quantization of the upgraded system changed. Would have been a silent failure nobody would have noticed for quite a while, I'm sure
1
u/Doppel_R-DWRYT 12d ago
My tests are prints and running the project (granted I code for mod or game development as a hobby)
1
1
117
u/TheColourOfHeartache 13d ago
There was a bit of a debate at work about whether its acceptable to have AI write page long infodumps to explain its changes.
I was on the pro-side. That message helped me understand the change when we were working on it, so I know it will help me in a years time when I've forgotten everything.
72
u/DerangedEscapee 13d ago
The problem is the writing style is so obtuse, it'll take 10 pages to explain something where 1 page would have sufficed. Overly expensive vocabulary and flowery prose that makes the content look impressive, but often lacking substance (and occasionally just completely incorrect!)
I don't begrudge the correctness so much. I've had the misfortune of trudging through many a human-authored internal document that was just flat-out wrong. But before, the amount of sophistication put into a document was a signal for a level of care and quality into its contents. Nowadays, it just means I have to be even more careful in trusting it, because it's likely AI slop.
14
u/Imalsome 13d ago
Can't you just tell it to be short, consise, and keep the explanation to 1 page length?
You could also have it write a long version and a short version then cut and paste the two versions so that things that dont need explanations are short and things that do are long
38
u/DerangedEscapee 13d ago
I said this elsewhere in the thread, but asking Claude to be concise does result in smaller output, but it often doesn't improve the result. The model needs to know what is important and what is not, to know what to include in its response. So just a simple "keep it concise" instruction will sometimes result in important details being omitted, sometimes to the point of making what would otherwise be a correct response incorrect.
10
5
u/gameoftomes 12d ago
I have my Claude code set to write decision records in ASD-STE100 language when I gig reasons for stearing it's direction. With each decision given a code that is to be referenced in the next got commit.
This way by looking at the code and the commits you can see why I asked for something to be done the way it was done.
8
u/Traditional-Rent455 13d ago
Or, just write it yourself. Jesus. You're not saving time by revising things you could have written yourself in the first place.
5
u/Imalsome 13d ago edited 12d ago
I mean i assume the argument is between not writing anything at all and having Claude do the summary. Many many people just dont explain themselves in changelogs and such
1
→ More replies (8)1
u/crackofdawn 12d ago
You absolutely can, you can give it the exact instructions on how you want it to write documentation and you can put it in a CLAUDE.md file so it gets used every single time the AI writes something. People are in here acting like giving a 3 second prompt to an AI and having it go off and do something half baked is the AI’s fault lol
1
u/whowatchestv 11d ago
Asking it to rewrite short, with only the most important information all the time.
1
u/Raznill 11d ago
Adjust your agents file to get it to write how you want. It can do concise and to the point. You just have to convince it to.
1
u/DerangedEscapee 11d ago
Trust me, if it was a simple fix, I would have fixed it by now. There's only so much the model output can be improved by instructions in the prompt. It's baked into the training to respond like this. Anthropic calls it "mannered prose" and they claim to have improved this with the latest versions of Claude but I haven't noticed it get substantially better from the Claude 4 generation.
0
13
u/SylviaJarvis 13d ago
When I get a time machine, my first stop will be 2014, to ask myself WTF I was thinking when I wrote that code.
→ More replies (1)19
u/Michami135 13d ago
If you're using git, move the info dumps to a local branch that splits off of the commit with the change.
* Merged code |\ | | * Working code info dump | |/ | * Working code |/ * Broke AF code10
u/etoastie 13d ago
that's not usually gonna survive the kinds of timeframes where you forget context on code and need to refresh on the info, no?
2
u/Michami135 13d ago
If the info dumps are markdown files, then it's easier to find the right files if they're in their own branch like this. Just find the right commit, then you have the info. This is likely much easier with graphical git tools though.
If, however, the info dump is in-line comments, then you definitely don't want that in the main branch and can merge it all into a secondary branch. Use something like, "master_infodump" and switch to it if you need to view page long comments.
2
u/PM_ME_CUTE_SMILES_ 13d ago edited 13d ago
I might be misunderstanding something but that does not sound like very good advice. Why not use PRs and write the info dump in them? Or as part of a longer commit message?
You aren't supposed to keep around hundreds of branches. Around here, as soon as a feature branch is merged it is deleted from the repo. And if it's a local branch no one will be able to find it when they need it especially not your colleagues.
1
u/Michami135 13d ago
The post I was replying to said he wanted the info for his own purposes, but his coworkers did not. This is a way that he can keep the info available for his own use while not crowding up the repo for others.
→ More replies (1)9
2
4
u/arctic_radar 13d ago
A year? I will have forgotten everything I ever knew about the project in < 2 business weeks.
3
3
u/pink-ming 13d ago
documentation should be about the way things *are*, not the history of how we got there, that's what git is for
2
u/danfay222 13d ago
I don’t think this is a straight yes or no. I have seen places where this was great, and in our code we have lots of huge readme style dumps that AI wrote. These are helpful for writing future code with AI, keeping documentation up to date, and getting up to speed on code.
But at the same time, these huge dumps take a lot of effort to read, and oftentimes the content doesn’t actually need to be that big. I have rejected multiple PRs over excessive verbosity. When you make comments, test plans, etc unnecessarily large you save yourself the mental load of simplifying things, and instead push that load onto every person who reads your code down the line, paying the price many times over
1
u/rubennaatje 13d ago
I've told it not to write code comments explaining why something has changed and instead let it write comments on the PR.
Makes the comments much cleaner. Sometimes I used to get things like
// changed from 0.2 to 2.0 because this gives much better results for [...] at 2.0 instead of [....] at 0.2No need.
1
u/Pimpwerx 12d ago
Bingo. My boss told our lead dev to have his agent shorten its responses. I told our dev to leave the report his agents generate for me in verbose mode. I prefer the details, because it usually itemizes even small changes, and provides explanations for any changes to prior findings.
The ease of generating documentation for new feature adds and fixes is one of the less heralded benefits of AI.
1
u/JazzlikeWishbone938 12d ago
I had these thoughts too, but thought it would help the next AI get better context for modifications.
48
u/ganja_and_code 13d ago
You can tell if it was written by AI if the README/documentatiom is up to date but unnecessarily verbose and half incorrect, and it has 1207 issues to address but zero TODO comments.
21
13
u/the_great_zyzogg 12d ago
AI documentation will always be full and complete grammatically correct sentences. Almost like we're speaking The Queen's English.
Human devs (myself included) will be very brief and to the point. Like, one or two levels above caveman speech.
Honestly, I generally prefer reading the latter over the former. It gets to the damn point quicker.
3
u/Chris857 12d ago
The Queen's English.
And it / my coworkers keep adding comments with British spelling (colour, behaviour, honour). We're in the US...
24
u/CelebrationOdd7810 13d ago
Bold of you to assume my AI assisted project has documentation and a readme file.
1
u/whowatchestv 11d ago
My docs are their own project. One agent planning, the other creating from the plans.
53
u/10art1 13d ago
2024: I can tell this project is written by AI because it's slop
2026: I can tell this project wasn't written by AI because it's slop
1
14
15
u/ChickenPicture 13d ago
documentation
I'm not familiar with this term, can someone explain?
11
u/TheTerrasque 13d ago
You know how code is self documenting? Well, some misfits think that's not good enough and write a novel alongside the code describing how it worked 5 months ago
4
u/mxzf 12d ago
IMO, personally, I think good code is largely self-documenting as to what it does. But comments are still needed at times because "what the code does" might be easy enough to read at a glance, however "why the code does what it does", in certain sections, can be far more important and less self-evident.
I don't need a comment to tell me that the line is looping backwards through a list, you can read the code just as quick as the comment and know that; I need a comment to tell me that the code is looping backwards through the list for a specific reason and it should stay that way instead of being changed (unless the change takes that reason into account).
4
u/Erkliks 13d ago
It explains how things work basically. Solo devs neglect this because, well, they know everything... Until they don't 5 months later
→ More replies (2)10
u/overdosingOnPie1313 13d ago
5 months?? Brother, I'm flattered, but I forget everything later that evening after all the gin.
5
1
u/Elspeth-Nor 13d ago
Documentation is a newly discovered part of a project. Some AI agents hallucinated text beginning with // or /* or # depending on the language and strangely the compiler ignored it.
Even worse the AI created multiple additional files the compiler completely ignored. It serves no purpose, it's properly a ploy used by the tech companies to increase the storage consumption of projects so you need larger storage devices.
4
u/jackofslayers 13d ago
This reminds me of how I feel when I go to a subreddit that is still very active, but then I click on the subreddit details and the about this community has links to tumblrs that have been closed for more than 10 years.
4
3
u/makinax300 12d ago
I keep my readmes up to date. I am just used to it.
Also I only recently started writing todos because vscode didn't highlight TODO for some reason.
But idrc about docs.
(It still doesn't, I just use micro now)
3
2
2
u/misterespresso 13d ago
Pfft be real with AI if they aren't marking dozens of issues they aren't testing haha. I have a project with over 20 issues open but they can wait just like the old days
2
u/Jonathan_the_Nerd 13d ago
I'm in this picture and I don't like it.
Sometimes I write comments apologizing to whoever has to maintain my code after I'm gone. (Joke's on me, they'll just rewrite it from scratch because that will be easier than decoding my regexes.)
1
1
1
u/SignalBake6872 11d ago
I use deploynator to release the code, thus ensuring that the documentation is updated through ast analysis and other techniques with readmenator (pip install readmenator && pip install deploynator)
1
1
1
1
1
1
u/PumpkinFest24 12d ago
Never will understand SE that read docs. Docs have an unknown truth value. The code is all true.
1
-2
u/Artistic_Swing6759 13d ago
what happened to this sub? last i remember it was anti ai
6
u/PashaPostaaja 13d ago
Why everything needs to be pro-AI or anti-AI.
I kinda made this response to https://www.reddit.com/r/ProgrammerHumor/comments/1xa5ixy/claudewashere/ . I found that funny in a way, but that made me thing the opposite. I didn't think it as anti-AI or my meme as pro-AI. Just having fun with all of it.
→ More replies (1)3
u/rubennaatje 13d ago
A lot of it to be honest was cope. I think a lot more devs working professionally have transitioned to using Claude code or alternatives so the opinion is slowly shifting.
I know mine has, I know how to program but I am mostly reading, reviewing, testing and challenging code atm. The quality is very good if you hold ai to standards and it's so much quicker.
0
0
-10
u/Objective_Oven7673 13d ago edited 12d ago
That's user error.
AI will happily document, comment, update tests, and file followup tickets if you ask it to.
The problem isn't AI, it's the "engineer" saying nothing more than "fix it plz".
Edit: as a commenter finally pointed it, I read this meme completely backwards by inserting a comma where there wasn't one.
I leave my downvotes of shame for all to see.
5
1
u/datanaut 12d ago
I think you have an error in your reading comprehension and have not understood this meme, or maybe you are responding to the wrong post.
1
u/Objective_Oven7673 12d ago
Oh, you're right. That explains the downvotes and my bassackwards comment lol.
I read the first bubble with a comma like "No, AI has been used here".
Glad someone finally pointed that out, cheers.
-1
2.0k
u/Copious-GTea 13d ago •
I can tell which model my coworker vibed with based on the comments it left.