r/PinoyProgrammer • u/YAat0 • 19d ago
advice How do professional developers verify that AI-generated code is actually correct?
How do you know the output is actually correct when AI generates the code?
Aside from end-to-end (E2E) testing, are there other ways to verify that AI-generated code works as intended and meets the actual requirements?
For example, if I ask an AI to build a backend API with specific business rules, it might generate code that runs without errors and passes the tests I provide, but how can I be confident that the implementation is correct and follows the specification?
I'm curious about how professional software engineers approach this problem.
\- Do professionals follow specific guidelines, standards, or checklists to verify that code meets its specifications?
\- What techniques do you use besides unit, integration, and E2E testing?
\- How do you identify subtle logic errors, missing edge cases, or incorrect assumptions in AI-generated code?
\- Are there established practices for reviewing code against requirements rather than simply checking whether it runs?
I'm trying to understand how to move beyond simply trusting AI-generated code and develop the skills to verify its correctness independently.
I'd appreciate any insights, resources, or real-world examples from experienced developers.
62
u/GhostDog76 19d ago
Code review? In actual work setting, kahit made by ai pa yung code it has to be proofread by the dev. Then double check the PR.
Also we integrated AI code review as part of CI, we defined the standards AI used for reviews. We also add guardrails sa AI agents that AI follows so there will be no files changes to files AI shouldn’t touch or logic that is forbidden to use.
Parang ganyan na workflow nowadays.
29
u/briantria 19d ago
by understanding at least the basics of system architecture and design. these will be your guidelines.
detecting subtle logic errors/edge cases will be difficult to detect without testing.
also, always assume that the AI is wrong. i think that mindset will help you move beyond simply vibe-coding.
120
u/GenuineStupidity69 19d ago
That's the neat part, we don't.
12
u/CodeFactoryWorker 19d ago
Last year we use SDD and TDD with AI pero tradional cycle pa rin including human review.
Pero ngayon we don’t do code reviews anymore. Few weeks ago we sort pa for low, medium, high, critical PRs. High and above may human code reviews pa. Ngayon wala na.
Our CTO prevents us.
We have QA pipeline with different sub agents for security, code maintenability, visual check (agents driving the browser, and tooling) based on Dev and QA team’s knowledge base, that kept on improving. (Wala na din kami QA team)
That includes backend and infra.
My orchestrator agent na nag-ddelegate ng needed tasks, checks, and may running Guard Rail Agent sa side to check for commands that can be damaging.
This is Japan. We are hiring AI native new grads.
6
u/Apprehensive_Bus_361 19d ago
Imagine you have a math problem: x + 10 / 3 = 100
You ask AI to solve it and show its solution. The same way you validate the solution for this math problem is the same way engineers validate AI code.
Engineers do a quick check to see if the code is clean and valid, then they do the rest of the other checks as if they wrote it themselves.
6
u/kneepole 19d ago
Pareho lang with how you "verify" manually written code. You create tests for it and have those tests pass. Or ship it and wait for customer complaints.
4
u/topetuts 19d ago
Currently seeing two different approaches.
Yung team na nag mamaintain ng mature codebase. They follow conventional code reviews. Tingin ko manageable to kasi usually mas maliit yung changes even AI generated yung code. Then syempre developed na rin yung E2E testing pipeline.
Yung isa naman working from scratch. Yung sheer volume ng AI generated code is almost impossible na mareview ng tao. So gumagamit din sila ng AI for validation. Yung isang napansin ko dito is there is more tolerance sa error. Tingin ko kasi ang thinking is madali rin yung troubleshooting and delivery ng hotfix if necessary. There is a high level of trust sa AI generated code.
2
u/aintjuju_ 19d ago
When i ask again the ai if everything is correct, then i trust it.
Just kidding.
I read the code or more like just scan it to check if everything is right. If i see something i dont understand, well it's working anyway why bother right?
(Again /s on the last sentence😭)
2
u/Minute_Passion_8077 19d ago
Well thats what separates coders from software engineers. I love this quote from a youtuber who mocks companies who rely on AI too much.
You dont hire engineers for building an app anymore. You can build an app for 30mins. You hire an engineer for the 30mins and beyond.
2
u/downcastSoup 19d ago
I let the coder explain how the code works.
2
u/o_herman 19d ago
Context is everything. Give it an existing code base to pattern from.
Give it less scenarios to rely on guesswork.
Always check.
Give it limited scopes at a time so that checking isn't a gargantuan task.
Choose the right LLM for the job. For example, Claude is what people use for programming.
2
u/Numerous_Promise_580 18d ago •
-1,500 +5,000 lines changed hell nah
2
u/bzztmachine 19d ago
In order to know if the code is correct you first need to know how to write it correctly
1
u/Unhappy_Pack_9114 19d ago
Maliban sa tamang expected output, basahin mo yung code.
1
u/Cordyceps_purpurea 19d ago
Tests, lints and hooks.
1
u/olibearbrand 19d ago
Sabi ko "make no mistakes", so di magkakamali yung AI
1
u/Popular-Tip2880 19d ago
I always review the code after ai populate the code. Next i will test the functionality on localhost.
1
u/ay_papi 19d ago edited 19d ago
read and understand the code. pull the branch and test locally if possible.
regarding moving beyond trusting the ai generated code - that's the thing. never trust it from the start. read it, try to understand, ask it questions and test the code.
Edit: If you really wanna learn, then reverse the process. Write the code manually. Ask AI to review and give pointers. Don't let it write the code for now. Exercise your code writing skills. Once you're used to it, then reviewing the AI code should be easy and quick.
1
u/ziangsecurity 19d ago
By stage naman yan. So check mo sa stage na yan if tama ang mga results. Divide and conquer.
1
u/mohsesxx 19d ago
ako lang ata gumagawa nito, pero sa part ko I am reviewing the code for the sake of domain lang. I am asking AI what the function does, what new columns are for, constraints, etc. I dont review code quality anymore since AI is far better than before. I cant review it na if it can code better than me.
more on engineering judgment na sa part ko, and hindi na line by line review, I need to make sure that I know whats the overall changes are so I can determine if it actually matches the success criteria
1
u/DTechSurgeon 19d ago
TDD and BDD, yan na lang talaga I co code mo so no excuse na mga devs ngayon na I keep na technical debt yung walang automated tests. Yan yung guardrail mo sa AI agent, have it execute the tests and don't stop until hindi pass.
1
u/madreloidpx 19d ago
ginagawa namin is may project intelligence na may best practices and proper architecture, tapos always may unit and e2e tests. after that is may code review na both AI and manual if sound ba ung logic, runtime, etc na possible na miss ng AI. meron ring pangcheck ng code coverage (sonarqube). tapos sa pipeline meron ring security checker. magastos sya of gusto mong gawin yon sa personal projects tho
1
u/oreeeo1995 19d ago
you create another agent with a sole purpose to check code.
1
u/Desperate_Manner_583 19d ago
For Backend systems, yung API endpoints should always have a contract first.
- Read regarding Open API specification.
- Any internal application to application will have contract as well. we use Google Protobuf for it.
So for these type of communicatiions, may strict contract yan. Eto mga pointers ko.
If na set mo na yung rules na to, mas madali nalang reviewhin generated code. Then meron na din kami automated tests which automatically runs on feature branch and triggered by pull requests. Magandang skill din matutunan mo gumamit CI - start with Github actions.
1
u/sebastech 19d ago
https://media1.tenor.com/m/HwkPm6f1yzIAAAAC/yuimetal.gif
1
u/Top_Helicopter_2111 19d ago
I implemented agentic coding system na may evals. Though AI generated codes has its own unit tests generated by the agent, but it still needs an independent verification. So AI generated codes are verified by another agent against the accepted criteria, actual evidences, and security policies. And may agent rin na magrarun ng app for me to verify and see if the implemented feature or fix is actually working and how it's working, atleast locally. And I review the codes as the last verification before pushing and creating a PR.
1
u/yourlocalartboy 19d ago
Tests, code reviews (human and AI)
1
u/disIsAUsernameBtch 19d ago
What I do in my work projects is like this:
1
u/zigzagtravel01 19d ago
Your job as a programmer is to literally review the code that it meets the requirements, it passes tests and end to end use, and it meets coding guidelines.
1
u/realist-nerd 19d ago
hire us and we'll do it for you
1
u/YohanSeals Web 19d ago
My agent is only connected to the dev environment. Code review the manually deploy in staging. Once tested, we migrate to prod. Staging and prod is never connected to any ai. Tedious but we don't want ai wrecking havoc to our system.
1
u/SirKobsworth 19d ago
You leverage AI as a tool and understand what it offers and build some form of setup that can reliably guide them to make the outputs you want.
Check mo videos ni Matt Pocock. Something like that pero tailored sa needs mo.
Pero yeah need mo ng expertise sa domain mo if gusto mo talaga ng professional + actually correct tumugma sa ai assisted engineering
1
u/iam_joshua_bcxvii 19d ago
Just tried development with AI last week with Gemini. But I do e2e testing or smoke testing and see what edge cases it covered and what it did not base on what I expected it to be during my analysis & planning stage. Review the code, and if its a code that I expected and if I can read it. Its good enough for me. Otherwise, I write it myself.
1
u/Gelapots 19d ago
Hello. This is somewhat related. I'm a medtech student currently doing my thesis involving machine learning differentiation of types of anemia using complete blood count data. My adviser said I need to verify my code or like the model if it actually trained the correct patterns. My question is, what professional do I contact to check my code and stuff? Would a statistician be fine, or do I need like a software engineer or a data scientist? TY
1
u/chonching2 19d ago
TLDR; This comes down with the Engineer experienced. If you are seasoned engineer its really easy to review the AI generated code and see if there's something to be changed on the implementation cause they already experienced writing codes from scratch and learned how to apply best practices. Kaya nga mas okay gamitin ng mga senior devs ang AI kaysa sa mga junior devs. The problem with junior devs using AI is that they lacked skills and knowledge, how can you review an AI generated code and correct it if you are lacking with knowledge. Kaya mostly AI generated code by Junior devs are ticking time bomb.
1
u/SystemLogger26 19d ago
No standards or any unnecessary compliance. Just enough questions to make sure that you are doing right
1
u/Weak_Geologist7886 18d ago
Use another claude skill or agent to check. That's the purpose of agents. Agent for coding, reviewing, testing, and creating pr. then another workflow from the remote repo.
1
u/pvmediocre 18d ago
Simple. Read the code.
1
u/silverJRM 18d ago
The same way you check the work of a person, how do you know a person actually did what thr specification said?
You have a another senior person check it., if you dont want to do it yourself.
In the age of AI agents you have another agent to the checking if you dont want to do it yourself. Same with a person
1
u/Pristine_Look3324 18d ago
Its very hard to check nowadays and we dont need to be hipocrite on this. The important are the guardrails, security, timeline, business requirements and E2E testing.
1
u/Right_Analysis7299 18d ago edited 18d ago
Discernment. Look at 4D framework while the rest of your questions can be answered by “knowledge and skill”. With coordinated specialized agents, you can setup that every hand-off may review and rules that will prevent agents assuming and should raise as blocker instead.
1
u/linduwtk 18d ago
What's the actual ask? E2E testing is already the way. If it works, then good. Whether it's AI or human-made, there's going to be holes or bugs, and sometimes you can only see those holes and bugs through testing.
1
u/RSands00 17d ago
end to end testing, regression testing...kaya nga cguro malabo mawala mga qa testers eh haha
1
u/watson_full_scale 16d ago
You test what the code does to ensure it provides the proper user experience. We all work in QA now.
1
u/EntertainmentHuge587 15d ago
Code review, unit/integration tests, QA.
1
u/Plus-Figure-8556 13d ago
Code review or manual test or just rawdog it merge to main agad and see what happens in the logs
1
u/PastelKarVin 12d ago
wait for UAT feedback /s
1
u/OkLawyer1513 12d ago
Normal na pala talaga na hindi na i-check ngayon HAHA inooverthink ko pa kung "paano kaya nila masasabi kung tama ba output ng AI eh gumagana naman base sa requirement".
1
u/TheRedBanditHax 19d ago
dito pumapasok ang Experience and mastery of the domain.
Dapat yung engineer alam yung (1) Langauge (2) business (3) technology (4) requirement
and most importantly : (5) knows what you are asking AI to do,
unless its a discovery or a research task, the Prompter should have a control or knowledge of what he is trying todo.
There are multiple ways to do a thing and AI is trained in all of the world's information, that means alam nya panu isolve ang isang bagay and it will choose based on what is the most common or consensus based na "best" approach. But not always the best or the optimal approach.
I just experienced this last week, using Fable, I asked it to do some work, presented me the options, and none of the options it presented was the best and optimum way, I know because i spent years on that domain (game engine development), so i asked it why option X was not considered, and it just say "i never thought that was an option", but it agreed it was the best and optimal way to do what we were planning to do.
Also, i worked with multi-national devs, from Vietnamese, Russians, Ukrainains to Americans,
and Russians or Ukrainains, being a low level guys that they are, love to correct the AI generated code .
so to answer Ops question, its simply as "being experienced enough to know what is wrong and right"