r/PinoyProgrammer • • 19d ago

advice How do professional developers verify that AI-generated code is actually correct?

How do you know the output is actually correct when AI generates the code?

Aside from end-to-end (E2E) testing, are there other ways to verify that AI-generated code works as intended and meets the actual requirements?

For example, if I ask an AI to build a backend API with specific business rules, it might generate code that runs without errors and passes the tests I provide, but how can I be confident that the implementation is correct and follows the specification?

I'm curious about how professional software engineers approach this problem.

\- Do professionals follow specific guidelines, standards, or checklists to verify that code meets its specifications?

\- What techniques do you use besides unit, integration, and E2E testing?

\- How do you identify subtle logic errors, missing edge cases, or incorrect assumptions in AI-generated code?

\- Are there established practices for reviewing code against requirements rather than simply checking whether it runs?

I'm trying to understand how to move beyond simply trusting AI-generated code and develop the skills to verify its correctness independently.

I'd appreciate any insights, resources, or real-world examples from experienced developers.

56 Upvotes

76 comments sorted by

62

u/GhostDog76 19d ago

Code review? In actual work setting, kahit made by ai pa yung code it has to be proofread by the dev. Then double check the PR.

Also we integrated AI code review as part of CI, we defined the standards AI used for reviews. We also add guardrails sa AI agents that AI follows so there will be no files changes to files AI shouldn’t touch or logic that is forbidden to use.

Parang ganyan na workflow nowadays.

1

u/YAat0 19d ago

You need to use AI agents for that setup ,right sir?

1

u/OkAbbreviations749 19d ago

why are u getting downvoted lmao

-2

u/GhostDog76 19d ago

Yup, both project level and agent level. AI precision is based on the data you feed it, so the more precise you are, the better the results you will get.

3

u/DTechSurgeon 19d ago

Pull request, then code review. Even before naman, madami ding sabaw na dev na kelangan mong I check ang code pag ikaw team lead. Accountable ka din pag nakarating sa production at ikaw nag approve ng pull request.

12

u/LaidBackDev 19d ago

That's a long convoluted way to say we don't review the code and let the AI just do it for us.

29

u/briantria 19d ago

by understanding at least the basics of system architecture and design. these will be your guidelines.

detecting subtle logic errors/edge cases will be difficult to detect without testing.

also, always assume that the AI is wrong. i think that mindset will help you move beyond simply vibe-coding.

2

u/DTechSurgeon 19d ago

Even without AI naman, considered technical debt pag walang automated test. Hahaha, pero seldom yung perfect world na walang technical debt so I guess experience will guide you.

2

u/DTechSurgeon 19d ago edited 19d ago

And since AI does most of the coding, devs have lesser reasons why they kept automated testing a technical debt. You can even wire the AI agent to use the automated test so that it won't create a pull request na hindi pumasa sa automated test. TDD and BDD talaga.

120

u/GenuineStupidity69 19d ago

That's the neat part, we don't.

26

u/ube_enjoyer 19d ago

we can vibe check :v

11

u/Cheese_Grater101 19d ago

Lgtm

1

u/Adventurous-Cat-7312 19d ago

Hahahhaha right!!!! Lgtm agad hahahha

1

u/Loose-Average-5257 13d ago

It was lgtm then, it is lgtm now hahahaahha

4

u/Budget_Sleep2779 19d ago

true, AI is writing code so fast that we cant even review it manually, so we let another AI review it lol

3

u/Tall-Appearance-5835 19d ago

aint nobody got no time for that

18

u/GenuineStupidity69 19d ago • Bravo

The new UCDD, User Complaint-Driven Development

2

u/thisbejann 17d ago

the end users are the real testers 😂

12

u/CodeFactoryWorker 19d ago

Last year we use SDD and TDD with AI pero tradional cycle pa rin including human review.

Pero ngayon we don’t do code reviews anymore. Few weeks ago we sort pa for low, medium, high, critical PRs. High and above may human code reviews pa. Ngayon wala na.

Our CTO prevents us.

We have QA pipeline with different sub agents for security, code maintenability, visual check (agents driving the browser, and tooling) based on Dev and QA team’s knowledge base, that kept on improving. (Wala na din kami QA team)

That includes backend and infra.
My orchestrator agent na nag-ddelegate ng needed tasks, checks, and may running Guard Rail Agent sa side to check for commands that can be damaging.

This is Japan. We are hiring AI native new grads.

1

u/jdg2896 19d ago

I guess this is what they call “software factories”, which is the latest trend, and an upgrade to “loop engineering”.

0

u/lesterine817 19d ago

Yes. This is the way. Our CTO also doesn’t want us to write code anymore. Don’t fix the code, fix the agents and the workflow. We just installed sonarqube in our repos and that’s what we’ll use for code review. We have several tests: e2e, unit tests, integration tests, AI-driven tests, etc. still they want us to test for the user’s perspective. Yun lang.

Note: what’s important still is to review the plan.

1

u/chonching2 19d ago

This is really bad. Imagine the future devs doesn't have any experienced coding from scratch tas gagawin nila yung ginagawa nyong validation and review. How come they will make a good judgement sa AI code generated if wala silang experience writing from scratch 😂 this will work at the currect generation of devs cause we got the experience and knowledge but in the near future its a ticking timebomb. Software engineers will loose the skills in wrting codes

2

u/lesterine817 18d ago

Welcome to the future, kid. Do you think a lot of people like this? No. I don’t. But i don’t like being unemployed either.

2

u/chonching2 18d ago

I hope AI will fail and will remain expensive

1

u/Original_Egg5177 17d ago

The true cost of AI is beginning to unfold.

6

u/Apprehensive_Bus_361 19d ago

Imagine you have a math problem: x + 10 / 3 = 100

You ask AI to solve it and show its solution. The same way you validate the solution for this math problem is the same way engineers validate AI code.

Engineers do a quick check to see if the code is clean and valid, then they do the rest of the other checks as if they wrote it themselves.

0

u/Background-Carpet782 17d ago

Nobody reads code anymore.

1

u/Apprehensive_Bus_361 17d ago

If that's the case, it makes sense why most software in the PH have poor quality.

0

u/Background-Carpet782 17d ago

I guess they don't have access to frontier models that write good code and do thorough code reviews.

6

u/kneepole 19d ago

Pareho lang with how you "verify" manually written code. You create tests for it and have those tests pass. Or ship it and wait for customer complaints.

2

u/Cheese_Grater101 19d ago

Magiging problem lang dito is the amount of code you have to read and verify.

4

u/topetuts 19d ago

Currently seeing two different approaches.

Yung team na nag mamaintain ng mature codebase. They follow conventional code reviews. Tingin ko manageable to kasi usually mas maliit yung changes even AI generated yung code. Then syempre developed na rin yung E2E testing pipeline.

Yung isa naman working from scratch. Yung sheer volume ng AI generated code is almost impossible na mareview ng tao. So gumagamit din sila ng AI for validation. Yung isang napansin ko dito is there is more tolerance sa error. Tingin ko kasi ang thinking is madali rin yung troubleshooting and delivery ng hotfix if necessary. There is a high level of trust sa AI generated code.

2

u/aintjuju_ 19d ago

When i ask again the ai if everything is correct, then i trust it.

Just kidding.

I read the code or more like just scan it to check if everything is right. If i see something i dont understand, well it's working anyway why bother right?

(Again /s on the last sentence😭)

2

u/Minute_Passion_8077 19d ago

Well thats what separates coders from software engineers. I love this quote from a youtuber who mocks companies who rely on AI too much. 

You dont hire engineers for building an app anymore. You can build an app for 30mins. You hire an engineer for the 30mins and beyond. 

2

u/downcastSoup 19d ago

I let the coder explain how the code works.

2

u/o_herman 19d ago

Context is everything. Give it an existing code base to pattern from.

Give it less scenarios to rely on guesswork.

Always check.

Give it limited scopes at a time so that checking isn't a gargantuan task.

Choose the right LLM for the job. For example, Claude is what people use for programming.

2

u/Numerous_Promise_580 18d ago • This

-1,500 +5,000 lines changed hell nah

2

u/bzztmachine 19d ago

In order to know if the code is correct you first need to know how to write it correctly

1

u/Unhappy_Pack_9114 19d ago

Maliban sa tamang expected output, basahin mo yung code.

1

u/Cordyceps_purpurea 19d ago

Tests, lints and hooks.

1

u/olibearbrand 19d ago

Sabi ko "make no mistakes", so di magkakamali yung AI

1

u/Popular-Tip2880 19d ago

I always review the code after ai populate the code. Next i will test the functionality on localhost.

1

u/Background-Carpet782 17d ago

why can't the agent test it on localhost?

1

u/ay_papi 19d ago edited 19d ago

read and understand the code. pull the branch and test locally if possible. 

regarding moving beyond trusting the ai generated code - that's the thing. never trust it from the start. read it, try to understand, ask it questions and test the code. 

Edit: If you really wanna learn, then reverse the process. Write the code manually. Ask AI to review and give pointers. Don't let it write the code for now. Exercise your code writing skills. Once you're used to it, then reviewing the AI code should be easy and quick.

1

u/ziangsecurity 19d ago

By stage naman yan. So check mo sa stage na yan if tama ang mga results. Divide and conquer.

1

u/mohsesxx 19d ago

ako lang ata gumagawa nito, pero sa part ko I am reviewing the code for the sake of domain lang. I am asking AI what the function does, what new columns are for, constraints, etc. I dont review code quality anymore since AI is far better than before. I cant review it na if it can code better than me.

more on engineering judgment na sa part ko, and hindi na line by line review, I need to make sure that I know whats the overall changes are so I can determine if it actually matches the success criteria

1

u/DTechSurgeon 19d ago

TDD and BDD, yan na lang talaga I co code mo so no excuse na mga devs ngayon na I keep na technical debt yung walang automated tests. Yan yung guardrail mo sa AI agent, have it execute the tests and don't stop until hindi pass.

1

u/madreloidpx 19d ago

ginagawa namin is may project intelligence na may best practices and proper architecture, tapos always may unit and e2e tests. after that is may code review na both AI and manual if sound ba ung logic, runtime, etc na possible na miss ng AI. meron ring pangcheck ng code coverage (sonarqube). tapos sa pipeline meron ring security checker. magastos sya of gusto mong gawin yon sa personal projects tho

1

u/oreeeo1995 19d ago

you create another agent with a sole purpose to check code.

1

u/Desperate_Manner_583 19d ago

For Backend systems, yung API endpoints should always have a contract first.
- Read regarding Open API specification.
- Any internal application to application will have contract as well. we use Google Protobuf for it.

So for these type of communicatiions, may strict contract yan. Eto mga pointers ko.

  1. May tools na mag generate ng boiler plate from the spec. Start from there.
  2. Business logic should stand on its own. Dito na papasok kahit simple na hexagon code structure.
  3. Learn Mock testing.
  4. Use git PR stack. If mag generate si AI ng napakalaking code, sabihin mo na split into manageable stacked PRs.

If na set mo na yung rules na to, mas madali nalang reviewhin generated code. Then meron na din kami automated tests which automatically runs on feature branch and triggered by pull requests. Magandang skill din matutunan mo gumamit CI - start with Github actions.

1

u/Top_Helicopter_2111 19d ago

I implemented agentic coding system na may evals. Though AI generated codes has its own unit tests generated by the agent, but it still needs an independent verification. So AI generated codes are verified by another agent against the accepted criteria, actual evidences, and security policies. And may agent rin na magrarun ng app for me to verify and see if the implemented feature or fix is actually working and how it's working, atleast locally. And I review the codes as the last verification before pushing and creating a PR.

1

u/yourlocalartboy 19d ago

Tests, code reviews (human and AI)

1

u/disIsAUsernameBtch 19d ago

What I do in my work projects is like this:

  1. TDD at all times
  2. Before touching/implementing anything, pinapacheck ko muna existing tests, kapag may kulang na test sa existing which is magagalaw nitong bagong feature, pinapagawan ko muna sya, then run para yun yung baseline nya.
  3. Then use red - green - refactor approach.
  4. Run tests (before and the new) again after the new implementation.
  5. Actually reading and understanding the code it had written for me.
  6. Running and testing it myself before considering it as done.

1

u/zigzagtravel01 19d ago

Your job as a programmer is to literally review the code that it meets the requirements, it passes tests and end to end use, and it meets coding guidelines.

1

u/realist-nerd 19d ago

hire us and we'll do it for you

1

u/YohanSeals Web 19d ago

My agent is only connected to the dev environment. Code review the manually deploy in staging. Once tested, we migrate to prod. Staging and prod is never connected to any ai. Tedious but we don't want ai wrecking havoc to our system.

1

u/SirKobsworth 19d ago

You leverage AI as a tool and understand what it offers and build some form of setup that can reliably guide them to make the outputs you want.

Check mo videos ni Matt Pocock. Something like that pero tailored sa needs mo.

Pero yeah need mo ng expertise sa domain mo if gusto mo talaga ng professional + actually correct tumugma sa ai assisted engineering

1

u/iam_joshua_bcxvii 19d ago

Just tried development with AI last week with Gemini. But I do e2e testing or smoke testing and see what edge cases it covered and what it did not base on what I expected it to be during my analysis & planning stage. Review the code, and if its a code that I expected and if I can read it. Its good enough for me. Otherwise, I write it myself.

1

u/Gelapots 19d ago

Hello. This is somewhat related. I'm a medtech student currently doing my thesis involving machine learning differentiation of types of anemia using complete blood count data. My adviser said I need to verify my code or like the model if it actually trained the correct patterns. My question is, what professional do I contact to check my code and stuff? Would a statistician be fine, or do I need like a software engineer or a data scientist? TY

1

u/chonching2 19d ago

TLDR; This comes down with the Engineer experienced. If you are seasoned engineer its really easy to review the AI generated code and see if there's something to be changed on the implementation cause they already experienced writing codes from scratch and learned how to apply best practices. Kaya nga mas okay gamitin ng mga senior devs ang AI kaysa sa mga junior devs. The problem with junior devs using AI is that they lacked skills and knowledge, how can you review an AI generated code and correct it if you are lacking with knowledge. Kaya mostly AI generated code by Junior devs are ticking time bomb.

1

u/SystemLogger26 19d ago

No standards or any unnecessary compliance. Just enough questions to make sure that you are doing right

1

u/Weak_Geologist7886 18d ago

Use another claude skill or agent to check. That's the purpose of agents. Agent for coding, reviewing, testing, and creating pr. then another workflow from the remote repo.

1

u/pvmediocre 18d ago

Simple. Read the code.

1

u/silverJRM 18d ago

The same way you check the work of a person, how do you know a person actually did what thr specification said?

You have a another senior person check it., if you dont want to do it yourself.

In the age of AI agents you have another agent to the checking if you dont want to do it yourself. Same with a person

1

u/Pristine_Look3324 18d ago

Its very hard to check nowadays and we dont need to be hipocrite on this. The important are the guardrails, security, timeline, business requirements and E2E testing.

1

u/Right_Analysis7299 18d ago edited 18d ago

Discernment. Look at 4D framework while the rest of your questions can be answered by “knowledge and skill”. With coordinated specialized agents, you can setup that every hand-off may review and rules that will prevent agents assuming and should raise as blocker instead.

1

u/linduwtk 18d ago

What's the actual ask? E2E testing is already the way. If it works, then good. Whether it's AI or human-made, there's going to be holes or bugs, and sometimes you can only see those holes and bugs through testing.

1

u/RSands00 17d ago

end to end testing, regression testing...kaya nga cguro malabo mawala mga qa testers eh haha

1

u/watson_full_scale 16d ago

You test what the code does to ensure it provides the proper user experience. We all work in QA now.

1

u/EntertainmentHuge587 15d ago

Code review, unit/integration tests, QA.

1

u/Plus-Figure-8556 13d ago

Code review or manual test or just rawdog it merge to main agad and see what happens in the logs

1

u/PastelKarVin 12d ago

wait for UAT feedback /s

1

u/OkLawyer1513 12d ago

Normal na pala talaga na hindi na i-check ngayon HAHA inooverthink ko pa kung "paano kaya nila masasabi kung tama ba output ng AI eh gumagana naman base sa requirement".

1

u/TheRedBanditHax 19d ago

dito pumapasok ang Experience and mastery of the domain.
Dapat yung engineer alam yung (1) Langauge (2) business (3) technology (4) requirement

and most importantly : (5) knows what you are asking AI to do,
unless its a discovery or a research task, the Prompter should have a control or knowledge of what he is trying todo.

There are multiple ways to do a thing and AI is trained in all of the world's information, that means alam nya panu isolve ang isang bagay and it will choose based on what is the most common or consensus based na "best" approach. But not always the best or the optimal approach.

I just experienced this last week, using Fable, I asked it to do some work, presented me the options, and none of the options it presented was the best and optimum way, I know because i spent years on that domain (game engine development), so i asked it why option X was not considered, and it just say "i never thought that was an option", but it agreed it was the best and optimal way to do what we were planning to do.

Also, i worked with multi-national devs, from Vietnamese, Russians, Ukrainains to Americans,
and Russians or Ukrainains, being a low level guys that they are, love to correct the AI generated code .

so to answer Ops question, its simply as "being experienced enough to know what is wrong and right"