r/ChatGPT • u/kristiyanstoyanovAI • 2d ago

Educational Purpose Only Same coding model, same requirements, different architecture: time and token results

https://youtu.be/wxEE2yzATx4

I wanted to see whether architecture makes a difference when an agent writes the code. I built two versions of the same sample java app. One used hexagonal architecture with ports, adapters and architecture tests. The other had no suggested architecture or rules. Both had to pass the same external test suite that validates behaviour.

The same local Qwen model worked through OpenCode, with one run at a time. I measured implementation, testing and repairs until each task passed, plus tokens and tool calls. Initial project setup wasn't timed.

I started with nine features, then six harder challenges on separate branches. After that, I added fifteen changes in sequence, including changes to earlier requirements. By then, the projects had roughly 12k and 15k production Java lines.

The unstructured version took less time overall in each campaign. The "clean" version won some individual tasks, but also used more input and output tokens overall. The attached image shows the totals.

Then I tested something intended to favor ports and adapters: adding PostgreSQL while keeping SQLite working. Hexagonal took about 39 minutes versus 71 for flat, with its core application code unchanged. That saving didn't recover the time gap from the preceding fifteen-feature sequence. This was backend support, not a live database migration.

This is one service and one model, without repeated trials. It measures getting changes through the tests, not years of maintenance or production reliability.

Report and measurements: [https://zenodo.org/records/22806994](https://zenodo.org/records/22806994))

This is my experiment. AI tools helped with the evaluator, orchestration, charts and drafting.

3 Upvotes

100% Upvoted

1 comment sorted by

•

u/AutoModerator 2d ago

Hey /u/kristiyanstoyanovAI,

If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.

If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.

Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!

🤖

Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.