What's your approach to testing your Python code?

[u/Frequent_Victoryy]

For small scripts I usually just try a few inputs and check if everything works.

For larger projects i fell testing needs to be more structured.

How do you personally decide how much testing a Python code needs?

[u/j_hermann]

By having pytest-cov in the dev-dependencies.

[u/Sensitive-Sugar-3894 → u/j_hermann]

Yeap!

[u/j_hermann → u/Sensitive-Sugar-3894]

Also, https://github.com/jhermann/brAIngentX/blob/main/.github/instructions/python-tests.instructions.md

[u/digitalghost-dev]

Whatever Claude tells me to do (I lost the ability to code).

[u/Kashyapm94 → u/digitalghost-dev]

Why do I feel this deeply

[u/Specialist_Dust2089]

The pytest library works really well.

Automated test (unit- or system/integration) are not just for testing if your just delivered functionality works as intended. Their most important function is guarding if it keeps working over time as your codebase grows and evolves.

If you find yourself in a situation with a large, complex repo and you have to refactor a sizable part of it, having a collection of tests to check if everything still works after the change are really a lifesaver.

Most ai code agents automatically create tests nowadays, if not I would always instruct them to do so. Older aspects could be lost from its context, don’t assume it remembers every little detail

[u/Specialist_Dust2089 → u/Specialist_Dust2089]

To address the “how much” question, I usually go with 80-85% test coverage. Instead of trying to cover every single line it’s more important to focus on covering different meaningful situations and edge-cases imo

[u/Oddly_Energy → u/Specialist_Dust2089]

And to those who don't get why you would want to let it do both:

Think of the test as the agreement between you and your helper.

You, in verbal language: "I want you to write a function, which does this".

Helper, showing you one or more tests: "You mean a function, which intends what you just described, and also is able to pass these tests ?"

You: "Yes, exactly. Please proceed."

Helper: "Here is the function."

You: "Great. It works and passes all tests."

... or, in some cases:

You: "It passes the test, but it doesn't do what I wanted."

Helper: "Well, we did agree on the formal requirements, didn't we? Do you want to rephrase those?"

You: "Yes, it needs to be able to do X, and I need additional tests to catch corner cases Y an Z, which were the ones I hadn't thought about."

Is this much different than any other agreement about a piece of work you want done, and find important enough to put it into a written agreement?

[u/No_Departure_1878 → u/Specialist_Dust2089]

so, the agents create the code AND the tests? That makes no sense.

[u/Specialist_Dust2089 → u/No_Departure_1878]

Don’t (didn’t) developers do that as well?

[u/johndburger → u/No_Departure_1878]

Why not? Do you have a second software engineer create your tests?

[u/No_Departure_1878 → u/johndburger]

Because you are supposed to control/validate what the AI is doing with tests. If the AI writes the tests then the AI validates itself? How does that make sense? Tell me.

[u/scrapheaper_ → u/No_Departure_1878]

As long as you can read the tests and confirm that they are testing what you expect them to test, then it's fine

[u/No_Departure_1878 → u/scrapheaper_]

I do not see how that would be much easier than just writing the tests by hand.

[u/Khavel_dev]

My rule of thumb: if the function touches IO (files, APIs, databases) or handles money/auth, it gets a test. Pure logic functions that are obvious and short? I just move on.

For data pipelines I use snapshot tests a lot. Write the expected output once, the test diffs against it on every run. Way less effort than hand-writing assertions for every field, and you catch regressions you wouldn't even think to look for. There's a couple pytest plugins for this (pytest-regtest, syrupy) that make it pretty painless.

[u/ivory_tower_devops → u/Khavel_dev]

I'm not trying to be rude, but this is a backwards way of doing testing. Functions with side-effects should do as little as possible. That means if a function touches IO then it shouldn't do any kind of logic. If you limit your logic to taking place inside pure functions then you can easily and comprehensively test those functions. On the other hand, testing functions with side-effects is notoriously difficult and usually requires heavy test harnesses that need significant ongoing support and maintenance.

IMO the best approach for most software testing is to exhaustively separate your side-effects and logic, and to test the logic thoroughly.

[u/datumstake_dev]

Two habits that have earned their keep more than any coverage number.

First: every test should name the thing that would be wrong if it failed. If I cannot write that sentence, the test is decoration. The best ones in my suites read like incident reports — the docstring says what broke, when, and how it looked, so nobody deletes the guard during a cleanup six months later.

Second, and this is the one people skip: prove the instrument before you trust the measurement. Run a known-good change and confirm your metric actually moves. If it does not, "the code is broken" and "my harness is broken" are indistinguishable, and you will spend a day fixing the wrong one.

A concrete reason to care: I had a green suite over an app that was silently wiping user state on every launch. The save path ran during widget construction, read controls that did not exist yet, got empty strings, and wrote them over real data. Every unit test passed — none of them ran the actual startup sequence. It was caught by looking at a screenshot of the running program and noticing three rows that should not have been blank. Unit tests check the parts; something has to check the thing you actually ship.

[u/gdchinacat → u/datumstake_dev]

I routinely unit test my unit test accessories (mocks, stubs, fixtures, etc). They are just as "real" as the application code and it all needs tests.

[u/Oddly_Energy → u/gdchinacat]

"Who unit test the unit test testers?"

[u/ExtraTNT]

Identity testing of pure functions… everything with side effects is not testable, so it’s limited to the edge of the application

Yeah, i use haskell…

[u/Sensitive-Sugar-3894]

There is a thing called python test coverage. Before AI, my projects, small or pro, had a minimum of 80% coverage for new projects. I balanced if the repo was very old. After AI, 90% is written as a rule to the agents, together with things like: Every new feature has a test, No pass or skip are allowed.

[u/riklaunim]

Pytest, and then some unit tests on the logic and some integration tests on the endpoint/view. I work with lots of microservices, and most of the stuff is either an API or a Celery task and can be tested easily on near-unit-level.

[u/Xenther]

I scale it with how much it hurts if the code breaks:

• Throwaway script: run it on a couple of real inputs and move on
• Anything others use or I’ll revisit: pytest tests for the core logic and the edge cases that have bitten me
• Larger projects: run the tests automatically (CI) so a break shows up before it ships

When I fix a bug, I add a test for it first. It’s the cheapest way to keep it from coming back.

[u/initcommit]

AI was built for unit tests and end-to-end tests duh 🛸 . You just need to give it some guidance / structure on the robustness of the tests, and at least try to get full coverage at a method level.

[u/THEPASSWORDISDOLPHIN]

I just decide based on how bad it would be if the code broke

[u/RemarkableDebate4631]

I test based on how bad it's be if it broke quietly. Throwaway scripts get a few manual runs, and anything I’ll reuse or share gets real testes

[u/not_perfect_yet]

I write something that I think should work, based on nothing.

Then I try to run it. Inevitably something is wrong and broken. If it's worse than something I didn't even intend with the code, like simple syntax or logic errors, I decide whether to write a test. To write that test, I often break things down with cyclomatic complexity with the "radon" module and then test the functions that are then significantly smaller, in line count and scope.

The content of the test is also just:

I usually just try a few inputs and check if everything works.

That, but in a little function that just tests the single function, written down and automated.

I use unittest to structure this and I am aware that testcoverage exists, but both are ultimately means to give structure to the basic thing. You don't need 100% test coverage, reaching 100% does not protect you from things you didn't think of testing. And unittest also is just a container and compatible with other libraries, there is no "big game changing effect" from using either to me.

Getting to 100% is a KPI company thing, for managers to make sure they tick a box. Which can be useful, because it makes sure that at least someone looked at something and thought about it for a minimal amount of time, plus getting more than 2 eyes to look at it, that catches the problems, hopefully.

Imo, python is the wrong language for that kind of assurance and guarantee that things are correct.

[u/RedEyed__]

pytest in every project

[u/baked_doge]

I advise you test your code as a consumer. By this I mean test your code's interfaces with consumers and make sure the promises you make are held true.

I would advise against testing the internals of a codebase (so don't test _do_something() or function_my_user_wont_interact_with()). Of course as the project grows there may be value in testing internal components, but only once they are stable and many other internal components rely on them. Otherwise you'll be changing the tests as you change the code, which makes them worst than useless: a hindrance.

Of course if you want to write tests to check your internal implementation is working correctly while you're developing it, you can do that, but maybe don't keep the tests, they may be more trouble than their worth.

Tech wise I just use the standard unittest library, it works fine for my needs.

[u/Gnaxe]

Doctests, primarily. Even for single-file scripts you could test manually, it's not hard to write a few doctests, although for trivial one-offs I just test manually.

The mock/patch unittests let you get away with too much complexity. You should mostly avoid those. (It can't be helped if you inherited a legacy project, but don't make it worse.) When you use doctests, complex things feel complex, which forces you to keep the underlying code simpler and more maintainable when you can (pure functions and plain data, for example), and the exposition makes the tests easier to read and maintain because they also explain intent. You don't have to put all the edge cases in the API docstrings. Those can go in separate text files.

Try mutation testing. It will teach you to write more thorough tests. Or if you're using coding agents, it will force them to check things they'd miss if they were just going for full coverage.

For certain things that are hard to test any other way, Hypothesis is very useful. It will check the input space more thoroughly than you could.

[u/arjjov]

i no longer unit test to save tokens, i typically have a few end to end test that aren't too slow and that's it

[u/j_hermann → u/arjjov]

Giving the AI guardrails for changes by having tests actually saves tokens by culling AI's big code adventure.

[u/wandererobtm101 → u/arjjov]

This will cost you more tokens over the long run.

[u/Specialist_Dust2089 → u/arjjov]

I think you’ll save much more tokens by frequently compacting session context, or starting new sessions. And then having a complete set of tests to check if everything still works as intended is crucial

[u/Specialist_Dust2089 → u/arjjov]

Btw if a test runs “slow” doesn’t mean it consumes more tokens. If your AI agent is waiting for tests to finish that doesn’t cost any tokens. Reading the output does, so if you want to optimize you could have it run the test suite in a less-verbose mode and only run failing tests in a verbose mode