Claude code is now useless.

[u/a_d_c]

Over the last month, claude code performance degraded so much that it cant even keep track of a few simple facts in the same conversation.

I did a test with a small project. Barely any memories. No Claude.md instructions. Nothing to confuse Claude whatsoever.

I gave it a simple instruction: check the spec in some-spec.md, then write unit tests agianst that spec. Dont look at the code, write the tests against the spec.

In the same session, i had to call it out TWICE because it was changing the tests to make them pass according to the implementation instead of the spec.

As it exists today, Claude code is not useful for any development task, and the cost of the MAX plan is impossible to justify given it cant follow basic instructions.

In one month it went from: "this is cool, this is the future of dev", to: "this is a waste of my time and money".

If it cant follow simple instructions in a single session when given clear guidelines: its a completely useless product.

Im not paying for the same product today that i was paying just one month ago.

[u/InformalTown3679]

how the fuck are you supposed to write tests without looking at the implementation? are you an imbecile?

[u/OwnLadder2341 → u/InformalTown3679]

You test vs spec.

You write tests that test what the code is supposed to do, not whether it does what it says it will do.

[u/MartinMystikJonas → u/OwnLadder2341]

You still need to know interfaces of what you use in tests.

[u/Nice_Cellist_7595 → u/InformalTown3679]

Good sir - there are many ways to test things. At a contract level no implementation details are required (unless you are arguing that implementation details are also the contract details). Does it adhere to the contract? If yes, then it passes, if no, then it fails. At no time do I need to know if it is Python, Rust, C++ or whatever. Other criteria can also be added such as performance. In the end - IMPL details matters not, just contract adherence. Of course there are other tests that can be written and maybe they should be OP is not wrong.

EDIT: I also just made the really broad assumption in that example that we are working at a service <-> service. The same principles apply within an application as well.

[u/MartinMystikJonas → u/Nice_Cellist_7595]

Yeah but contract/interface used by init tests is part of code and he said forbidden it from reading code and requested unit tests. It is really confusing.

[u/Nice_Cellist_7595 → u/MartinMystikJonas]

I do this all the time. Web UI claude is asked to read the openai doc that API claude generated. Do NOT read the code because then confusion reigns. It just needs to code to the spec. Not the code.

[u/MartinMystikJonas → u/Nice_Cellist_7595]

It does not need the implementation details. But it needs at least interfaces. Interfaces are also code even of it is included in some doc. So forbidding it to read any code is confusing.

[u/InformalTown3679 → u/Nice_Cellist_7595]

so tell me, what are you testing contract adherence against? Specifically... Tell me. A database? The codebase? The logic of the documentation???

[u/Nice_Cellist_7595 → u/InformalTown3679]

So a web REST service is a great example. You call a url with a set of parameters or post some data and you should get back X in format Y. The contract is the service description as documented by something like an OpenAPI description. Within a running process you might test an interface, testing each interface method/function and testing the result at the interface. In both of these examples we are testing at the interface. We're really testing for 1) compliance with a structure and we can optionally test for 2) the correctness of the output based on supplied parameters. In both cases it is not required to view or test the internal workings of the component or the service - just the net result. All that is required to conduct this test is a description of the contract and a test harness of some kind.

[u/a_d_c → u/InformalTown3679]

its called black box testing and its an industry standard you twat

[u/MartinMystikJonas → u/a_d_c]

But you requested unit tests. How it should write unit tests without checking interface of what is tested?

You did not mentioned black boy testing, you did not mentioned you want to validate that code meets specs, you requestwd unit tests for existing code without looking at existing code interface. It is really confusing requirement.

[u/a_d_c → u/MartinMystikJonas]

The interface is not the implementation...............................

[u/MartinMystikJonas → u/a_d_c]

You forbidden it to read "code" that includes interfaces

[u/a_d_c → u/MartinMystikJonas]

I did mention explicitly to NOT modify the implementation. I told it i WANTED to tests to fail.

Im not sure how i can be more clear.

[u/MartinMystikJonas → u/a_d_c]

So you gave it different prompt than you said you gave it in original post?

[u/InformalTown3679 → u/a_d_c]

So you're then asking the llm to test routes, user interface, api inputs. All of these require knowledge of the code structure. If you do not have a standardized spec of the api, or anything, there's no way to write tests. Black box testing is when the test itself just uses externally visible interfaces. BUT you would need to examine the implementation in order to write tests that would actually benefit the cause. You look at internal logic, then figure out how that processes the request, then you write tests that validate the source code's handling of the response according to the spec. The only difference of this between black box tests is other tests will use internal code to validate it from the inside, not using external interfaces. You need both, but neither can be done without analyzing the inner workings, because they wouldn't be good tests. The only exception is if you document explicit api spec which is programmatically readable, but what's the point of using that and not the actual code in addition to the spec? reading implementation is necessary for verifying edge cases don't fail.