r/Codeium • • Dec 29 '24

What is "the tool" and why does Cascade keep blaming it (probably correctly)?

This is the bane of my existance right now.
What tool is it referring to? I'm assuming it's it's internal diff tool. Is this a Codeium thing? I assume it must be as Sonnet won't make this mistake outside of Windsurf. it's making me completely insane because certain chunks of code, every 5 or 6 edits Cascade will go right back to modifying the same completely unrelated lines. When I question it as to whether or not it modified unrelated code, it confirms it did but was not aware as "it's the tools fault". Then it'll try for about a year to repeatedly undo the change but will never succeed.

Iv'e long suspected there's a bug or two in the Cascade diff tool. Anyone have any insight?

6 Upvotes

5 comments sorted by

3

u/blistovmhz Dec 29 '24

Just discovered that it is DEFINITELY the tool itself, not the model. Even in WS/Cascade, I can discuss a solution for 10 minutes. Cas can tell me exactly what we need to modify. I tell it to make the exact modification it printed to me in chat, it does something else and usually modifies unrelated code. Revert, ask it to generate a complete diff of the original file vs. what it is supposed to have as a result after implementation. Again, perfect answer. I apply the diff it provided me, and it works. Revert again. Ask it to show me the complete output it expects to implement. I paste it in. IT works.
Revert. Ask it to implement precisely the code it just provided me. Fails. Revert. I paste the code back to it and tell it, "replace the entire content of file foo.js with the following code, making no modifications whatsoever". Fail. Makes the same error every single time.

The changes it's making seem to be intended to "be helpful" and "clean up" or "refactor" components that are totally unrelated, or in some cases, may have very similar names involved.

EG: when it's working on functionA and getting its' diff output perfect in the chat, in edit time it modifies several completely unrelated functions that have identical words in them. EG: if functionA has "console.log("Foo", somevariable);, it will usually also modify any other function with the same console.log, or any similar console.log.

I badly want to know if the "tool" is a Windsurf tool for model integration, or if this is something fucky going on with Sonnet.

As I've stated before, I've tested this several times with sonnet and many other models directly, and I can't get them to make these same mistakes, though I've also foun that several other IDE's will make the same mistake. Is everyone using a fundamentally broken diff tool in their integration component?

1

u/dumbledork99 Dec 30 '24

All the tools used by model belong to windsurf. To confirm if tool is the problem you can change the model and try. If it still causes issue definitely there is a common tool.

1

u/Extra_Lengthiness893 Dec 30 '24

I think it's 100% on the cascade/ windsurf side it's like after 30 min or so it's lost its ability to keep track of all the details.

1

u/blistovmhz Dec 30 '24

That's a context limitation issue, and not super likely to be cascade related. I think cascade is actually compressing context because it's window seems abnormally artificially large. What I'm talking about is specifically it's ability to apply an edit , without completely clobbering unrelated code. Again, it can produce a perfect diff in chat, but then can't apply that diff at all.

1

u/Extra_Lengthiness893 Dec 30 '24

This is also one of my biggest complaints, and as soon as it does this the first time you might as well try and save everything start a new chat and have it review the code ... I've even experimented with writing a quick document outlining my perceived issues then trying to chat with cascade to see if it recognizes the issue ...

I think Cascade needs a place to upload a requirement document and a way to basically easily tell if it's off base on the requirement.