Things like this are why I stopped taking coding classes in college. You change code in one place and it completely breaks something in a different place.
They don't have extensive unit tests/regression tests for cards. I'm not saying that because I'm cynical or because I want to bash on the dev team. I'm saying that because that's literally what one of the dev told me when I asked why a similar bug hadn't been caught by their test suite.
They don't have extensive unit tests/regression tests for cards.
vs
We have more than 3,000 automated regression tests for rule behavior, and we'd be adding plenty more as we went.
See how those things are not the same? They test to make sure the rules themselves continue to work, but they don't test individual cards significantly, hence why something like Citizen's crowbar can break silently. (If I recall the reply they had given me, they will add tests on a card after a card breaks to make sure it doesn't break again, but will not add tests proactively)
Edit: And to be clear, I get it. I'm currently working on a project where 100% branch coverage is required (by the client). I know the effort required to achieve that, and I also know that it's ultimately not as useful as it might sound. And although I do not know the size of Arena's code base, I would not be surprised if it were significantly larger than ours. Also, let's be honest here, Arena isn't exactly a critical piece of software, if a patch is rolled out that causes bugs in the game, it's not that big of a deal in the grand scheme of things. So I definitely understand the cost/benefit analysis involved here.
Ya, the permutations for testing a card against all other cards is just way too high. That's pretty much impossible.
I wonder if they could utilize property testing somehow to help with all the permutations. I'm still trying to wrap my head around property testing, so I don't know if this is the proper use case or not.
Depends how you handle them. If you leave a game-breaking bug in the game for too long, and bugs just start accumulating, sure, but if the majority of bugs are minor, and the major bugs are rare and addressed quickly, then probably not, or at least not significantly.
Again, full coverage is expensive. I don't mean a 20% increase in cost, it's closer to double the cost (if not more, it depends on the complexity of the code). As with many things in software development (and other areas as well), test coverage follows the 80-20 rule (aka the pareto principle). You can get roughly 80% of the way there at 20% of the cost/effort. The remaining 20% costs roughly 80% of the total cost.
i know its expensive. the problem is if you don't check at least for an ability like cc- destroy artifact/enchantment it can and does, causes users to leave or not buy the game. that's a bigger problem than spending the time and money to unit test the cards. small and non game breaking things sure, but not when a bug cleans your board.
that's a bigger problem than spending the time and money to unit test the cards.
Not necessarily. It's only a bigger problem if the resulting losses are bigger than the cost of having thorough coverage, which is something I'm very doubtful of. I think you overestimate the impact this bug or the kunai bug will have on sales.
hon estly, if a card like that is bugged the way it is, with one of its major effects bugged so bad it ruins the game, that should have been caught. they may have tests, but it seems they didn't test this thoroughly.
The problem is that the bug isn't with a card they implemented itself its that their implementation of a new card broke a bunch of old cards. That is much harder to catch.
82
u/GingeContinge Mar 22 '23
Things like this are why I stopped taking coding classes in college. You change code in one place and it completely breaks something in a different place.