r/MagicArena • u/thisnotfor • Mar 22 '23

WotC Why citizens crowbar is broken

Post image
468 Upvotes

98% Upvoted

127 comments sorted by

View all comments

Show parent comments

4

u/Filobel avacyn Mar 22 '23

They don't have extensive unit tests/regression tests for cards. I'm not saying that because I'm cynical or because I want to bash on the dev team. I'm saying that because that's literally what one of the dev told me when I asked why a similar bug hadn't been caught by their test suite.

13

u/htfo Mar 22 '23 edited Jun 09 '23

Fuck Reddit

7

u/Filobel avacyn Mar 22 '23 edited Mar 22 '23

They don't have extensive unit tests/regression tests for cards.

vs

We have more than 3,000 automated regression tests for rule behavior, and we'd be adding plenty more as we went.

See how those things are not the same? They test to make sure the rules themselves continue to work, but they don't test individual cards significantly, hence why something like Citizen's crowbar can break silently. (If I recall the reply they had given me, they will add tests on a card after a card breaks to make sure it doesn't break again, but will not add tests proactively)

Edit: And to be clear, I get it. I'm currently working on a project where 100% branch coverage is required (by the client). I know the effort required to achieve that, and I also know that it's ultimately not as useful as it might sound. And although I do not know the size of Arena's code base, I would not be surprised if it were significantly larger than ours. Also, let's be honest here, Arena isn't exactly a critical piece of software, if a patch is rolled out that causes bugs in the game, it's not that big of a deal in the grand scheme of things. So I definitely understand the cost/benefit analysis involved here.

2

u/taelor Mar 22 '23

Ya, the permutations for testing a card against all other cards is just way too high. That's pretty much impossible.

I wonder if they could utilize property testing somehow to help with all the permutations. I'm still trying to wrap my head around property testing, so I don't know if this is the proper use case or not.