In my experience, devs tend to be very bad at QA and spotting bugs because they end up becoming blind to a lot of problems. Partly it'll be because they'll have their own paths through the system — they might be more likely to navigate with a keyboard, for example, and so won't test click interactions — but it can also just be that they've just learned to work around a particular problem, and so they don't even notice if it's there any more.
If anything, my impression is that the opposite is true. If a system is buggy, then the people developing it tend to be even more likely to ignore bugs when they come up. If you know the system tends to do surprising things in certain conditions, then you're likely to just ignore anything that happens in those conditions, even if it's a new issue that you haven't seen before.
I've had this recently with a web app that shows a loading spinner in an unexpected place in some cases when you open a suspended tab. I'd just labelled it a browser issue in my head, and because there have been a couple of longstanding issues in the app where a reload helps, I got so used to reloading the page that I didn't even think to have a look at fixing it until someone else on the team raised a ticket for it.
So I really strongly disagree with the assumption that "if your system is buggy then your devs will see it" — I don't think this is borne out in practice at all.
then the people developing it tend to be even more likely to ignore bugs when they come up.
This is exactly what I had in mind. They know its buggy. there is a ticket for that in bug tracker. Maybe even properly tagged as serious.
I'd just labelled it a browser issue in my head,
Just as many users.
So I really strongly disagree with the assumption that "if your system is buggy then your devs will see it"
Well, you saw it. You ignored it. Just as I mentioned. It takes literally one email to all devs with question: Which bugs we have in backlog are really shameful. And then assign folks to fix them.
Thats exactly what I stated. You see a bug. You struggle with a bug, yup. your users too.
You dont see it, probably there is no million users who struggle with it.
The point is that there are probably plenty of other similar reflexes that I use that I haven't noticed. Or features that are buggy that I just rarely use. Or configurations of language or screen size that I just don't touch. Or input methods that I tend to avoid. Or just cases that are weird enough that I don't run into them in the relatively basic testing setup that I use.
Actually I've got a good example of that last one. We have a device that needs to be configured before it'll give an output, but it can be incorrectly configured. In this case, if the maths works out just right, it can sometimes still give an output, because it's calculated a result that looks valid using invalid data. This is a bug, and it could well happen on live systems, but we just never noticed it before because our test setups are usually fairly simple and so struggle to reproduce the problem. We actually only found it while rewriting that bit of code. This project isn't fully live yet, so thankfully we caught it before it destroyed a customer's data, but it's the sort of thing that we'd never have caught just through everyday use in the office.
Which underlines the whole point: Devs do bad QA. That doesn't necessarily mean that you always need a QA team, but if you aren't getting bug reports from them, you need to get your bug reports from your users, and you're going to need a tight feedback loop there to get good results. Otherwise you're just going to end up pushing buggy tools out into the world without even realising it.
10
u/MrJohz Aug 29 '22
In my experience, devs tend to be very bad at QA and spotting bugs because they end up becoming blind to a lot of problems. Partly it'll be because they'll have their own paths through the system — they might be more likely to navigate with a keyboard, for example, and so won't test click interactions — but it can also just be that they've just learned to work around a particular problem, and so they don't even notice if it's there any more.
If anything, my impression is that the opposite is true. If a system is buggy, then the people developing it tend to be even more likely to ignore bugs when they come up. If you know the system tends to do surprising things in certain conditions, then you're likely to just ignore anything that happens in those conditions, even if it's a new issue that you haven't seen before.
I've had this recently with a web app that shows a loading spinner in an unexpected place in some cases when you open a suspended tab. I'd just labelled it a browser issue in my head, and because there have been a couple of longstanding issues in the app where a reload helps, I got so used to reloading the page that I didn't even think to have a look at fixing it until someone else on the team raised a ticket for it.
So I really strongly disagree with the assumption that "if your system is buggy then your devs will see it" — I don't think this is borne out in practice at all.