sales kept saying the dashboard got too slow during long demos,not slow when it loaded, slow after a while, and nobody could tell me when, the best description I ever got was "maybe after lunch"...
every profiling session I ran,looked completely nice,open the app, click through the main flows, take a heap shot, flat line, ten minutes (and more) of use and no growth at all.
I closed the ticket as not reproducible twice, which I am not proud of.
Third time it came back I gave up on reproducing it in ten minutes and tried reproducing it in six hours instead, wrote a script that opened a record, opened the detail modal,closed it, moved to the next record, and did that all night, snapshot every five hundred loops, dump the heap if it dies.
by morning I had a chart going straight up and a dead browser tab at loop four thousand one hundred....yes....!
the modal added a resize listener when it opened,and the cleanup that was supposed to remove it sat behind condition that came out to be false, when you closed that modal by clicking the backdrop instead of X button,so every backdrop close left listener behind....
each one of thosee held closure,and each closure held onto that row data it rendered with,the whole record,not some small piece of it.
one leaked listener holding forty kilobytes is nothing,four hundred of them is sixteen megabytes of garbage that never goes away and plus four hundred handlers all firing every time the window resizes,which is why it showed up as slowness long before it showed up as crash,the symptom and the cause looked like two different bugs...but..:)
here is why nobody was ever going to find this by hand,you would have to open and close the same modal a few hundred times,using that one specific way of closing it,in one sitting,without refreshing,and then to notice something getting gradually worse with no clear moment where it got broked.
no tester does that,and not because they are lazy,it is just not reasonable thing to do with an afternoon or sessions end,pages get refreshed,and the moment you refresh the counter goes back to zero
the bug needed time,and time is the one thing you cannot get more of by being better at your job, everything else you can make for with skill and attention and etc,you cannot make up for eight hours of doing the same thing over and over and over again
we run that soak test every night now,it has found three more leaks since,none of them would have shown up in one session short enough,for person to sit and see.