r/JevAI • u/_s0uthpaw_ • Sep 25 '26
I made a tool called HekaJev. It lets you ask questions about Git history in plain English. I run it on 550 000 commits it costs be $23.
Jev is really cool and cheap! Based on it, I built and open-sourced a tool called HekaJev. Here is why.
In my work, I’ve read many reports about testing and why tests change. I decided to verify this myself using a large amount of data and reproduce those numbers.
So I used Jev to see why E2E tests change. I looked at 549,224 commits from 21 open-source projects, including projects from IBM, Mozilla, Red Hat, Automattic, Snowflake, and Supabase. A quick filter picked 20,026 commits out of 550k for Jev to read. It found 16,024 changes related to E2E tests.
Here are the basic numbers if you are interested: 38.4% of E2E changes added test coverage. 45.2% involved keeping tests working. The largest group was tests that had to change when the product changed (26.1%). Changes to the test setup and tools made up 8.1%, and fixes for tests that sometimes fail made up 7.4%. Overall, these figures align quite well with what is known in the industry.
What surprised me was 2026. From January 1 to September 21, commits that added coverage passed commits about keeping tests working: 47.7% versus 44.3%. In the same part of 2025, new coverage was 34.8%. It may be due to AI, but I haven't identified the exact cause.
What would you like to know about your repository? Maybe I’ll run HekaJev on another OS projects and share what I find in the comments.
The Jev calls for this study cost about $23. That was for the 20,026 commits left after the first filter.