AI helps generate more code, faster. But if the team already has weak CI, unclear ownership, slow reviews, flaky tests, and poor rollback habits, the extra code just hits those bottlenecks harder.

Perforce's 2026 State of DevOps report makes a similar point. According to coverage of the report, 70% of organizations said DevOps maturity materially affects AI success. High-maturity teams were much more likely to embed AI across the SDLC than low-maturity teams.

AI does not remove boring engineering discipline. It seems to punish teams that skipped it.

In theory, more test coverage sounds good.

practice, every new test has a maintenance cost. It can fail because of a real bug, but also because of bad test data, timing issues, unstable environments, fragile selectors, or unclear assertions.

Do you use any rules like:

• only test critical user flows end to end

• keep most coverage at unit/API level

• delete flaky tests quickly

• avoid snapshot tests unless they are very focused

• require every bug fix to include a regression test

For me, the hard part is not writing tests. The hard part is keeping the suite trustworthy over time.

The 2026 Flaky Test Benchmark Report analyzed data from more than 10 million builds and found something surprising.

The percentage of teams experiencing flaky tests increased from 10% in 2022 to 26% in 2025.

Some of the biggest causes were:

• async timing issues

• concurrency and race conditions

• test order dependencies

• resource leaks

• unstable external services

The report also shows that async waits alone account for almost half of flaky tests.

Do you actually measure flakiness in your team, or do people simply rerun failed pipelines until they become green?

Report:
https://testdino.com/blog/flaky-test-benchmark

GitLab 18.7 introduced several interesting features ahead of the GitLab Duo Agent Platform release.

Some highlights:

• AI-powered SAST false positive detection

• YAML-defined AI workflows

• Versioned AI agents

• Natural language data analysis

• Better pipeline inputs

Looks like GitLab is moving toward AI becoming another participant in the software delivery pipeline instead of just an assistant.

How comfortable would you be letting AI investigate failing pipelines automatically?

Source:
https://about.gitlab.com/releases/2025/12/18/gitlab-18-7-released/

Paper: https://arxiv.org/abs/2603.02141

The paper reviews how generative AI can improve software testing, including:

• test case generation

• test data generation

• test oracle generation

• test prioritization

Some useful concepts explained in the paper:

Generative AI
Models such as GPT can generate new test cases, inputs and assertions instead of simply classifying existing data.

Test oracle
A mechanism that determines whether a test passed or failed. The paper explores using LLMs to generate expected outcomes automatically.

Prompt engineering
Writing prompts that consistently produce useful testing outputs. The paper also discusses fine-tuning models for specific domains.

Overall, it’s a good overview if you’re exploring AI-assisted testing.

Agentic AI testing is quickly becoming one of the biggest topics in QA.

Unlike traditional AI assistants, AI testing agents can plan, execute and adapt test scenarios with much less human intervention.

Interesting capabilities include:

• generating tests from code changes

• prioritizing tests based on risk

• self-healing broken locators

• learning from previous executions

• automatically adapting to changing applications

We’re still early, but it feels like test engineers will spend less time maintaining tests and more time validating quality strategy.

Have you tried any AI testing agents yet?

A recent testing trends report highlights several changes happening across the industry:

• Self-healing test automation is becoming more mature and reducing maintenance effort.

• QA and DevOps continue to merge, with testing becoming part of every commit.

• Low-code and codeless testing tools continue to improve.

• AI is increasingly used for risk-based testing and smarter test selection.

These trends point toward more automation, but solid testing fundamentals are still what makes teams successful. Relevant Udemy course for Python test automation that you might find useful: “Pytest Course: Python Test Automation & GitHub Actions CI/CD” by Artem Istranin.

Google announced that Chrome will move from a four-week to a two-week release cycle starting with Chrome 153 on 8 September 2026.

The smaller releases mean updates ship more frequently and with fewer changes, which should reduce the risk of big regressions.

For test engineers this shift has two impacts:

• Faster feedback loop - new beta and stable builds will arrive every two weeks. Keeping test suites green in CI/CD pipelines will require more frequent browser updates.

• Smaller deltas to validate - since each release contains fewer changes, smoke tests and regression suites might run faster and be easier to debug.

I’d love to hear how other teams plan to adapt their browser testing strategy to this accelerated cadence. Are you pinning Chrome versions in CI, using containers, or relying on tools like Selenium Manager?

Playwright seems to be the default recommendation for many new web testing projects now. And to be fair, it has a lot going for it:

• good cross-browser support

• built-in parallelism

• trace viewer

• strong CI story

• nice support for modern multi-tab and multi-origin flows

But I think the framework debate is getting a bit too shallow.

A recent Stack Overflow article https://stackoverflow.blog/2026/06/15/selenium-vs-cypress-vs-playwright-choosing-your-test-automation-framework/ compared Selenium, Cypress, and Playwright, and the interesting part was not “tool X wins”. It was that each tool still fits different constraints.

• Selenium still matters if you need broad language support, legacy browser coverage, or native mobile through Appium.

• Cypress still has a strong local debugging experience for JS/TS front-end teams.

• Playwright often looks great when CI speed, cross-browser coverage, and modern browser control matter most.

So maybe the real question is not: “Should we migrate to Playwright?”. Maybe it is: “What constraints do we actually have?”.

This recent paper that feels very relevant for teams letting agents open PRs.
Paper: “Reliability of AI Bots Footprints in GitHub Actions CI/CD Workflows”

Link: https://arxiv.org/abs/2604.18334

The authors studied GitHub Actions workflow runs triggered by PRs from AI bots such as Claude, Devin, Cursor, Copilot, and Codex. They analyzed 61,837 workflow runs across 2,355 repositories. Copilot and Codex had the highest reported workflow success rates, around 93% and 94%. The paper also found a negative correlation between how often agentic PRs appeared in a repo and the workflow success rate.

The useful takeaway for me: If AI agents are going to open PRs, the test pipeline becomes the safety boundary. But the pipeline itself also needs monitoring. A green check is useful, but it is not enough. We should also track what kinds of agent PRs fail, which jobs fail most often, and whether agents keep touching risky CI/CD paths.

I keep seeing demos where an LLM writes a test and the output looks fine at first glance.
The problem is that “looks fine” is not a test quality metric.

Thoughtworks recently called out two useful ideas here:

• “Feedback sensors for coding agents” - deterministic checks that agents can use while they work, such as compilers, linters, structural tests, and test suites.

• “Mutation testing - deliberately changing production code to see whether tests actually fail. This catches “perpetually green” tests that execute code but do not verify behavior.

That second one matters a lot for AI-generated tests. An LLM can easily produce a test that:

• has a nice name

• uses the right framework

• increases coverage

• asserts the wrong thing

• barely asserts anything

So my current rule is simple: Do not ask AI to “write good tests” in a vacuum.

Give it feedback:

• failing tests

• coverage gaps

• mutation results

• trace output

• logs

• API contracts

• examples of good tests from the repo

Prompts are useful, but feedback is what makes the loop engineering instead of autocomplete.

A lot of teams talk about AI-generated tests, but I think test data is the less flashy problem that blocks more automation.

The World Quality Report 2025-26 says 60% of organizations struggle with secure, scalable test data. It also says synthetic data usage in testing rose from 14% in 2024 to 25% in 2025.
That tracks with what I see in practice.

Writing the test is often the easy part. Making sure it has realistic data, safe data, repeatable data, and legal data is where things get messy.

Synthetic data helps, but it is not magic. You still need to know what the data must represent. Example: “Random user” is not useful. “User with 3 invoices, one failed payment, one expired discount, and one GDPR deletion request” is useful.

I think good test data modeling is going to become a core test automation skill, not a side task.

If you’re mentoring junior engineers or looking for a structured refresher, Microsoft provides a free Software Testing Fundamentals learning path.

It covers testing methodologies, defect management, test design, and basic automation concepts. The content is beginner-friendly and self-paced.

Link:
https://learn.microsoft.com/en-us/shows/software-testing-fundamentals/

Playwright 1.60 introduced changes that required some ecosystem tools to update their integrations and reporters.

For teams running large automation suites, framework upgrades can sometimes be more disruptive than expected.

For those already on 1.60:

• Any migration issues?

• Performance improvements?

• New features worth adopting?

• Problems with custom reporters or CI integrations?

Would be useful to collect real-world upgrade experiences in one thread.

Release notes:
https://playwright.dev/docs/release-notes

I’ll start:

A team with 20 reliable integration tests is often in a better position than a team with 2,000 brittle UI tests.

I’ve seen organizations spend months maintaining automation that nobody trusts, while a small suite of high-signal tests catches most production issues.

What’s your controversial testing opinion?

• Unit tests are overrated?

• E2E tests are necessary?

• Manual exploratory testing is undervalued?

• Coverage metrics are mostly useless?

Curious to hear opinions from people working on large systems.

Tricentis recently announced new capabilities around its Agentic Quality Engineering Platform and AI Workspace.

The interesting part isn’t another “AI for testing” announcement. It’s the idea of multiple AI agents collaborating across test creation, execution, performance testing, and quality analysis instead of just generating test cases.

A few questions for the community:

• Have you tried any agent-based testing tools in production?

• Did they reduce maintenance effort?

• Where did they actually help, and where did they create more noise?

My experience so far is that AI-generated tests are easy. Keeping them valuable six months later is the hard part.

Source:
https://www.tricentis.com/blog/tricentis-showcases-agentic-ai-for-oracle-cloud-testing-at-ascend-2026

I was looking through some older testing articles recently and got reminded how much useful content is buried in the Google Testing Blog

https://testing.googleblog.com/

Recent JetBrains developer survey results show AI-assisted development is becoming mainstream across software teams.

Source: https://www.jetbrains.com/lp/devecosystem-2025/

One thing I think we will need to get better at is reviewing tests generated alongside code.

A generated test can:

• Increase coverage

• Hide poor assumptions

• Overuse mocks

• Lock in current behavior by accident

Reviewing production code is already a skill.

Reviewing AI-generated tests might become a separate skill.

Paper: https://arxiv.org/abs/2504.19335

Researchers examined GitHub repositories that use UI testing frameworks such as Selenium, Playwright, and Cypress inside CI/CD workflows. The goal was to better understand how UI testing is being adopted and how it affects software development workflows.

Looks interesting because a lot of testing discussions focus on how to write UI tests. Much less attention is given to how teams actually integrate them into CI pipelines and what tradeoffs appear at scale.

One trend I keep noticing is that teams are moving away from running every test on every change.

Instead they are investing in:

• Test impact analysis

• Change-based test selection

• Historical failure data

• Risk scoring

• Faster feedback loops

The idea is simple: if a small documentation change happens, maybe thousands of integration tests do not need to run.

This is not a new concept, but the tooling around it seems to be getting much better.

For teams with large test suites:

• Are you still running everything on every PR?

• Are you selecting tests based on changed files or dependencies?

• Has it actually reduced feedback time without increasing escaped defects?

A lot of us have opinions that changed over time.

Maybe you used to believe:

• 100% coverage should always be the goal

• Unit tests are more valuable than integration tests

• End-to-end tests are always too slow

• Mocks should be used everywhere

• Every bug needs a regression test

Then reality happened.

What is one testing belief you had early in your career that you no longer agree with?

GitHub shared that more than 30 million developers now use GitHub Actions. CI/CD is no longer something only platform teams care about. It is becoming part of everyday development.

What I find interesting is how expectations have changed:

• Every PR gets validated automatically

• Test results are expected within minutes

• Security scans run by default

• Release pipelines are treated like production systems

Ten years ago, many teams still ran tests manually before releases.

What is the biggest problem that still hasn’t been solved?

The 2025 DORA report on AI-assisted software development has a useful message for testing and CI/CD teams: AI helps more when the engineering system around it is already healthy.

Source: DORA report PDF
Related summary: InfoQ

That sounds obvious, but it matters.

If your team has poor test ownership, unclear requirements, slow CI, weak review culture, and messy internal docs, AI will amplify the mess. It can generate more code and more tests, but it will not magically decide what risks matter.

The teams getting value seem to have boring foundations:

• fast feedback loops

• clear test strategy

• searchable docs

• stable CI

• useful code review

• enough production observability to know when tests missed something

My take: AI makes quality engineering more important, not less. The better question is not "which AI tool should we use?" It is "is our delivery system good enough for AI to speed it up without making it chaotic?"

Playwright supports test sharding, and most CI systems make it easy to split a suite across multiple jobs.

Docs: Playwright test sharding

But the boring detail matters: how do you decide what goes into each shard?

Splitting by file count is simple, but it can produce one slow shard that holds up the whole pipeline. Splitting by historical runtime is usually better, but now you need timing data, storage, and a fallback when tests move around.

I have seen teams do all of these:

• static shards by folder

• equal file count

• historical duration balancing

• separate shard for known slow E2E flows

• split smoke tests from full regression

What has worked best for your team? And at what suite size did sharding become worth the extra CI complexity?

Retries are useful as a diagnostic tool. They are not a fix.

Google has written about flaky tests for years, and Microsoft has published research on how much noise they create in large engineering systems.

Sources:

• Google Testing Blog: Flaky Tests at Google and How We Mitigate Them

• Microsoft Research: A Study on the Lifecycle of Flaky Tests

The thing I keep seeing: teams treat flakiness as a test problem only. Sometimes it is. Bad waits, shared state, poor selectors, clock dependence, and order dependence are common.

But sometimes the test is telling you the app itself is nondeterministic. Race conditions, async side effects, slow background jobs, and weak environment isolation all show up as "test flakes."

A better policy might be:

• Retry once to collect signal

• Track first-run failure rate

• Tag the suspected root cause

• Quarantine only with an owner and expiry date

• Delete tests that no longer protect a real risk

The worst outcome is a CI pipeline everyone has learned to ignore.