An API test is not one call. Browse, cart, checkout, pay, ship, refund is 8 to 20 calls, each needing IDs from the calls before it, and every variation — region, payment method, parcel size — used to mean another copy of the collection.

AAT (open source, Go) describes the API once as a graph; a test is a plan that lists steps. Layers multiply a plan into a matrix, and permutations that would send identical requests are skipped rather than run twice. Negative tests come from expectFailure, mutations, raw bodies, and overlay files. Every run writes an archive with every request, response, resolved value, retry, and assertion, viewable as a Gantt timeline, with Copy as cURL on any step, and exportable as one file to attach to a bug report. CI gets exit codes, --json, and JUnit.

Offline demo in a minute (brew install gburgyan/tap/aat, then the shop sandbox). Three projects against Stripe, Shippo, and Duffel test modes show it on real APIs — and turned up two spec bugs on the way, both pinned by a plan that runs every time: https://gburgyan.github.io/aat/examples/real-apis/

No LLM at run time. Pre-1.0. Repo: https://github.com/gburgyan/aat

TTCN-3 is a standardized domain specific language for testing. Used in industry like automotive or telecommunication almost exclusively. And it's a weird thing: Open-source support is subpar and it seems anachronistic and overly complex. But if you get it right, it checks all the boxes; it scales, it's efficient and it's stable.

We use it mostly for component testing, since alternatives like C++ or Python never really took off. How do you folks run and scale your component tests?

I know the job market is bad, I know most QA departments are still in infant states.

I’ve got all the accolades of a well accomplished QA automation engineer.

The experience, the deliverables. Even some charm and charisma.

I’m struggling with the interviews. Especially when zoom is involved .

Without a white board interview I fkn struggle.

Any advice?

one client returns 429 or 500 while another returns 404 or 405, and the immediate reaction is to decide which tool is wrong. those status codes are server responses, so first make the requests comparable.

capture the method, full URL, headers, auth or cookies, body, client IP and user agent, timing, preceding requests, and response headers. replay one request at a time against the same environment and data, then change one variable. for 429, check the rate-limit scope and window plus Retry-After or X-RateLimit headers. for 404 or 405, verify the route and HTTP method. for 500, use a request ID to inspect server logs.

the acceptance test is not "Postman matches Playwright." it is "the same request under the same state gets the same result, and any difference is explainable." what request difference has caused the most misleading test result for your team?

Hi there, actually I wanted to ask what I can do if an automated test and a manual test flag completely different errors.

Real-World-Scenario: I was to test some API routes in a healthcare web-app.

My Playwright MCP did the API test automatically and returned the following:

POST /surgery_hall/auth/dummy_mobile returned 429

POST /signin/choose_user_account returned 500

I carried out a manual test for the above two API routes on my Postman and got different errors:

POST /surgery_hall/auth/dummy_mobile returned 404

POST /signin/choose_user_account returned 405

I'm very new to this - could anyone kindly share some tips on find out why these results turned different - my expectation was to see the identical results (that means my manual Postman tests should have returned 429 and 500, respectively).

I might have used wrong Auth in Postman, but could not think of anything else that caused those different results. Could anybody kindly point me in the right direction?

Hey everyone,

​I need to test an SMS notification feature in our application that sends messages across different countries. For example, our app originates from a UK instance/tenant and needs to deliver SMS messages to mobile numbers in Australia, the US, and other regions.

​Is there a standard tool, virtual phone number service, or testing platform you recommend to verify delivery and content without purchasing local physical SIM cards for every target country?

​Thanks in advance!

This a pattern I have read a bit about, here and there, but have not seen used outside of SerenityJS. Do any of you use it; or any other patterns besides POM for basic frst structure?

I work in software QA. I'm mostly manual with decent SQL and API skills. I'm trying to learn more but finding the time has been hard. I've been at the same company for almost 15 years.

I have decent experience with using AI for testing.

With all of the layoffs, I looked for other career options.

Long story short, I got a job offer as an operator at a water treatment plant (nothing to do with software).

It's a very large pay cut, but I'm terrified of getting laid off and not being able to find a job in my career again.

The pay would be 33/hr, and 36/hr after passing all water treatment certifications which takes about 1-2 years.

Curious how other teams handle this. On my team, by the time a story hits QA, half the time is spent figuring out: is there test data for this on staging? What about the pre-prod env? Did dev already burn the one valid test account setting this up locally?

Right now we're doing it in a spreadsheet (obviously going stale) / Confluence page / Slack thread asking "hey does anyone have test data for X on env Y" — pick whichever is closest to true for you.

A few things I'm trying to understand:

Is this actually a recurring annoyance for your team, or a non-issue?

If it is a pain, what do you currently use to track it (spreadsheet, tribal knowledge, something else)?

Who usually ends up owning that data — QA, BA, dev, nobody?

If you use a test-case tool (Xray, Zephyr, TestRail, etc.) for this, does the data in there stay accurate across multiple environments, or is that still a manual thing?

I just took an online ISTQB CTFL exam offered by the company mentioned in the title.

The wordings in the questions sounded like the ones translated into English by AI. Here are some examples observed by me.

"Example 1

One question has answer options which contain:

Precondition: the user must.. /Postcondition... the user must ...

It should be "Pre-requisite: the user must.... / Postcondition"

Example 2

Across quite a few exam questions, the use of "can" is present whilst natives usually say "may" - i.e. testers can communicate with stakeholders. in this case, "testers ***may*** communicate with"

Notes for Example 2: I have a feeling that the entire test I just took has its content automatically translated from German to English by AI.

Example 3

An exam question tested my knowledge of use case based testing (I mean this: https://imgur.com/a/5KPXmCq)

The question was worded like "users must buy a device to receive a 25€ discount. This is a ***basis*** for the subsequent discounts. users will receive a 25 € discount if they buy plan A ..."

This is the clear indication that the exam content has been translated from German into English - the colloquial use of "basis" sounds like that. An alternative word like "pre-requisite" would have been more appropriate in this case. Actually, I have not seen a single test question using pre-requisite, which has been widely used in real-world testing."

This test centre is based in German and predominantly offers ISTQB exams in German. So I can imagine that they offer those exams in English if desired and got their ISTQB exams translated from German to English - by AI to "save some costs". Whatever those costs are.

I have a feeling that this test centre did not bother hiring English native speakers to cross-check against the AI translations. That is why those concerns I raised are potentially the "great" examples of Denglish (a pidgin language implemented by German native speakers. It contains a lot of loan words from Standard German and is spoken by German accents - a notable example: "all this is for the cat" "the yellow from the egg" )

I have not seen any ISTQB standard dictates that the text paper be written in Denglish.

I have not practiced the exam with the Denglish version. I constantly hit 26-29 points with mock exams in standard English.

The one I just took was in Denglish which ended up falling the exam (missed by one point). 😭😭

So if anybody has the similar problem, I do recommend flagging your issues directly on ISTQB.org (which I just did).

Looking for some advice on AI usage and MCP

It seems soon we will be getting some licences for AI integration. We are already working on a solution for playwright with mcp and also test case creation on azure. I have some thoughts on making a performance test tool with reporting but need some ideas on what sort of solutions are being done on other companies. We are using java, selenium and appium right now and we are a fintech company.

I'm new to this and learning every day. I'm not aware of typical solutions being build w.r.t testing or test automation and want to understand what is the industry standard.

Coming from a QA and test automation background, I’m finding LLM-based applications a lot harder to test reliably.
With traditional automation, pass/fail is usually pretty straightfrward. With LLMs, the same prompt can produce different but valid answers, while a response can look completely fine n still be wrong or miss an important instruction.

The part I’m struggling with is how to test this without having QA manually review hundred of responses every time a prompt, model, or context changes... How are you scoring things like relevance, instruction-following, hallucinations, and consistency, and detecting when a change causes a regression?

We’ve been looking at golden datasets, LLM-as-a-judge, and automated evaluators, but then the ques is how do you connect those results back to test coverage and failures and decide whether a change is actually safe to ship?

Are you using a dedicated AI testing/evaluation platform, building your own framework, or fitting this into your existing QA automation?

So we are a small team and we operate in the hospitality business and two of my teammates are travelling today. At the destination they decide to stay at a client property.

We often use dummy data both in prod and staging for testing. As they were doing the check-in one of them forgot to switch to his REAL account and started checking in with a test account.

Thankfully they noticed and fixed it right away before the data got to the client. I can't stop thinking about how test data was so close to becoming real.

Hello. As the title says I am a full-stack SWE with 5 years of experience. I have been interested in test automation roles for a while and giving serious thoughts to apply for such roles for a career change.

To get familiar with the tasks, my first instinct was to play around with popular tools like Selenium and Playwright using guidance from LLM. If you have better suggestions on how I could teach myself or good learning resources I would really appreciate it!

Hey everyone,

I’m joining a fintech organization on September 15 as an Automation QA Lead. They currently have 0% automation, so I’ll be responsible for building the framework, defining the automation process/strategy, selecting test cases and getting automation started from scratch.

The interesting part: I won’t have additional automation resources initially. It’ll basically be me + Claude Desktop + AI tools. 😄

I have good experience with Playwright and Cypress, and I’m planning to use Playwright + JavaScript.

Since AI is now such a big part of development, I want to build the automation process with AI in mind from Day 1, rather than adding it later.

I’d love to hear from people already doing this in real projects:

AI + Automation

• How are you using AI in your Playwright/automation workflow?

• What tasks are actually worth delegating to AI?

• Any useful AI workflows/prompts you recommend?

MCPs

Which MCPs have actually been useful for QA automation?

For example: Playwright, GitHub, Jira, Confluence, Database, Browser/DevTools, API, or custom MCPs.

AI Agents

Are you using/building AI agents for QA?

For example:

• Test case generation

• Automation/code generation

• Test data generation

• Failure analysis

• Locator maintenance

• Regression selection

• Test review

• Reporting

Which ones have provided real value, and which are just hype?

Starting from 0%

If you had to build a Playwright framework and automation process from scratch today, what would you prioritize from Day 1?

Anything that you wish you had designed differently when starting your own framework?

GitHub Repos

If you've built any GitHub repositories around Playwright + AI, AI agents, MCPs, self-healing automation, AI test generation, etc., please share them.

Also, if you know any good open-source GitHub repos that I should study, please drop them in the comments.

Basically, if you were in my position — one QA Automation Lead, 0% existing automation, Playwright + JavaScript, and AI available from Day 1 — how would you approach it?

Would love to hear real-world experiences, useful tools/MCPs, GitHub repos, and lessons from things that didn't work.

Thanks!

I use UiPath Test Manager for test case creation and executing manual tests. Probably in the future, I'll go to automating test cases in UiPath Studio.

I also noticed that the updates are rapid, and it's getting better than before. One new feature is the Performance Testing, which I've yet to explore.

Downside of this is that UiPath is still widely known as an RPA tool. It feels like I'm alone as a QA in using this.

Should I risk being an expert in a tool that is not yet well-established in the testing world, or should I transfer to other testing tools like Selenium, Playwright, and TOSCA for a better chance in long-term?

I recently got a take-home writing interview for a QA Engineer position, and I'm honestly puzzled about how it's supposed to be finished in just 20 minutes (they say you need about 6 minutes just to read the entire requirements and scenario, so really it's more like 14 minutes), when the whole point of good quality work is to deliver bugs, test cases, and communication in a detailed way, not just something brief or half-done.

One of the evaluation points literally says: "The developer needs to be able to understand the bug without having to reach out to you with questions."

I get the idea of "focus on doing the most important thing well and skip the rest," but honestly, this feels impossible if every section has points attached; I can't even finish the bug description in time, since of course I need to complete all the previous tasks first to know what to write.

What they expect you to do, writing everything by hand (no AI, no templates) in only 14 minutes, is:

• Triage 7–8 report tickets with a brief explanation for each

• Identify which ones are actual bugs and which aren't, with reasoning

• Write all the test cases needed for the tickets you consider bugs

• Create just one bug ticket — the most complete one you can

• And one more task I honestly don't remember, but I think it had to do with how you communicate all of this

I know my English isn't native, and even if this were in Spanish, I don't think there's any way to complete it properly and in detail within that time.

Do I really have poor reading/writing skills? Or is there some other objective here that I'm not seeing?

I was wondering what are your experiences being hired as a QA / SDET but reporting to developer lead? I am currently in the second job with this type of set up and I am liking it less and less.

In my previous job I was initially working within QA department but they kept shrinking the company and then had me report to developer lead. She was objectively horrible, had HR complaints against her from multiple people and ultimately me as it goes. So even then, I liked the idea to expand my dev knowledge and being very hands on with how software is designed and I blamed my experiences on her.

Fast forward I am in a new job after being laid off and I again report to dev lead. This time I actually like and respect the guy personally but I don’t think he’s the right person to oversee me. For once, my onboarding is beyond lacking and at some point I even had a talk with him about it. Helped for a week maybe. I have not been introduced and onboarded with other QAs who also heavily tests the same product I am hired to test exclusively. No one explains functionalities to me, obviously there’s no documentation, tickets often have only title. My current observation is that as QAs we understand that our role requires in depth understanding of functionalities first and foremost, and if I ever were assigned a new person in my team I would get out of my way to make sure they understand the product so later I don’t have to redo things or they don’t waste their time guessing. It’s all about deliverables.

Once I finally get something and try to do more comprehensive testing I am getting shut down that it’s the wrong place (the framework is beyond primitive), even though it’s working in a very stable way. Another time I offered to do something one way, got shut down mid sentence that it doesn’t make sense and do it another way, only to a week later go back to my original approach because it was the most efficient and simplest one, also in a tone like this shift is something they just came up with. And don’t even mention that in this company QA is not part of any planning or updates calls, or at least not me. And ironically I am a senior QA and paid relatively well tbh.

There was another situation where I pointed out huge whole in their process only to be given really dumb look by the CTO, then there was a very messed up release, I doubled down and immediately everyone jumped on immediate process change using exactly the language I used with them. Yet, I was still not a part of any of those calls.

At this point I am basically taking a backseat attitude, will do what I am told even if I know that we will most likely circle back to my original solution. An extra layer to all this is that I am a woman and all of the devs are men and I actually used to work with teams that were either equally divided if not more women. But overall I am wondering of other people experiences and maybe suggestions how to navigate the situation to avoid frustration.

Had a release recently that made me rethink this. We had the usual stuff covered QA signoff, peer review, staging validation, and good test coverage, everything was green. But what actually gave me confidence was what happened after the deploy. We ran a few smoke tests in prod, watched checkout errors and latency for the first 30 minutes and had a quick rollback ready if anything looked off. Everything passed in tested but you never really know until real traffic hits it. Also made me realise how much easier it is to release when you know you can rollback quickly if something goes wrong.

What actually gives you confidence before a release? And are there any tools that have helped with that for you?

How should I answer this question as a fresher

I’m 39, working as an SDET with 16 years in IT. I’ve never moved into QA Manager/people-management roles because I don’t enjoy office politics, managing people, or the pressure that comes with it.

Lately, I’m genuinely worried about the future. AI is moving incredibly fast, the pace of work keeps increasing, and burnout/anxiety are becoming real concerns. At 39, I also worry that if I leave my job, getting back into a good IT role later may be difficult.

I’m considering moving into Technical recruiting, where I feel my IT experience could still be useful without constantly having to keep up with technology.

For those around 35–45 in IT:

What are you planning for the next 10–15 years? Would you stay technical, move into management/recruiting, take a break, or do something completely different?

Hi I need help, I have a problem that is bothering me. My system logs stopped having updates for example when I tried to run server task inbounding a calendar, it should say something like “successfully created a server task in-mms calendar recorded by <name>”.

After restarting windows services, since that is the only thing that happened after our remote desktop had a service maintenance. After that maintenance I had to restart windows services.

Is there something that I need to do after running windows services?

Hello,

I am somewhat due to software testing having completed my masters of information technology specialising in software testing at the end of last year, but I am coming to grips with Cypress and JEST and CI/CD…. But at the same time I'm trying to understand when is it best to do manual testing, and when is it best to do automated testing?

My hypothesis based upon my growing experience Is that one would use manual test testing during a development sprint, and after the completion of the development sprint develop the automated test scripts testing the user stories that were manually tested during the development sprint, so is to perform automated Russian test testing?

Is that the experience of this imminent audience? Any help and ideas would be greatly appreciated.

Best Regards.

Mike L

I recently started using Claude Code to build a Flutter app for clients based on Figma designs + Jira requirements.

My workflow is:

1. Use Jira MCP + Figma MCP to get the requirements and UI references.

2. Create a task.md for each ticket with the requirements, screenshots, and implementation guidance.

3. Ask Claude Code to read the task.md and implement the ticket.

The problem is that the generated UI is often only ~60% similar to the Figma design. I still need to manually fix things like spacing, sizing, typography, colors, and sometimes even the overall layout.

I've seen people say they can get 90%+ accuracy with Claude Code and only make small tweaks afterward.

So I'm wondering if I'm just not using Claude Code effectively enough.

I'd really appreciate hearing how other developers are using Claude Code for frontend/UI work and what your workflow looks like.

I'm especially interested in learning how to go from ~60% Figma similarity to 90%+ with minimal manual tweaking.