The Clean Test Automation Monthly is a monthly blog post that shares interesting or useful test automation content that I consumed during the current month. This blog post is always published on the first day of the "next" month.
Let's begin!
Test Design
Self-Healing Tests with AI: Triage Before Repair explains when self-healing tests can be useful, argues that we shouldn't blindly heal all failing tests, and demonstrates how we can implement self-healing tests with AI.
Testability: The tests aren’t enough identifies the key practices needed to create a useful regression test suite and explains how a successful test suite becomes useless if the team doesn't consistently follow these practices.
Mutation Testing for Agent-Written Code provides a quick introduction to mutation testing and explains how we can use it to validate and improve AI-generated tests.
Test Data Three Ways divides test data into three categories, explains the purpose of each category, and identifies the key characteristics of test data that belongs to each category.
TDD inside the agent loop - theater or actual value? explores how we can use AI agents for test-driven development and evaluates if this approach is worth it.
Lessons Learned from Fixing Flaky Tests with Claude describes how the author used Claude to fix flaky integration tests which appeared when he started running his integration tests in parallel.
Test coverage does not tell you whether a change is safe enough argues that we shouldn't rely only on test coverage when we evaluate if a release is safe and introduces five factors that help us to evaluate risk.
Test-Driven Review: Reading the Tests First argues that we should review AI-generated tests before we review the implementation and identifies three things we should check when we are evaluating the generated tests.
Should testers write unit tests? explores this age-old question and argues that we should concentrate on collaboration instead of putting people into arbitrary silos.
Backend
API Mocking and Testing With Microcks provides a quick introduction to Microcks and explains how we can generate mocks from an OpenAPI specification and use Microcks for running contract tests against a real backend.
Do We Still Need to Write API Tests? An Experiment with LLM Agents Testing a REST API documents an experiment where the author used AI agents to test a REST API and evaluates if AI agents are a viable option for writing automated API tests.
The Invisible Difference: Unicode Normalization in Test Assertions explains why a text assertion can fail even though the two strings look the same on screen and argues that we should solve this issue by adding Unicode normalization to our application code, not to our test code.
UI / End-to-End
Balancing Playwright Test Shards explains how we can cut the execution time of our Playwright tests by leveraging historical test duration data when we ensure that all of our test shards have a balanced workload.
Your green tests are lying identifies some anti-patterns which produce hollow Playwright tests that cannot fail and demonstrates how we can use an inverted assertion for verifying that a test can actually fail.
Should Page Objects Assert? Where Test Assertions Belong argues that we should put assertions to our test methods (or functions) and add technical guards to our page objects.
How to Make Playwright Tests Less Flaky: Practical Rules That Actually Work shares 15 practical tips that help us to eliminate flaky tests from our Playwright test suite.
Network Interception with Playwright TypeScript provides a quick introduction to network interception and describes how we can control, inspect, and manipulate network traffic with Playwright.