The Clean Test Automation Monthly is a monthly blog post that shares interesting or useful test automation content which I read during the current month. This blog post is always published on the last day of the month.
Let's begin!
Test Design
Your Performance Tests Are Only as Good as Your Requirements advocates that we should leverage historical production data, understand the nature of peak load, and select our testing strategy based on the production data and the nature of peak load. Finally, the author identifies five questions that help us to write better requirements for our performance tests.
Netflix App Testing At Scale describes the testing strategies which are used to write automated tests for the Netflix Android application which has over over 400 modules and one million lines of Kotlin and Java code. This blog post introduces the different layers of the testing pyramid used by Netflix, highlights the testing tools used to write different tests, identifies the challenges faced by developers, and describes how they were able to solve these problems.
Good Test Automation Doesn’t Start with Code is a thought-provoking article which argues that if you want to write good tests, you shouldn't get obsessed with testing tools, best practices, or writing code. Instead, you should learn to ask the right questions.
Why Property Testing Finds Bugs Unit Testing Does Not explores the limitations of traditional example based tests and highlights the strengths of property-based testing. The author argues that example based tests are effective when the number of inputs is relatively small. However, when the number of inputs (and the number of combinations of inputs) grow, traditional tests often miss the errors which are caught by the randomized inputs used in property-based testing.
AI Test Generation: A Dev’s Guide Without Shooting Yourself in the Foot identifies two common problems often found in AI generated tests, explains how we can catch at least some of these errors with SonarQube, and shares seven tips that helps us to write good tests with AI.
The Testing Tower introduces the concept of testing tower and argues that it's a modern, more expressive, and context-driven replacement for the outdated testing pyramid. The author divides tests into five levels — from fast automated tests which are the foundation of the testing tower to human insight and validation that's found from the battlements of the testing tower.
Backend
Spring Boot AI Evaluation Testing describes how we can build a simple AI agent with Spring AI and explains how we can write automated tests for our AI agent by using a technique called evaluation testing.
Automating Security Testing in CI/CD Pipelines with OWASP ZAP: A Comprehensive Guide describes how we can can configure and run security tests with OWASP ZAP, explains how we can integrate OWASP ZAP with our CI/CD pipeline by using Github Actions, and helps us create a Github Actions workflow which runs our security tests for an application that's running in a staging environment.
Automating Contract Testing: A Developer’s Guide with Spring Cloud Contract provides an introduction to contract testing, introduces the key features of Spring Cloud Contract, and describes how we can write automated contract tests with Spring Cloud Contract.
Spring Boot TestContext Cache Best Practices identifies three common mistakes which cause unnecessary cache misses and shares four best practices which help us to maximize context reuse and improve the performance of our test suite.
Things I Wish I Knew When I Started Testing Spring Boot Applications highlights four things which you should know if you are writing tests for Spring Boot applications.
Writing Unit Test With MockMvcTester: Returning an Object as JSON is my own blog post that identifies what kind of tests we should write, helps us to eliminate duplicate request building code, and describes how we can write unit tests for a REST API endpoint that returns an object as JSON.
PITest — a Hands‑On Guide to Mutation Testing in Java provides an introduction to mutation testing and the PITest library, describes how we can integrate PITest with Maven, and explains how we can optimize its performance using the incremental analysis feature.
Building Cloud-Ready Apps Locally: Spring Boot, AWS, and LocalStack in Action demonstrates how we can develop and test Spring Boot applications which use AWS services in our local development environment with LocalStack and Testcontainers. It explains how we can integrate a Spring Boot application with Amazon SQS and S3, and describes how we can test cloud interactions in a local and cost-effective environment without relying on real AWS resources.
Combine Testcontainers and Spring Boot with multiple containers introduces three different approaches for starting multiple Docker containers with TestContainers when we are writing integration tests for Spring Boot applications. It provides sample code for each explored method, and identifies the advantages and disadvantages of each approach.
Automating Java Style Guide Enforcement with Checkstyle and OpenRewrite describes how we can enforce coding standards automatically with Checkstyle and OpenRewrite. It provides a quick introduction to both Checkstyle and Openrewrite, and explains how we can identify "code style" violations with Checkstyle and fix them automatically with OpenRewrite. Additionally, this blog post explains how we can use these tools with Maven, leverage IDE plugins, and integrate them with our CI pipeline.
Testing MongoDB Atlas Search Java Apps Using TestContainers provides a short introduction to MongoDB Atlas Search and describes how we can write comprehensive integration tests for a service service with JUnit Jupiter and Testcontainers.
Testcontainers + Spring Done Right: Cleaner, Faster, Smarter introduces a new open source project called Spring-TestContainers. The main benefit of this project is that we don't have to write any infrastructure configuration code if we want to start a docker container before our integration tests are run and stop it after our integration tests have been run. This blog post explains why the author wrote the Spring-TestContainers library, describes why we should use it, and helps us to write integration tests for a repository which uses the PostgreSQL database.
UI / End-to-End
Using Playwright Custom Matchers to Automate Layout Testing provides a quick introduction to layout testing, identifies existing layout testing tools, and explains how we can write layout tests which won't rely on visual regression tools (aka compare the expected and the actual screenshot).
Vibe testing with Playwright explains how we can integrate Github Copilot with Playwright MCP server and demonstrates how we can leverage Github Copilot for writing automated tests with Playwright. The author installs the Playwright MCP server, asks Github Copilot to analyze his website and write tests for it, and shares the results with us.
Speeding Up Playwright Tests with Dynamic Sharding in GitHub Actions describes how we can create a Github Actions workflow which improves the performance of our Playwright end-to-end tests by using dynamic sharding. This blog post explains how we can determine the optimal number of shards based on our test count, execute tests in parallel by using the created shards, and merge the test results into the a single HTML report.
Handling Multi-User Flows in Playwright the Right Way describes how we can improve the performance of our Playwright tests and make them less flaky when have to write tests where multiple users invoke the same flow.
Offline but Not Broken: Testing Cached Data with Playwright explains how we can write tests which verify that our web application is working as expected when the user of the web application is offline.