Happy Friday, everyone. Today I am going to talk about visual testing.
I am working on a UI library that requires strict quality checks. That means that any updates to the styles and components code should be isolated and not break any other UI. That means no visual regressions to components that have already delivered if sudden changes to styles take place. As such, I am implementing a way to incorporate visual regression testing into my project workflow.
What is Visual Testing?
Visual testing intersects both Functional and Manual Testing. Functional tests make sure that software is doing what it should. Visual testing makes sure that the software “looks” the way it should. The visual tests generate screenshots of the web application UI during test automation in the CI. The screenshots get sent to the Percy servers, and their cloud infrastructure compares them to the previous screenshots saved as the baseline. If there’s a difference, then Percy gives generates a new image highlighting the differences.
Visual testing addresses unit test brittleness by comparing screenshots between different pull requests in a web app repository. It doesn’t necessarily “fail” an identified difference but rather notifies you that diff was found. The code changes are not merged into the repository. It forces manual human intervention by the developers, designers, or product owners to determine if it is an issue requiring a code change or to create a new baseline. Simple.
Where should Visual Testing be done?
There are different types of automated tests that we can create for web UI projects; unit testing, integration testing, and end-to-end testing. Unit testing is confirming the smallest piece of code works as intended. Integration testing is verifying that multiple code modules work together correctly. End-to-end testing is all about confirming that the complete UI works.
In the Angular world, unit testing would be testing each small piece of the code for the presentation components. Integration testing would be testing the container components, making sure that each presentational component renders appropriately inside the container, depending on different states. End-to-end tests are done on a full-page UI and in the context of the user’s interaction with the real application.
In my experience, it is tough to confirm visual requirements via unit tests or integration tests. In the Angular framework, the best we can do is to confirm that the style classes haven’t changed or that element styles match hardcoded properties. But these tests are incredibly brittle and time-consuming. My recommendation is to do visual tests in the end-to-end testing phase. Not only are these long-running tests going to run in your CI environment, but they will be much more straightforward to implement.
What tool for visual testing?
There are many different tools available to perform visual regression testing. Some of them available as independent libraries or frameworks, while others are SaaS/cloud solutions. Given that I don’t want to spend too much configuration time, I have decided to go with a SaaS solution called Percy.io. I will be leveraging the service for the next couple of weeks to see how it works.
If you have experience working with Percy in a past project, drop me an email and let me know how it went for you.
Have a safe and restful weekend!
– Guillermo
