To verify that a web application really does what it is supposed to do, sometimes it is not enough to use the built in functions of Selenium. In some cases you don’t just want to know whether Text “xyz” exists or HTML-element-abc is present. You really want to know whether your website looks like you want it to. In most cases you don’t care if there are a few pixels that differ, so you don’t want an exact screenshot comparison but a fuzzy screenshot comparison.

The Selenium WebDriver offers a built in function to take screenshots. This example is from stackoverflow:

WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com/");
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
// Now you can do whatever you need to do with it, for example copy somewhere
FileUtils.copyFile(scrFile, new File("c:\\tmp\\screenshot.png"));

What Selenium doesn’t offer out of the box is a function to compare two screenshots in order to know whether something has changed or not. We wrote our own Java-Library to do this fuzzy screenshot comparison for us. We do not provide any support for this library, but feel free to download and use it as it is:

Download:

image-comparison-1.0.jar 2015-06-23

Basic usage example:

import java.io.IOException;
import org.frontendtest.components.ImageComparison;

public class RunComparison {
	public static void main(String
[] args) throws IOException {
		String imgOriginal = "original.jpg";
		String imgToCompareWithOriginal = "new_screenshot.jpg";
		String imgOutputDifferences = "new_screenshot_with_changes.jpg";
	
			ImageComparison imageComparison = new ImageComparison(10,10,0.05);
	
			if(imageComparison.fuzzyEqual(imgOriginal,imgToCompareWithOriginal,imgOutputDifferences))
			System.out.println("Images are fuzzy-equal.");
		else
			System.out.println("Images are not fuzzy-equal.");
	}
}

What it does (simplified):

  1. First the two images that should be compared are divided in squares with the width and height, that you defined in the constructor.
  2. For every square in each image an average RGB-Value is calculated.
  3. If the average RGB-Values of the corresponding squares differ more than the threshold that you defined in the constructor (0.05 = 5%), the function fuzzyEqual(…) will return false.
  4. If you passed a path to save an image with the found differences a copy will be save at this path with all the differences marked with red squares.
  5. Note: The sensitivity of the fuzzy-equal-test will be influenced by the defined threshold as well as the size of the squares!

Here are some examples of the comparison results:

We will compare these two images:

Original for Comparison

Original

New Screenshot

New Screenshot

Here are the results:

Square: 10x10pix Threshold: 10% (0,10)

Square: 10x10pix Threshold: 10% (0,10)

Square: 10x10pix Threshhold: 5% (0.05)

Square: 10x10pix Threshhold: 5% (0.05)

Square: 10x10pix Threshhold: 1% (0.01)

Square: 10x10pix Threshhold: 1% (0.01)

Square: 30x30pix Threshold: 1% (0.01)

Square: 30x30pix Threshold: 1% (0.01)

E.g. comparing these images with square-size 30x30pix and a threshold of 5% (0.05) will tell you that the images are fuzzy-equal!

License:

The MIT License (MIT)

Copyright © 2015 Nils Schütte

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

 

This library makes use of the Thumbnailator which is already included and also published under the MIT License.