When To Use Selenium

Lately, Selenium became a go-to tool for almost every QA and SDET. You probably will have a hard time finding a QA/SDET job ad which doesn’t include Selenium in the job description. In one of my previous companies, even manual testers without any programming or automation knowledge and experience were advised to use at least Selenium IDE while developers were pushing them to use their own ‘special’ Selenium testing frameworks.

I have spent some time working as an automation engineer in one of the leading Scandi banks. I was hired to work mainly with JAVA/Selenium, one of my main goals was to help the team to develop and set up a functional, stable (haha) and maintainable Selenium testing framework. In addition, the project owner that I was working for wanted me to automate massive regression tests explicitly using Selenium. So I had no choice, but to put all of my effort and heart into the new testing framework. Luckily I had amazing colleagues who were making great progress on that. So this is my background and how I came to learn Selenium.

I have to say automating huge regression test which has about 30-70 input fields and several different scenarios which varies at runtime was more than challenging. Having a Jenkins job which takes about an hour to run was even worse, especially when it fails so often and the developer of the case has to go through terrible, terrible console output to find hints why the case might have failed and finally connect the points and draw the conclusions. Not to mention constant changes in requirements during the developing process or even post-dev stage.

This for me is a great example when Selenium is overused and overrated. Everyone who used it knows that it’s slow and very unstable, however, it is very easy to fall in the trap of overusing it, since it’s so powerful when it comes to front-end testing.

Test automation is all about the speed and stability, as a QA/SDET, you have to deliver as much information about the state and quality of the product ASAP, to both sides (developer and product owner/stakeholders). If your test lacks speed then obviously the response time suffers greatly, no feedback from devs means that they’ll move on to the other task. This slows down the sprint, makes QA and DEV process more complicated since it’s harder to communicate. The other important factor is stability, if your tests are flaky and unstable, no one will take them seriously and this happens way too often. Unstable autotest equals to no test, you have to avoid that at all costs. So that only means that having good AT’s, Binding tests and UT’s for the developers is much more important than any other information.

What do product owners want from your tests? That’s a good question, most of the time they’ll only need a confirmation that everything works. If you have a non-technical PO (like I did once) it will be harder. They don’t care that much about the speed, however, they care about the quality of the test, stability and most of all a report which looks cool. This is exactly what Selenium does (except stability), it has great reporting tools and there’s nothing better for someone who’s not technical to see how the test executes on the screen or knowing that it’s possible to screenshot everything and basically imitate the user. Product owners (non-technical) will be too excited for all of that and that’s usually the reason why Selenium is sometimes overused and overrated.

Don’t get me wrong I’m not saying that Selenium is a bad tool and you shouldn’t use it. It is a great and powerful tool (especially for performance and integrations tests) but it has it’s used and trying to make it work for AT’s or covering everything is not efficient, stable or fast. I would always use Selenium as the last ditch effort in order to just to make sure that everything is in places. Integration test shouldn’t exceed 30 mins and should have a clear indication of the outcome. If it failed then it should be red and fixed immediately, leaving it red and ignoring the issue is the same thing as not having a test at all.

68747470733a2f2f646f63732e676f6f676c652e636f6d2f64726177696e67732f642f316c4972645a3052526b314a3175596476727247316e4577783752304d6c4f55756745314d6768745630776b2f7075623f773d39363026616d703b683d373230

ReportNG

reportng

Reporting plays a very important part of test automation and ReportNG is a very helpful TestNG plugin which is powerful and easily customizable. Actually, the extent of customization is huge.

This is going to be a quick intro tutorial to ReportNG, my goal is to quickly show how to set it up and add basic functionality.

How to set up?

Download reportNG library

Download Guice

Download Velocity dep

  • reportNG – HTML report plugin for TestNG
  • Guice – Dependency injection framework
  • Velocity dep- Java-based template engine

Next step.

Click on your testNG test case file which you want to run and find “TestNG” and click on “Convert to TestNG

reportng

Now you’ll see the generated XML windows, you’ll have to add these four listeners to the XML file.

So the generator screen will look like this:

Capturelistners

Now when you have generated an XML file, click on XML file and choose TestNG -> Run as TestNG

So this will create a new directory in your project:

test-output -> HTML

Here you’ll find index.html which will have the basic reportNG template.


Improving ReportNG

You can add screenshot and text to the file by using Reporter.log(“””) this is TestNG specific function which adds a line to the report file. Since we are using ReportNG which is HTML5 compatible, we can easily add short HTML “scripts” and it will execute.

Add an Expand/Collapse Screenshot:

Reporter.log(<details> <summary>Click to expand</summary><img alt="" /></details>");

Just change pathToScreenshot to your full path which includes the file name. the expand collapse will mainly work only on chrome.


Advanced stuff

You can achieve much more and add almost anything to the report file by downloading the git source from
ReportNG Git

Most of the stuff is located in ReportNG-base.java, however, you can find useful stuff through all of the projects. You can inject jQuery library and use it to add the script to the template so it’s possible to make these expand/collapse images for all of the browsers, also you can edit almost everything there, adding your logo, descriptions, CSS improvements and jQuerry components.

reportNGexample

Handling tables

Handling tables is something that I saw a few people on the internet and in my company have struggled a couple of times. You can do so many things with tables if it’s done right. So in this short post, I will talk about handling elements within the table.

In one of the previous blogs, there was a snippet how to extract all elements to the list. When you have it on the list you can make all kinds of stuff. For example usually, you would need to check or assert if table element contains some kind of text or if it has changed. When you get the list of full table elements just go through it and assert/check what you need.

		for(WebElement e : list){
			if(e.getText().contains(fileName)){
				return true;
			}
		}

That’s an easy one. But because you add it as a full web element it maintains all of the functions of web element so you can do all kind of stuff with it. Also, because you are in a loop and going through these elements one by one it means that the selected element is isolated and there should be no fear to use some stuff which would be considered dangerous in the plain site, outside the element.

Example:

		for(WebElement e : list){
			if(e.getText().contains(fileName)){
				e.findElement(By.tagName("input")).click();
			}
		}

As you can see I’m not afraid to use tag name input, which in this case mean checkbox because the WebElement is isolated and I’m searching stuff within it.

XPath/CSS int the table

If you are trying to click table link or checkbox and you can’t find anything to catch on don’t be afraid to use regular for loop.

		for(int i = 0 ; i &lt; list.size() ; i++){
			if(list.get(i).getText().contains(fileName)){
			driver.findElement(By.xpath("//td[@id='table:"+i+":subT_rowHeader']")).click();
			Thread.sleep(3000);
			}
		}

This is a good example how can you go through the table elements and click certain things that are hardly reachable, however you’ll need to spend some time finding out the best CSS/XPath pattern for the checkbox/link/expand arrow.

All of these examples work well even if used on a grid with multithreading and the table is manipulated, expanded, customized so take care!

Get all data from tables

This is a simple solution to get all of the data from a table as web element. Later on, when you get the web element you can manipulate it as much as you want, select, compare, assert.

public List extractTableElementsToList(ETable tbl){
    System.out.println("Entering extract all elements from the table via helper.");
    List allTable = null;
    allTable = tbl.findElements(By.tagName("tr"));
    return allTable;
}

This will return the list with all visible table elements.

If you have a case that your table expands and reveals more content after you click on previous elements, you can always get the list of table child elements by using a simple script:

public List extractTableChildElementsToList(ETable tbl){
    System.out.println("Entering extract all elements from the table.");
    List allTable = null;
    allTable = tbl.findElements(By.tagName("td"));
    return allTable;
}

It’s essentially the same but uses TD instead of tr which indicates a child in a table, so this collects all of the visible table child elements and adds it to the list.

Auto download in Firefox

So today I came up with a few challenges, uploading and downloading files from an external server.

Uploading wasn’t that difficult, I have used awt.Robot which handled windows native window pretty well, although it’s very sensitive and I would only advice to use it when you are running test cases on your own dedicated virtual/physical computer.

Downloading was even more tricky. Especially because my framework uses a threaded browser which besides profile requires some capabilities to be implemented.

This is the solution:

firefoxProfile.setPreference("browser.download.folderList",2);
firefoxProfile.setPreference("browser.download.manager.showWhenStarting",false);
firefoxProfile.setPreference("browser.download.dir","\\\\bXXXXX\\workspace\\TEST_DATA\\savedFiles");
firefoxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk","text/csv, text/plain, text/txt, text/html, application/octet-stream");

What does it all mean?
this is explanation line by line:

    • browser.download.folderList tells it not to use default Downloads directory
    • browser.download.manager.showWhenStarting turns of showing download progress
    • browser.download.dir sets the directory for downloads
    • browser.helperApps.neverAsk.saveToDisk tells Firefox to automatically download the files of the certain type

here are a few useful types for the last line:

  • text/html for normal web pages
  • text/plain for plain text
  • application/octet-stream meaning “download this file”
  • application/x-java-applet for Java™ applets
  • application/pdf for Adobe® PDF documents.

Is element visible?

For me and I believe for any decent automation framework is very important to have a reliable method which checks elements visibility before interaction and returns a boolean value instead of exception. Even though it’s an easy solution but for those who just started it will save a lot of time.

this is a neat and very useful method

 

public boolean isElementVisable(By by){
return driver.findElements(by).size() > 0;
}

 

However if you are using PageFactory this gets a bit tricky since WebElement doesn’t have .size() method and getSize returns coordinates which are not useful to make a check. I have tried to make a method avoiding exceptions but in this case it seems that you’ll have to handle it.

 

public boolean isElementVisable(WebElement element){
 try{
   return element.isDisplayed();
 }catch(NoSuchElementException e){
   return false;
 }
}

 

I know that it’s a bit messy solution since exceptions are involved, also webdriver can throw ElementNotVisibleException or something else so this makes it hard to control, unless you throw in a generic exception or catch-all possible ones.

Select By Partial Text

Select in Selenium works very well, but only if you know what you are looking for. If you are working with manual testers who migh make mistakes defining variables or dealing with very huge strings or foreign language this solution should be very handy to you.   Ofcourse for that you’ll need Selenium WebDriver and […]