Lately, Selenium became a go-to tool for almost every QA and SDET. You probably will have a hard time finding a QA/SDET job ad which doesn’t include Selenium in the job description. In one of my previous companies, even manual testers without any programming or automation knowledge and experience were advised to use at least Selenium IDE while developers were pushing them to use their own ‘special’ Selenium testing frameworks.
I have spent some time working as an automation engineer in one of the leading Scandi banks. I was hired to work mainly with JAVA/Selenium, one of my main goals was to help the team to develop and set up a functional, stable (haha) and maintainable Selenium testing framework. In addition, the project owner that I was working for wanted me to automate massive regression tests explicitly using Selenium. So I had no choice, but to put all of my effort and heart into the new testing framework. Luckily I had amazing colleagues who were making great progress on that. So this is my background and how I came to learn Selenium.
I have to say automating huge regression test which has about 30-70 input fields and several different scenarios which varies at runtime was more than challenging. Having a Jenkins job which takes about an hour to run was even worse, especially when it fails so often and the developer of the case has to go through terrible, terrible console output to find hints why the case might have failed and finally connect the points and draw the conclusions. Not to mention constant changes in requirements during the developing process or even post-dev stage.
This for me is a great example when Selenium is overused and overrated. Everyone who used it knows that it’s slow and very unstable, however, it is very easy to fall in the trap of overusing it, since it’s so powerful when it comes to front-end testing.
Test automation is all about the speed and stability, as a QA/SDET, you have to deliver as much information about the state and quality of the product ASAP, to both sides (developer and product owner/stakeholders). If your test lacks speed then obviously the response time suffers greatly, no feedback from devs means that they’ll move on to the other task. This slows down the sprint, makes QA and DEV process more complicated since it’s harder to communicate. The other important factor is stability, if your tests are flaky and unstable, no one will take them seriously and this happens way too often. Unstable autotest equals to no test, you have to avoid that at all costs. So that only means that having good AT’s, Binding tests and UT’s for the developers is much more important than any other information.
What do product owners want from your tests? That’s a good question, most of the time they’ll only need a confirmation that everything works. If you have a non-technical PO (like I did once) it will be harder. They don’t care that much about the speed, however, they care about the quality of the test, stability and most of all a report which looks cool. This is exactly what Selenium does (except stability), it has great reporting tools and there’s nothing better for someone who’s not technical to see how the test executes on the screen or knowing that it’s possible to screenshot everything and basically imitate the user. Product owners (non-technical) will be too excited for all of that and that’s usually the reason why Selenium is sometimes overused and overrated.
Don’t get me wrong I’m not saying that Selenium is a bad tool and you shouldn’t use it. It is a great and powerful tool (especially for performance and integrations tests) but it has it’s used and trying to make it work for AT’s or covering everything is not efficient, stable or fast. I would always use Selenium as the last ditch effort in order to just to make sure that everything is in places. Integration test shouldn’t exceed 30 mins and should have a clear indication of the outcome. If it failed then it should be red and fixed immediately, leaving it red and ignoring the issue is the same thing as not having a test at all.