Using JavaScript In Feature Files to Create Dynamic Cucumber Tests

Sometimes when working with cucumber, the static nature of the tool limits your options quite a bit. For example, you can declare variables but these variables are static. If for example, every time you run a test you want to generate unique variable (i.e ID) you are limited to an option of implementing this functionality in a step rather than the test case itself. In my opinion, all steps should be as generic as possible and follow DRY development pattern, therefore it’s important to exclude some of the code which is specific to one scenario and make steps as generic as possible.

In this blog post, I will try to cover a way of implementing javascript support in your JAVA Cucumber feature files, to make them more dynamic and flexible.

Bellow, you can check an example of how this functionality work. Essentially, when you declare your variable there is an option to include a javascript function which allows you to generate a dynamic variable.

Example:

Feature: Test to load a js file in Genie js context

@load.js.file.in.genie.context
Scenario: Load a local js file in genie js context
	Given I load this js file "/data/test.js" in genie's js context
	Given I define the following user defined variables:
                | currentDate    | $js{getDate()}     |
		| repeatedString | $js{repeat('A',5)} |
		| sum            | $js{sum(4,5)}      |
                | randomNum      | $js{ranNum = Math.floor((Math.random() * Math.pow(10,4)) + 1).toString();} |

How can this be used in a test case? Simple just pass it as variable in the test case:

And I send this event to Kafka:
    | unique_ID_SEND        | name               | sum     |
    | QA_TEST_${randomNum}  | ${repeatedString}  | ${sum}  | 

The trick how to do it is to use the Cucumber data table, convert it into a map, and evaluate every variable to check if it’s a dynamic “$js{}” or it’s a regular one. If it is, in fact, a dynamic js variable, evaluate the expression. A quick snippet from my personal code stack:

@Given("^I define the following user defined variables:$")
public void iDefineTheFollowingUserDefinedVariables(DataTable dataTable) {
   dataTable.asMap(String.class, String.class).forEach((k, v) -> 
   { v = VariableUtil.evaluateEnvVariable(v);
        if (v.equals("${UUID}"))
           v = UUID.randomUUID().toString();
        else if (v.equals("${ShortID}"))
           v = MessagingUtils.getInstance().getShortNumericID();
        else if (v.equals("${AlphaNumericShortID}"))
           v =MessagingUtils.getInstance().getShortAlphaNumericID(15);
        else
           v = VariableUtil.evaluateJSExpressionOnFrameworkVariable(v).toString();

HTMLLogUtil.addLongMessage(String.format("Set the value to UDF variable '%s' as '%s'", k, v));
            setCentralStorage(k, v);
        });
    }
    

As you noticed, when I recognise it follows dynamic variable declaration As you noticed, when I recognize it follows dynamic variable declaration pattern then I try to evaluate JavaScript with using this static method: VariableUtil.evaluateJSExpressionOnFrameworkVariable(v)

In this utility we start the evaluation JavaScript expression:

public static String evaluateJSExpressionOnFrameworkVariable(String str) {
        str = replaceAllFrameworkVariableValues(str).replace("\n", "");
        String subMatch = null;
        String expression = null;
        String subValue = null;

        if(str.startsWith("$js{function ")) {
            expression = PluginUtils.getInstance().getJSExpression(str);
            return jsUtils.getInstance().execute(expression).toString();
        }
        
        Matcher matcher = JS_PATTERN.matcher(str);
        while(matcher.find()) {
            subMatch = matcher.group();
            subMatch = matchCurlyBrackets(str, subMatch);
            LOGGER.debug("Evaluating JS expression : " + subMatch);
            expression = PluginUtils.getInstance().getJSExpression(subMatch);
            subValue = jsUtils.getInstance().execute(expression).toString();
            str = str.replace(subMatch, subValue);
            matcher = JS_PATTERN.matcher(str);
        }
        return str;
    }

To recognize a pattern of JavaScript expression this quick utility can be used:

public boolean isJsExpression(String jsExpression) {
        return jsExpression.matches("[$][j][s][{](.*?)[}]$");
    }

    public String getJSExpression(String str) {
        Pattern pattern = Pattern.compile("[$][j][s][{](.*?)[}]$");
        Matcher matcher = pattern.matcher(str);
        if(matcher.find()) {
            return matcher.group(1);
        } else {
            LOGGER.info("Not a JS Expression");
            return "";
        }
    }

To evaluate JavaScript expression you have plenty of options, like using native JAVA inbuilt ScriptEngine or an external library. A good example of evaluation would be:

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("js");
Object result = engine.eval("4*5");

On top of all that if you want to push it even further it is possible to use ENVIRONMENT_VARIABLES as a javascript expression which enables so much more dynamic functions (like CI integration and all that good stuff):

private static String environmentVarPattern = "\\$Env\\{(.*?)\\}";    
private static boolean isEnvVariable(String var){
        return var.matches("(.*?)" + environmentVarPattern + "(.*?)");
    }

private static String getEnvVariableName(String varString){
     Pattern pattern = Pattern.compile("(.*?)" + environmentVarPattern + "(.*?)");
        Matcher matcher = pattern.matcher(varString);
        if (matcher.find()) {
            String envVarName = matcher.group(2);
            String envVarValue = ConfigUtil.getString(envVarName);
            return varString.replace(String.format("$Env{%s}",envVarName),envVarValue);
        }else{
            LOGGER.error("Error in evaluating the Env Variable");
            throw new RuntimeException("Error in evaluating the Env Variable: " + varString);
        }
    }

All in all, by utilizing a cucumber data table it is possible to create very dynamic test cases that support javascript expressions or can read environment variables.

This can be used in many ways if you want a unique ID, if you want to generate today’s date, if you want to grab the environment from your environment variables, if you want to grab a variable from Jenkins (env variable). All this can be easily achieved with this clever solution.

Using Centralised Web Editor for Test Automation

Having a local copy of the code and using one of the major desktop IDEs is bread and butter for developers and test engineers. However, I do believe that having a centralized web editor in the organization for test automation is very beneficial and can significantly improve and streamline testing. Benefits:

  • Accessible to anyone from anywhere (bookmark the link and that’s it)
  • Very high visibility of tests (anyone can check and run)
  • Easy access (no need to checkout, fight with maven, no setting.xml)
  • Easy to run test cases (just select the feature file and run)
  • Reports are generated after every run as a popup. (no need to run maven commands to generate reports)
  • Every report has a direct URL since it’s stored on a VM and saved for as long as you want. It can easily be shared with anyone as a link.
  • VCS integration (if you need to do some quick changes just use built-in VSC tool)
  • It doesn’t take too many resources on VM.

I personally like Eclipse’s solution which has quite a few plugins and is very lightweight. There are some other alternatives but I will provide more info and screenshots of Eclipse Orion web editor. You can find it here:

The whole web editor is quite straightforward to use (open the image in the new tab to see it in more detail):

Cucumber Plugin allows to run .feature files straight from the webeditor (open the image in the new tab to see it in more detail):

As soon as you run the tests, logs appear allowing to debug/track of what’s going on very conveniently. (open the image in the new tab to see it in more detail)

As soon as the run is over Cucumber generates and launches a standard report with all of the information like expected / actual values.

I believe it’s a great addition to test automation stack, which provides more exposure, accessibility and visibility. Separate your code from test cases and you have a perfect way to share test cases / implementation/ reports with everyone across the organisation.

Creating Dynamic .CSV Files in Cucumber Using .VM templates

Have you ever got into a situation while testing where you had to automate creation of a .CSV file and upload it on to the system? One way of doing this is to create a separate Cucumber step that does that for you, however, it’s a very static and clunky solution that required for you to create a new step every-time you want to change or edit the structure of the .CSV file. Also, you don’t have the freedom of parameterizing the document, values have to be preset or in a best-case very few fields can be parameterized.

Luckily there is a solution to this problem, using .VM templates you can create a dynamic .csv files on the go in the .feature file. This gives you all the control in the world over .csv file creation and population inside of the feature file. These are the simple steps that allow you to achieve this solution:

  • Create a .VM template file with expected structure of the .csv
  • Create a step in cucumber which takes in your desired data set and the template
  • Using the template and data set, create a .csv file.

Bellow are some snippest that can be utilized to achieve this solution:

.VM template example:

Example_1,Example_2,Example_3
#foreach($c in $o)
#set ($example1 = $c.getOrDefault('example1','1'))
#set ($example2 = $c.getOrDefault('example2','1'))
#set ($example3 = $c.getOrDefault('example3','1'))
$example1,$example2,$example3
#end

This is how this whole solution looks like in the feature file, just pass a template file and create a cucumber table which will be used to generate a .csv file with whatever values you want. One method will cater for all .csv related functions.

And I define the following user defined variables:
    | ranNum1   | $js{ranNum = Math.floor((Math.random() * Math.pow(10,4)) + 1).toString();} |

And I update the below list of values in the template file "data/data_loaders/idr/idr_template.vm" and save it in file "/target/IDR/<dataFile>":
    | example1                  | example2 		   | example3       |
    | VALUE_1_${ranNum1}  	| VALUE_2                  | VALUE3  		| 

This is my example implementation of Cucumber step, which takes in a template, data table and generates a .csv file in a target location. This method can be used in any .csv file creation.

    @And("^I update the below list of values in the template file \"([^\"]*)\" and save it in file \"([^\"]*)\":$")
    public void iUpdateTheBelowListOfValuesInTheTemplateFileAndSaveItInFile(StepParam templateFile, StepParam targetFile, List<Map<StepParam, StepParam>> dataTable ) throws Throwable {

        List<Map<String,String> > values = new ArrayList<>();
        dataTable.forEach(stepParamStepParamMap -> {
            Map<String,String> map = new LinkedHashMap<>();
            stepParamStepParamMap.forEach((k,v)-> map.put(k.val(),v.val()));
            values.add(map);
        });
        String value = TemplaterUtil.getValueAfterReplacing(templateFile.val(),values );
        FileUtils.write(new File(userDir + targetFile.val()),value,false);
        PluginUtils.moveFilesToCrucialFilesFolder( userDir + targetFile.val());
    }
	

A further implementation of how to use .VM template.

public static String getValueAfterReplacing(String filePath, Object object) throws Exception {
        VelocityEngine engine = new VelocityEngine();
        engine.init();
        Template template = engine.getTemplate(filePath);
        VelocityContext vc = new VelocityContext();
        vc.put("o", object);
        StringWriter writer = new StringWriter();
        template.merge(vc, writer);
        return writer.toString();
    }

That’s a quick solution to all of your .csv testing problems. No need to build an excel file using Apache POI, and having so many different methods to cover all scenarios. Just create a quick template file and define the data directly in the feature file.

Also, this can be utilized when creating .XML files as well!

Testing Solace Queue

Testing a system which utilizes solace queue is a little bit challenging due to the nature of solace queue being a bit of a black box. Most of the time you are not going to have access to Solace code or your company is going to use a solace wrapper which complicates testing even more. Due to these factors there is a very limited scope of what you can test. When testing a system which uses Solace queue I like to treat is as part of the system rather than component therefore I include Solace queue in any end to end system integration test to make sure it is covered. In this post, I will try to go through what are critical areas to test in solace queue and then I’ll expand on how to send a test message to Solace queue using Java.

If the system that you are testing uses Solace queue, I would mostly concentrate on two things when testing it:

Connectivity – Any system that relies on Solace Queue needs to be tested for connectivity. Is it resilient? Can it recover after losing connection? What happens to messages that accumulate in the queue while connection is down. Answering these questions is very important and your tests should cover that.

Throughput – Another very important point is to test it for throughput. It’s a simple performance test to see how many messages can Solace Queue handle. Check what happens when the maximum throughput is reached, does Solace queue goes down? Do you lose messages?

Bellow you can find an example for a simple fire and forget solace queue publisher which will help test it for the scenarios listed above.

Solace Queue Maven dependency:

com.solacesystems
sol-jcsmp
10.6.3

Simple publisher:

public void fireAndForget() throws JCSMPException {

        JCSMPSession session = establishSolaceSession();
        session.connect();

        final Topic topic = JCSMPFactory.onlyInstance().createTopic("example/topic");

        XMLMessageProducer prod = session.getMessageProducer(new JCSMPStreamingPublishEventHandler() {
            @Override
            public void responseReceived(String messageID) {
                System.out.println("Producer received response for msg: " + messageID);
            }
            @Override
            public void handleError(String messageID, JCSMPException e, long timestamp) {
                System.out.printf("Producer received error for msg: %s@%s - %s%n",
                        messageID,timestamp,e);
            }
        });

        TextMessage msg = JCSMPFactory.onlyInstance().createMessage(TextMessage.class);
        final String text = "Hello world!";
        msg.setText(text);
        System.out.printf("Connected. About to send message '%s' to topic '%s'...%n",text,topic.getName());
        prod.send(msg,topic);
        System.out.println("Message sent. Exiting.");
        session.closeSession();
    }

    public JCSMPSession establishSolaceSession() throws InvalidPropertiesException {
        JCSMPProperties properties = new JCSMPProperties();
        properties.setProperty(JCSMPProperties.HOST, "host");
        properties.setProperty(JCSMPProperties.USERNAME, "username");
        properties.setProperty(JCSMPProperties.PASSWORD, "password");
        properties.setProperty(JCSMPProperties.VPN_NAME,  "vpnname");

        return JCSMPFactory.onlyInstance().createSession(properties);
    }

You can also use queue to connect instead of a topic:

Queue queue = JCSMPFactory.onlyInstance().createQueue("TEST_QUEUE");

Solace can accept different types of messages, in this example we are sending a text message which is very simple structure, however it does have quite a few different types like:

        MapMessage mapMessage = JCSMPFactory.onlyInstance().createMessage(MapMessage.class);
        SDTMap solaceMsg = JCSMPFactory.onlyInstance().createMap();
        solaceMsg.putString("KEY","VALUE");
        mapMessage.setMap(solaceMsg);
        prod.send(mapMessage,topic);

There are some other additional types of messages like XMLMessage and lesser used BytesMessage, BytesXMLMessage or RAWSMFMessage and others.