Tuesday 29 October 2013

Running selenium tests with TestNg (Unit testing tool)

While exploring different selenium forums , I have observed that ,Many test engineers asks the question “Is it mandatory to use any unit testing tool (Java : testNg,Junit or dotNet: Nunit Or python :PyUnit) with selenium for running the tests
If someone will ask this to me I will answer it as a ‘NO’ ,we can run our selenium tests in a suite without using TestNg/Junit/Nunit or any other unit testing tool.

How can we run selenium tests without using TestNg,Junit or any Unit testing tool.
Simply write a client class with main method as follow and give the call to methods that you want to execute in a suite.
public class SeleniumClient {
      public static void main(String[] args) {
            SeleniumTest testClient = new SeleniumTest();
            testClient.LoginToApplication("loginName", "loginPassword");
            boolean isSuccessfullyLoggedIn = testClient.VerifySuccessfulLogin();
      }
}

What is a drawback of doing this ?
Testwriter has to explicitly call his methods to get it executed in a suite which is a big overhead for him.

What is a best way to achieve this i.e executing selenium tests in a suite using any unit testing tool ?
First You can configure testNg plugin to eclipse IDE as per the documentation given on testNg SIte
For the Eclipse plug-in, we suggest using the update site:

  1. Select Help / Software updates / Find and Install.
  2. Search for new features to install.
  3. New remote site.
  4. For Eclipse 3.4 and above, enter http://beust.com/eclipse.
  5. Make sure the check box next to URL is checked and click Next.
  6. Eclipse will then guide you through the process.
Once you have configured testNg  with your eclipse there are two ways to run the tests in a suite :
You can either write testNg class as per the documentation given here 
Or You can write a testNg.xml as per your need to run the tests in a suite, mentioned here
How to use TestNg features while integrating it with selenium.

I observed many automation teams are integrating selenium with testNG.While doing this exercise of integrating Selenium with lot of questions comes in mind of automation engineer,here I am trying to answer few of the use cases that I come across while using selenium with testNg.

How to classify selenium tests in different groups using testng ?
Requirement 1 : If I have a requirement to categorize all my automated tests in 3 different groups
                “Smoke Test” ,”Critical Test” and “Regression Test” how can I Differentiate this tests in different group.

@Test(groups = { "Smoke Test" })
      void testMethod1() {
            >>>>>Method definition>>>>>>>>
      }

      @Test(groups = { "Critical Test" })
      void testMethod2(String... b) {
            >>>>>Method definition>>>>>>>>
      }

      @Test(groups = { "Regression Test" })
      void testMethod3(int a, int b) {
            >>>>>Method definition>>>>>>>>
      }
Now I have to execute only Smoke test, how to handle it with testNg.xml
                                <groups>
                  <run>
                        <include name="Smoke Test"/>
                  </run>
            </groups>
              
Requirement 2 :  I have 100 tests automated out of that I have to exclude the tests which is having a product bug.
@Test(groups = { "Product Test", "Smoke Test" })
      void testMethod1() {
            >>>>>Method definition>>>>>>>>
      }

      @Test(groups = { "Product Test", "Critical Test" })
      void testMethod2() {
            >>>>>Method definition>>>>>>>>
      }

      @Test(groups = { "Product Test", "Regression Test", "Bug" })
      void testMethod3() {
            >>>>>Method definition>>>>>>>>

      }
I don’t want to execute the tests which are already having a product bug, remaining all tests needs to be executed.
                                <groups>
                  <run>
                        <include name="Product Test"/>
                        <exclude name="Bug"/>
                  </run>
            </groups>
     

What is a SoftDependancy and HardDependancy in TestNg ?
                Most of the time we need to deal with 2 type of dependencies ,
1.       Hard Dependency:  If there are two methods and method2 needs to be executed only if method1 is passing then this is a Hard Dependency.

@Test(groups = { "Product Test" })
      void startUp() {
            System.out
                        .println("this method will be responsible for all the prerequisites ");
      }

      @Test(groups = { "Product Test" }, dependsOnMethods = { "startUp" })
      void hardDependantTest() {
            System.out.println("Perform actual test execution code.");

      }
                                Here hardDependantTest() will get executed only if startup() gets passed.

2.       Soft Dependency:  If we bother only with the sequence of the test, means method2 needs to be executed only after method1, irrespective of its execution status pass or fail ( mehtod2 needs to be executed even after method1 fails)
@Test(groups = { "Product Test" })
      void testMethod1() {
            System.out.println("Perform actual test execution code.");
      }

      @Test(groups = { "Product Test" }, dependsOnMethods = { "testMethod1" }, alwaysRun = true)
      void testMethod2() {
            System.out.println("Perform actual test execution code.");

      }
More complex part related to selenium and testNg will get covered in next blog !!

Thursday 8 August 2013

Simple solution to automate Selenium tests of Uploading/Downloading file from/To local Disk for Firefox.

I have observed that many automation engineer gets stuck while automating Upload/download File  from/to local disk ,related tests with selenium, I would like to take this opportunity to throw some light on it

Autoit is one of the option to download or upload the file but many automation framework developer avoids the use of third party tools with their framework, Even though it’s an open source, it could get licensed in future.
Apart from AutoIt, there is couple of other solutions to achieve it.here I tried to list it down along with the open issues for the provided solution.
Here is what we already tried and the problems we faced along with the actual solution :)

Solution 1 : Uploading a file using Autoit & WebDriver
Refer a blog : blog1 and blog2

Solution 2: Download programmatically using href attribute of button/link/image
Problem: Works only for button/link/image having href pointing to actual file location. IPP uses javascript as value for href. So cant be used.

 Solution 3: Use user32 (JNA)
            Fine for tasks like checking window existence, setting focus on specific window etc.
Problem: But can’t identify objects on window (buttons, checkboxes etc. on window).

 Solution 4: Use SendKeys on window under focus
            Problem:  Firefox popups doesn’t respond to keys send programmatically.

 Solution 5: Use Send keys using selenium
             Problem:  Selenium sends keys only on objects which Selenium identifies. Firefox’s ‘File Download’ popup is not recognised by Selenium.

 Solution 6: Use AutoHotKey tool
             Problem:  Firefox’s ‘File Download’ popup doesn’t respond to keys send by AutoHotKey.

Solution with which I moved ahead and found very simple.
Simple Solution: Use firefox profile to handle file download
Firefox settings to manage File Download dialog in Selenium:

1.       Disable “Show the Downloads window when downloading a file”:
2.       Add default download path to profile being used for Selenium run.

3.       Download each file type (which you will be downloading during test runs e.g. .xls, csv, txt etc.) once and check the “Do this automatically for files like this from now on” option.
Note:
1.       Above options (1,2 and 3) once set, will automatically download selected file types to folder specified in step 2.
2.       Above options are stored in the current firefox profile.
3.       All these options can be changed using Applications tab in Firefox’s options.

Even internet explorer and chrome are having those options so that we can download the file at one fix location so that  you will not have to work on download window.
Feel free to revert back.


Friday 2 August 2013

How to deal with multiple frames with dynamic Ids using webdriver.

Here I would like to address an issues that I faced while upgrading the existing automation framework of selenium webdriver  to support HTML5 UI.
Earlier Application Under Test (AUT) was developed in Icefaces means all tables, popups,controltypes were developed in icefaces. later on management decided to revamped the IceFaces UI with HTML 5.
To render the old UI of Icefaces on HTML5 , developers has deployed an additional Iframe for each view. And it has created a big challenge for me to keep existing tests running on the new UI without modifying the tests. I followed following approach in updating the framework/keywords to make sure test won’t need an update.

Main challenge :
In this assignment the main challenge was to attach an iframes with dynamic IDs .Switching between different iframes, once user has changed the Views.
In this I was not aware of the frame ID to whom I have to attaché webdirver , because each frame was having a dynamic ID starting with String ‘Frame’ for e.g Frame1256893
I could use this as an xpath :
String frameXpath = "//iframe[starts-with(@id,'Frame')]";


But it will not solve all my problems because there could be multiple views opened at a time with only one view in a focus.if I would have been used only above xpath then it would have always attach the webdriver control to first frame

How can we achiev it ?

protected void attachToVisibleViewFrame() {
  String frameXpath = "//iframe[starts-with(@id,'Frame')]";
  driver.switchTo().defaultContent();
  WebElement[] framesElement = wedriverInstance.findElements(
    By.xpath(frameXpath), 3);
  if (framesElement != null) {
   for (WebElement frameElement : framesElement) {
    if (frameElement.isDisplayed() && frameElement.isEnabled()) {
     System.out
       .println("PortalView : WebDriver control switched to a frame with label : "
         + frameElement.getAttribute("view-label"));
     driver.switchTo().frame(frameElement);

     break;
    }
   }
  }
 }
Above code additionally checks whether iframe for which webdriver is trying to get attach isDisplayed and isEnabled.so it will always attach to a frame which is in focus and not to a frame which is hidden.
Hope this will help you to avoid an overhead of multiple frames within your application.

Thursday 13 June 2013

Easy Integration of Sikuli with existing framework of selenium

Nowadays Selenium is widely used tool for automating web application. In real life, products are enterprise application, so application would be an integration of  Silverlight ,Java applets,Raphael java script, canvas objects,Javascripts,Jquery. We could not expect everything to get identified with selenium and can face challenges in automating the few of the above mentioned UI with selenium.

Sikuli is one of the easiest tool to learn and can be used to automate anything using image recognition technique.
Here I am trying to share my experience about automating an enterprise application with integration of selenium and SIkuli.

I  was involved in automating the BPM product (business process modeling ),I have developed a framework in selenium to test this product .our functional team has developed around 1000+ tests using this framework.
Few months back our product development team has introduced new functionality of browser based modeling in which model can be developed in browser itself. This functionality was developed using canvas objects and Raphael javascripts.Automating this functionality was very important for us. Unfortunately selenium was unable to identify it. I tried all the options from selenium webdriver but all the efforts were in vain. Then We tried with different tools like Watir, Watij , AUTOIT and white to identify (business process modeling )BPM canvas objects but unfortunately these tools are also failed to get it automated.
                                                                                                                                                                                                                                            
While evaluating few other tools for BPM automation, I come across Sikuli (http://www.sikuli.org/), it’s an intelligent tool for recognizing the Canvas UI. It’s an open source tool.
It automates the UI using image recognition, and this image recognition technique was finest fit for us to identify Canvas objects and other modeler components from browser based modeler written in Raphael Java scripts.

Sikuli permits us to write the modularize tests in Java. Using this feature of sikuli, I  have integrated it in our selenium framework written in JAva
Sikulli team has provided neat and precise documentation about tool from basic to advance level, which guided us wherever we stuck up at any point while scripting with this tool.

As I mentioned earlier our functional team was expert  in writing the test using my existing framework written for selenium java. And I don’t want to expose them to new automation tool for automating one functionality from our huge product.so the main challenge for me was
ü  To integrate the sikuli with existing framework of selenium
ü  Developing the new keywords for sikuli , in a same way that I followed for selenium keywords,so that for our selenium test writer approach would remain same for writing the tests.
ü  With a detailed feasibility study of sikuli we observed that sikuli has its limitation and sikuli nowhere sits near selenium, so I decided to keep Sikuli’s use restricted to only for the UI which couldn’t get automate with selenium and remaining stuff with selenium.

How to integrate sikuli with existing framework of selenium ?
Sikuli provides sikuli-script.jar.you can download it from sikuli site  as recommended on the download page, Import this jar is your java application, once you have this jar in your java application.
You can invoke sikuli api’s in your selenium framework by creating Screen objects instance.

protected static Screen sikuliObject = new Screen();

Following is one of the simple way to call selenium and sikuli api alternatively as per the need of the test.

in following code ,Line 2 to 6 is of selenium code and line 11 to 13 is of sikuli code.

public void dragModelElementInCanvas(String modelElement, int x,int y) {
String elementToclick = "//a[contains(.,'"+ modelElement + "')]";
WebElement clickabelModelElement = seleniumDriver.findElement(By.xpath(elementToclick));
if (clickabelModelElement != null) {
 Actions action = new Actions(driver);
 action.clickAndHold(clickabelModelElement).perform();
 sleep();
//If you have been observed that selenium is not able to identify the component (e.g canvas)where you would like to above dragged element.
   System.out.println("moving mouse to the location " + x + " : " + y);
   sikuliObject.mouseMove(new Pattern(getAbsolutePath()).targetOffset(x, y));
   System.out.println("simulating mouse up");
 sikuliObject.mouseUp(Button.LEFT);
 action.release().perform();
  } else
   throw new KeywordException(
     "Timeout recieved while finding element with Xpath:"
       + clickabelModelElement);
 }
Important part here is for testwriter whomsever is using your framework will not have to bother which underline tool you are using to automate the stuff. For e.g in above code selenium testwriter will call the method dragModelElementInCanvas ,where he/she will not realize which tool has been used to achieve the task.

Above is the just the reference code that I have documented here for the one who will be getting started withselenium and sikuli’s integration. In real life we have developed many classes where we are using selenium and sikuli Hand to hand. Post your doubts if you are facing any specific issues while integrating selenium and sikuli.i would like to share my experience as we already overcome many challenges that we faced in integration of two automation tools in our existing framework of selenium.

Happy integration !!

Tuesday 14 May 2013

Working on Iframes with selenium

While working with different automation team members, I have observed that people are facing lot of challenges when they start working on UI with multiple iframes.here I tried to document my personal experience about handling iframes with selenium.

If you are beginner in automating UI having frames/ iframes, then please make a note that selenium doesn’t recognize any object from a frame, unless you attach the selenium control to respective frame.
I have listed few scenarios where automated test can stuck :


Scenario 1:
How to attach the control to iframe with selenium webdriver :
        driver.switchTo().frame(frameID or any other unique attribute of frame);
Above code will take care of attaching to frame after that you can identify any objects from the frame using the same way you are identifying the objects from webpage.
                  driver.switchTo().defaultContent();

Scenario 2:
If I have two independent Iframe (FrameID1 and frameID2) how to work on these two frames
driver.switchTo().frame(frameID1);
<perform the operation on frame1>
:
:
:
<Now if you have to work on controls from frame2 then >
driver.switchTo().defaultContent();
driver.switchTo().frame(frameID2);

Scenario 3:
If I have childFrame inside the parentFrame how to work on child frame . Refer below HTML for detail understanding

<frame name="Frame1" id="parentFrameID">
        -----------
            <frame name="Frame2" id="childFrameID1">
        -----------
   
            </frame>
<frame name="Frame2" id="childFrameID2">
        -----------
   
            </frame>

 </frame>

In this case, you cannot directly attach to childFrameID,first switchTo parentFrame inside which childframeis is placed then look for required child frame.
            driver.switchTo().defaultContent();
            driver.switchTo().frame("parentFrameID");
            driver.switchTo().frame("childFrameID1");


How to switch from parent frame from one childframe ?
you cannot toggle directly between child frame just by switching from one child frame to another child frame, refer below code

            driver.switchTo().defaultContent();
            driver.switchTo().frame("parentFrameID");
            driver.switchTo().frame("childFrameID1");
            //now as you have to switch for a childframe2 then
            driver.switchTo().defaultContent();
            driver.switchTo().frame("parentFrameID");
            driver.switchTo().frame("childFrameID2");

You can also switch to child frame using its index
 
driver.switchTo().frame(indexoftheframe);
                         
index starts from ‘0’
  
Believe me, there are many more scenarios related to frames that supposed to be handle in this post but for now I want to keep it simple.
Please feel free to discuss if you are facing any other challenge which I haven’t mentioned here. we can surely extend this post to resolve your creepy  issues related to iframe with selenium.


Sunday 21 April 2013

Video recording of failed test Scripts in selenium framework

Failure analysis is very crucial part for any automation team. We can automate the tests , we can automate the process of executing the tests once build is arrived for testing,but automation tester have to have spare some time in manually analyzing the report to make sure whether it’s a product issue or not. To make failure analysis job bit easy, here I tried to explain how to capture the screens on failures and how to do video recording of the test during execution.

What is main challenge in failure analysis ? 

If test is taking 3 minutes to get executed and failing somewhere at last set of lines, during failure analysis, complete test needs to be rerun to find where exactly it’s failing. Instead of that, this feature will help us, to quickly play the video. In fast track, we can reach to the step, where exactly it failed and can take corrective action before rerun.
Sometimes it also happens test gets failed due to some intermittent time out issues, and it became difficult to reproduce those issues in rerun. For such failures this video file will support us in analyzing the page or component taking more than expected time to get rendered.


How can I Capture the screen shots on test failures with selenuim ? 

Selenium is having a method which capture the screen shot, but this method need to be use smartly in framework to capture the screen shots only when the test gets failed.so by referring the screen shot test can be fixed
Attached code is an example how to capture the screen shots on failures. 
public class KeywordException extends RuntimeException {

 /**
  * Throw this exception if keyword is failing.
  * @param message
  */
 public static String callerClass;
 public static String callingMethodName;
 private static WebDriver driver = BasicTestContext.driv;
 public CaptureScreenShotOnException(String message) {
  super(message);

  File scrFile = ((TakesScreenshot) driver)
    .getScreenshotAs(OutputType.FILE);
  String screenShotPath = "C:\\screenshot\\" + callerClass + "_"
    + callingMethodName + "_"
    + getCurrentTimeStamp("yyyyMMMdd_hhmmss") + ".png";
  try {
   FileUtils.copyFile(scrFile, new File(screenShotPath));
   System.out.println("Saved screenshot: " + screenShotPath);
   System.out.println("Failed : " + callerClass + "_"
     + callingMethodName + " . REASON : "+message);
  } catch (IOException e1) {
   e1.printStackTrace();
  }
 }
}
 
How to use it in your test ?

public void setText(String value) {

            if (INF_Common_Utility.SyncForElement("xpath=" + getRootXpath()))
                  browser.type("xpath=" + getRootXpath(), value);
            else
                  throw new CaptureScreenShotOnException("Timeout recieved while finding element with Xpath: xpath=" + getRootXpath());
                 
      }
Capturing the Video on failures for the selenium tests:

You can evolve your automation framework by introducing a feature of recording the selenium test execution.Using this feature selenium test execution can be recorded as a video file.it will generate one video file for each failed tests.
 
 How can I implement it in my framework ?

To implement this feature, we used Monte Media Library .This library is having a class ScreenRecorder.this class provides the api for recording screencast of Selenium Tests in Java.
To play the recorded video: ScreenRecoder supports “AVI” format for recording the video. To play the video file generated in  “AVI” format, you need to install TSCC Codec (Techsmith Screen Capture Codec).
/**
       * This method will start the recording of the execution. recorded file will
       * get generated in the folder name as className_methodName. recording can
       * be started from any particular time.
       */

      public void startRecording() {

            RecordingClassName = Reflection.getCallerClass(2).getSimpleName();
            RecordingMethodName = new Throwable().fillInStackTrace()
                        .getStackTrace()[1].getMethodName();
            fileRecordingLocation = "c://screenshot//" + RecordingClassName + "_"
                        + RecordingMethodName;
            RecordedFile = new File(fileRecordingLocation);

            GraphicsConfiguration gc = GraphicsEnvironment//
                        .getLocalGraphicsEnvironment()//
                        .getDefaultScreenDevice()//
                        .getDefaultConfiguration();
            try {
                  ScreenRecorder = new ScreenRecorder(gc, null, new Format(
                              MediaTypeKey, MediaType.FILE, MimeTypeKey, MIME_AVI),
                              new Format(MediaTypeKey, MediaType.VIDEO, EncodingKey,
                                          ENCODING_AVI_TECHSMITH_SCREEN_CAPTURE,
                                          CompressorNameKey,
                                          ENCODING_AVI_TECHSMITH_SCREEN_CAPTURE, DepthKey,
                                          (int) 24, FrameRateKey, Rational.valueOf(15),
                                          QualityKey, 1.0f, KeyFrameIntervalKey,
                                          (int) (15 * 60)), new Format(MediaTypeKey,
                                          MediaType.VIDEO, EncodingKey, "black",
                                          FrameRateKey, Rational.valueOf(30)), null,
                              RecordedFile);
                  ScreenRecorder.start();
            } catch (IOException e) {
                  e.printStackTrace();
            } catch (AWTException e) {
                  e.printStackTrace();
            }
      }

      /**
       * This method will stop the recording.
       */
      public void stopRecording() {
            if (ScreenRecorder != null) {
                  try {
                        ScreenRecorder.stop();
                  } catch (IOException e) {
                        e.printStackTrace();
                  }
            }
      }
/**
* to delete the recorded file.if the test is passing then this method will delete the recorded file.
*/
      public void deleteRecordedFile() {
            if (RecordedFile != null) {
                  String[] children = RecordedFile.list();
                  for (int i = 0; i &lt; children.length; i++) {
                        new File(RecordedFile, children[i]).delete();
                        System.out.println("Deleted this " + RecordedFile + "//"
                                    + children[i]);
                  }

                  // The directory is now empty so delete it
                  RecordedFile.delete();
            }
      }

      /**
       * This method works similar to the org.testng.Assert.assertTrue and also
       * performs following actions: 
       * it takes the screenshot of the active screen if the assertion fails.
       * it also deletes the recorded vedio file if the test is passed. Call this
       * method only once, at the end of the test.
       *
       * @param condition
       * @param message
       */
      public void finalAssertTrue(boolean condition, String message) {
            stopRecording();
            if (condition)
                  deleteRecordedFile();
            _assertTrue(condition, message);
      }

How to use it in the test ?
public class MySeleniumTest extends BasicSeleniumTest {

 @TestDetails(author = "Bhushan Pawar", description = "Show the demo of video recording.")
 @Test
 public void testToGenerateVideoFile() {

  startRecording();

  /**
   * <> 

   * <> 

   * <>
   * 
   */
  finalAssertTrue(condition,
    "test is failed and I want to see the video file where exactly it failed !!");
 }

This way, Video recording feature will assist the one who is maintaining the tests  .
Happy Automation testing !