Wednesday 11 December 2013

Re-run failed tests Automatically | TestNG | Log4J

Here, I've used Maven for Test Execution; Make sure Log4J dependency added in your POM.xml file. You must run your tests on TestNG framework to achieve this.

Please find the optimized test in my GIT with the use case mentioned below| https://github.com/prashanth-sams/re-runFailedTests

1| Assume you have a class file with 10 Methods [@Test]
2| Lets say one got failed due to Network issue
3| The class file, RetryAnalyzer will make the failed test-case to re-run with desired count; But, it display errors for all the test failures during re-run. [Explained in this post]
4| You are ought to add a Listener class, RetryTestListener; it will print the failure only once.
5| If the test got passed in the final count, failure exception won't be thrown in the console.

Note| 
Steps 4 and 5 are missing in this post; Find it in GIT.


Test Class

@BeforeTest
public void setUp()
{
driver = new FirefoxDriver();
baseUrl = "https://www.google.co.in";
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}

@Test(retryAnalyzer=RetryAnalyzer.class)
public void test01() throws Exception
{
driver.get(baseUrl + "/");
String save = driver.findElement(By.id("als")).getText();
System.out.println(save);
        // Assert.fail();
Assert.assertEquals(save, "qwerty");
}


RetryAnalyzer.java

Create a Java class, 'RetryAnalyzer' inside the same package and paste the following snippet with minor edits.

import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryAnalyzer implements IRetryAnalyzer  { 
private int count = 0; 
private int maxCount = 4; // set your count to re-run test
protected Logger log;
private static Logger testbaseLog;

static {
    PropertyConfigurator.configure("test-config/log4j.properties");
    testbaseLog = Logger.getLogger("TestclassName");
}

public RetryAnalyzer()
{
    testbaseLog.trace( " ModeledRetryAnalyzer constructor " + this.getClass().getName() );
    log = Logger.getLogger("TestclassName");
}

@Override 
public boolean retry(ITestResult result) { 
    testbaseLog.trace("running retry logic for  '" 
            + result.getName() 
            + "' on class " + this.getClass().getName() );
        if(count < maxCount) {                     
                count++;                                    
                return true; 
        } 
        return false; 
}
}


POM.xml

<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.37.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>            
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/src/test/java/abilash/yourxmlfile.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>  
</plugins>
</build>


P.S| TestNG will create a xml file, 'testng-failed.xml' after your test failure; It contains all the failed test Methods. It can be used to re-run your Failed tests manually.

23 comments:

  1. Great going Prashanth. Your blogs are really informative for automation testers.
    It will be really helpful if you can provide some insight on parallel execution Selenium with testng framework.

    ReplyDelete
    Replies
    1. Thank you, I have already did a post on parallel execution using selenium. Please check my previous posts.

      _
      Sams

      Delete
  2. Hi Prashanth,

    very helpful blog for me and automation testers.

    Thanks,
    Tarique Khan

    ReplyDelete
  3. Hi Prashanth,

    Very good blog, I just want to integrate this with jenkins, could you also please let how to run failed test cases with Jenkins.

    ReplyDelete
  4. Hi prashanth, nice blog. I have question here. If i have a 20 tests to be run and if any of these test fail I should perform a log collection function, how can I do that.If i have a aftersuite method, how can i get the status of the tests. What should be my approach. THanks.

    ReplyDelete
  5. 've found various methods to Rerun your failed TESTNG tests like by using IRetryAnalyzer,ITestResult , but if my configuration methods fail due to any reason like my @BEFORECLASS or @BEFORETEST or @BEFOREMETHOD how to rerun your test in such cases.
    -Aditya

    ReplyDelete
  6. Hi Prashanth,
    Nice info, But have some doubts...
    * In case of failure I am expecting total no. of Failure and Pass results.
    But Instead M getting Skip results as well. Somehow i am not very sure how this wrks.
    PFA my code:

    private int count = 0;
    private int maxCount = 2; // set your count to re-run test

    public boolean retry(ITestResult result) {
    if(count<maxCount){
    count++;
    return true;
    }

    return false;
    }


    With only 1 test , which failed ,
    I got result as :::
    ===============================================
    suite 1
    Total tests run: 3, Failures: 1, Skips: 2
    ===============================================

    Whereas I am expecting :
    ===============================================
    suite 1
    Total tests run: 3, Failures: 3, Skips: 0
    ===============================================

    Please let me know , if my understanding is correct

    ReplyDelete
  7. Is it possible to run the testng-failed.XML with automation?.actually the problem i am getting is if I run with suite.XML file,the testing.XML doesn't contains updated failure methods

    ReplyDelete
  8. It was very nice blog to learn about SAP BASIS. Thanks for sharing.SAP basis

    ReplyDelete


  9. IRetryAnalyzer does not seem to be working if i try to run the same set of tests multiple times in different threads/same threads.

















    Where i have 3 failed tests in ParallelTest class. It is observed that only are run the second time. failed tests are not run.

    ReplyDelete
  10. Hi,
    Can you share details on running the test class again instead of a test method. I have a lot of pre-requisites done in the before class method and hence would like to start the whole test again.

    Regards

    ReplyDelete
  11. After reading this web site I am very satisfied simply because this site is providing comprehensive knowledge for you to audience.
    Thank you to the perform as well as discuss anything incredibly important in my opinion. We loose time waiting for your next article writing in addition to I beg one to get back to pay a visit to our website in


    Selenium training in Chennai
    Selenium training in Bangalore
    Selenium training in Pune
    Selenium Online training

    ReplyDelete
  12. آء هن مضمون کي ڏاڍي خوش نصيب آهيان. خواهش جو توهان هميشه خوش ۽ خوش ٿيندو

    giảo cổ lam giảm cân

    giảo cổ lam giảm béo

    giảo cổ lam giá bao nhiêu

    giảo cổ lam ở đâu tốt nhất

    ReplyDelete
  13. Awesome,Thank you so much for sharing such an awesome blog.INFORMATICA training in bangalore


    ReplyDelete
  14. Learned a lot of new things from your post! Good creation and HATS OFF to the creativity of your mind.HADOOP BIGDATA training in bangalore

    ReplyDelete
  15. Thanks for posting such kind of information in this blog. More useful to refer and appreciating one.

    Java Training in Chennai

    Java Course in Chennai

    ReplyDelete