Wednesday, 11 December 2013

Re-run failed tests Automatically | TestNG | Log4J

Here, I've used Maven for Test Execution; Make sure Log4J dependency added in your POM.xml file. You must run your tests on TestNG framework to achieve this.

Please find the optimized test in my GIT with the use case mentioned below| https://github.com/prashanth-sams/re-runFailedTests

1| Assume you have a class file with 10 Methods [@Test]
2| Lets say one got failed due to Network issue
3| The class file, RetryAnalyzer will make the failed test-case to re-run with desired count; But, it display errors for all the test failures during re-run. [Explained in this post]
4| You are ought to add a Listener class, RetryTestListener; it will print the failure only once.
5| If the test got passed in the final count, failure exception won't be thrown in the console.

Note| 
Steps 4 and 5 are missing in this post; Find it in GIT.


Test Class

@BeforeTest
public void setUp()
{
driver = new FirefoxDriver();
baseUrl = "https://www.google.co.in";
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}

@Test(retryAnalyzer=RetryAnalyzer.class)
public void test01() throws Exception
{
driver.get(baseUrl + "/");
String save = driver.findElement(By.id("als")).getText();
System.out.println(save);
        // Assert.fail();
Assert.assertEquals(save, "qwerty");
}


RetryAnalyzer.java

Create a Java class, 'RetryAnalyzer' inside the same package and paste the following snippet with minor edits.

import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryAnalyzer implements IRetryAnalyzer  { 
private int count = 0; 
private int maxCount = 4; // set your count to re-run test
protected Logger log;
private static Logger testbaseLog;

static {
    PropertyConfigurator.configure("test-config/log4j.properties");
    testbaseLog = Logger.getLogger("TestclassName");
}

public RetryAnalyzer()
{
    testbaseLog.trace( " ModeledRetryAnalyzer constructor " + this.getClass().getName() );
    log = Logger.getLogger("TestclassName");
}

@Override 
public boolean retry(ITestResult result) { 
    testbaseLog.trace("running retry logic for  '" 
            + result.getName() 
            + "' on class " + this.getClass().getName() );
        if(count < maxCount) {                     
                count++;                                    
                return true; 
        } 
        return false; 
}
}


POM.xml

<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.37.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>            
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/src/test/java/abilash/yourxmlfile.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>  
</plugins>
</build>


P.S| TestNG will create a xml file, 'testng-failed.xml' after your test failure; It contains all the failed test Methods. It can be used to re-run your Failed tests manually.

14 comments:

  1. Great going Prashanth. Your blogs are really informative for automation testers.
    It will be really helpful if you can provide some insight on parallel execution Selenium with testng framework.

    ReplyDelete
    Replies
    1. Thank you, I have already did a post on parallel execution using selenium. Please check my previous posts.

      _
      Sams

      Delete
  2. Hi Prashanth,

    very helpful blog for me and automation testers.

    Thanks,
    Tarique Khan

    ReplyDelete
  3. Hi Prashanth,

    Very good blog, I just want to integrate this with jenkins, could you also please let how to run failed test cases with Jenkins.

    ReplyDelete
  4. Hi prashanth, nice blog. I have question here. If i have a 20 tests to be run and if any of these test fail I should perform a log collection function, how can I do that.If i have a aftersuite method, how can i get the status of the tests. What should be my approach. THanks.

    ReplyDelete
  5. 've found various methods to Rerun your failed TESTNG tests like by using IRetryAnalyzer,ITestResult , but if my configuration methods fail due to any reason like my @BEFORECLASS or @BEFORETEST or @BEFOREMETHOD how to rerun your test in such cases.
    -Aditya

    ReplyDelete
  6. Hi Prashanth,
    Nice info, But have some doubts...
    * In case of failure I am expecting total no. of Failure and Pass results.
    But Instead M getting Skip results as well. Somehow i am not very sure how this wrks.
    PFA my code:

    private int count = 0;
    private int maxCount = 2; // set your count to re-run test

    public boolean retry(ITestResult result) {
    if(count<maxCount){
    count++;
    return true;
    }

    return false;
    }


    With only 1 test , which failed ,
    I got result as :::
    ===============================================
    suite 1
    Total tests run: 3, Failures: 1, Skips: 2
    ===============================================

    Whereas I am expecting :
    ===============================================
    suite 1
    Total tests run: 3, Failures: 3, Skips: 0
    ===============================================

    Please let me know , if my understanding is correct

    ReplyDelete
  7. Thanks for the great information in your blog Selenium Training in Chennai

    ReplyDelete
  8. Is it possible to run the testng-failed.XML with automation?.actually the problem i am getting is if I run with suite.XML file,the testing.XML doesn't contains updated failure methods

    ReplyDelete
  9. It was very nice blog to learn about SAP BASIS. Thanks for sharing.SAP basis

    ReplyDelete


  10. IRetryAnalyzer does not seem to be working if i try to run the same set of tests multiple times in different threads/same threads.

















    Where i have 3 failed tests in ParallelTest class. It is observed that only are run the second time. failed tests are not run.

    ReplyDelete