why maven-failsafe-plugin doesn't show serenity tests executed?
why maven-failsafe-plugin doesn't show serenity tests executed?
I setup a basic project using Serenity and I am having issues displaying results when I run mvn clean verify
mvn clean verify
[INFO] --- maven-failsafe-plugin:2.20:integration-test (default) @ functional-tests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running starter.SmokeTestSuite
Feature: Booking
@Smoke
Scenario Outline: Book a reservation # src/test/resources/features/book/booking_creation.feature:4
Given an agent with role "ADMIN" has logged in
Examples:
@Smoke
Scenario Outline: Book a reservation # src/test/resources/features/book/booking_creation.feature:19
Started InternetExplorerDriver server (32-bit)
3.13.0.0
Listening on port 11642
Only local connections are allowed
Sep 10, 2018 7:50:29 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Given an agent with role "ADMIN" has logged in # LoginSteps.agent_has_logged_in(String)
1 Scenarios (1 passed)
1 Steps (1 passed)
0m18.568s
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.681 s - in starter.SmokeTestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
These are some facts I have from this issue:
This is my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.test</groupId>
<artifactId>functional-tests</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<name>functional tests</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<serenity.version>1.9.26</serenity.version>
<serenity.maven.version>1.9.26</serenity.maven.version>
<serenity.cucumber.version>1.9.8</serenity.cucumber.version>
<encoding>UTF-8</encoding>
<tags></tags>
<parallel.tests>4</parallel.tests>
<webdriver.base.url></webdriver.base.url>
</properties>
<dependencies>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-core</artifactId>
<version>$serenity.version</version>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-junit</artifactId>
<version>$serenity.version</version>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-screenplay</artifactId>
<version>$serenity.version</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-screenplay-webdriver</artifactId>
<version>$serenity.version</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.serenity-bdd</groupId>
<artifactId>serenity-cucumber</artifactId>
<version>$serenity.cucumber.version</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.6.2</version>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.20</version>
<configuration>
<includes>
<include>**/*TestSuite.java</include>
</includes>
<!--
<systemPropertyVariables>
<webdriver.base.url>$webdriver.base.url</webdriver.base.url>
</systemPropertyVariables>
-->
<parallel>classes</parallel>
<threadCount>$parallel.tests</threadCount>
<forkCount>$parallel.tests</forkCount>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>$serenity.maven.version</version>
<configuration>
<tags>$tags</tags>
</configuration>
<executions>
<execution>
<id>serenity-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I appreciate any help, thanks!
1 Answer
1
This doesn't answer you question as to why maven-failsafe-plugin doesn't show tests executed, but I have the same problem and needed to fail the build if tests failed.
The tests run but executions are not reported:
Reports view generated with 288 stories (of which 1 pending) containing 2463 scenarios (of which 1 pending)
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23,875.336 sec
I upgraded the Serenity, JBehave and Selenium dependencies with no luck.
I have resorted to using the serenity-maven-plugin:check goal to fail the build (instead of the maven-failsafe-plugin) during the verify phase:
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
...
<execution>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
Which results in a build failure as required:
[INFO] --- serenity-maven-plugin:1.9.45:check (default) @ project-testing ---
[INFO] Checking Serenity test results
[INFO] ----------------------
[INFO] SERENITY TEST OUTCOMES
[INFO] ----------------------
[INFO] - Tests executed: 2628
[INFO] - Tests passed: 1823
[INFO] - Tests failed: 48
[INFO] - Tests with errors: 757
[INFO] - Tests pending: 0
[INFO] - Tests compromised: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.serenity-bdd.maven.plugins:serenity-maven-plugin:1.9.45:check (default) on project project-testing: An error occurred in the Serenity tests -> [Help 1]
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
