Configuration Suite Status

Problem

Configuration suites allow you to execute the same set of tests against multiple configurations, for example multiple browsers or operating systems. To be able to make reasonable statements related to quality and reliability of your application under test you will want to keep track of the results for each individual configuration.

Solution

Use the data mart view RV_ConfigurationSuiteStatus to create a report that returns the passed, failed, and not executed counts for each configuration per build.

SELECT BuildName, ConfigurationName, PassedCount, FailedCount, NotExecutedCount
FROM RV_ConfigurationSuiteStatus 
WHERE ConfigurationSuiteID = ${configSuiteID|97|Configuration Suite ID}
ORDER BY BuildOrderNumber, ConfigurationName

The query does the following:

Retrieves the status counts per build of test runs from the RV_ConfigurationSuiteStatus view.
Narrows the results down to the configuration suite (ConfigurationSuiteID).

The result of the SQL query shows the status of your test runs for each configuration.

BuildName	ConfigurationName	PassedCount	FailedCount
350	Chrome	0	1
350	Firefox	0	1
350	Internet Explorer	0	1
351	Chrome	1	0
351	Firefox	1	0
351	Internet Explorer	0	1
352	Chrome	1	0
352	Firefox	1	0
352	Internet Explorer	1	0

In this example, we use the ID of the configuration suite to get all configurations. It is also possible to restrict the result to specific builds, in which case you would have to include BuildID, BuildName, or BuildOrderNumber in the where clause.

Note: The view RV_ConfigurationSuiteStatus only contains aggregated status counts without any test-specific data. To retrieve additional test-specific data, you can use, for example, the view RV_LatestTestStatus.