This version is still in development and is not considered stable yet.
For the latest stable version, please use JUnit 6.0.1!

Test Engines

A TestEngine facilitates discovery and execution of tests for a particular programming model.

For example, JUnit provides a TestEngine that discovers and executes tests written using the JUnit Jupiter programming model (see Writing Tests and Extension Model).

JUnit Test Engines

JUnit provides three TestEngine implementations.

Custom Test Engines

You can contribute your own custom TestEngine by implementing the interfaces in the junit-platform-engine module and registering your engine.

Every TestEngine must provide its own unique ID, discover tests from an EngineDiscoveryRequest, and execute those tests according to an ExecutionRequest.

The junit- unique ID prefix is reserved for TestEngines from the JUnit Team

The JUnit Platform Launcher enforces that only TestEngine implementations published by the JUnit Team may use the junit- prefix for their TestEngine IDs.

  • If any third-party TestEngine claims to be junit-jupiter or junit-vintage, an exception will be thrown, immediately halting execution of the JUnit Platform.

  • If any third-party TestEngine uses the junit- prefix for its ID, a warning message will be logged. Later releases of the JUnit Platform will throw an exception for such violations.

In order to facilitate test discovery within IDEs and tools prior to launching the JUnit Platform, TestEngine implementations are encouraged to make use of the @Testable annotation. For example, the @Test and @TestFactory annotations in JUnit Jupiter are meta-annotated with @Testable. Consult the Javadoc for @Testable for further details.

If your custom TestEngine needs to be configured, consider allowing users to supply configuration via configuration parameters. Please note, however, that you are strongly encouraged to use a unique prefix for all configuration parameters supported by your test engine. Doing so will ensure that there are no conflicts between the names of your configuration parameters and those from other test engines. In addition, since configuration parameters may be supplied as JVM system properties, it is wise to avoid conflicts with the names of other system properties. For example, JUnit Jupiter uses junit.jupiter. as a prefix of all of its supported configuration parameters. Furthermore, as with the warning above regarding the junit- prefix for TestEngine IDs, you should not use junit. as a prefix for the names of your own configuration parameters.

Although there is currently no official guide on how to implement a custom TestEngine, you can consult the implementation of JUnit Test Engines or the implementation of third-party test engines listed in the JUnit wiki. You will also find various tutorials and blogs on the Internet that demonstrate how to write a custom TestEngine.

HierarchicalTestEngine is a convenient abstract base implementation of the TestEngine SPI (used by the junit-jupiter-engine) that only requires implementors to provide the logic for test discovery. It implements execution of TestDescriptors that implement the Node interface, including support for parallel execution.

Registering a TestEngine

TestEngine registration is supported via Java’s ServiceLoader mechanism.

For example, the junit-jupiter-engine module registers its org.junit.jupiter.engine.JupiterTestEngine in a file named org.junit.platform.engine.TestEngine within the /META-INF/services folder in the junit-jupiter-engine JAR.

Requirements

The words "must", "must not", "required", "shall", "shall not", "should", "should not", "recommended", "may", and "optional" in this section are to be interpreted as described in RFC 2119.

Mandatory requirements

For interoperability with build tools and IDEs, TestEngine implementations must adhere to the following requirements:

  • The TestDescriptor returned from TestEngine.discover() must be the root of a tree of TestDescriptor instances. This implies that there must not be any cycles between a node and its descendants.

  • The hierarchy of test descriptors returned from TestEngine.discover() must be mutable, but the test descriptors must otherwise be immutable after discovery.

  • A TestEngine must be able to discover UniqueIdSelectors for any unique ID that it previously generated and returned from TestEngine.discover(). This enables selecting a subset of tests to execute or rerun.

  • The executionSkipped, executionStarted, and executionFinished methods of the EngineExecutionListener passed to TestEngine.execute() must be called for every TestDescriptor node in the tree returned from TestEngine.discover() at most once. Parent nodes must be reported as started before their children and as finished after their children. If a node is reported as skipped, there must not be any events reported for its descendants.

Enhanced compatibility

Adhering to the following requirements is optional but recommended for enhanced compatibility with build tools and IDEs:

  • Unless to indicate an empty discovery result, the TestDescriptor returned from TestEngine.discover() should have children rather than being completely dynamic. This allows tools to display the structure of the tests and to select a subset of tests to execute.

  • When resolving UniqueIdSelectors, a TestEngine should only return TestDescriptor instances with matching unique IDs including their ancestors but may return additional siblings or other nodes that are required for the execution of the selected tests.

  • TestEngines should support tagging tests and containers so that tag filters can be applied when discovering tests.

  • A TestEngine should cancel its execution when the CancellationToken it is passed as part of the ExecutionRequest indicates that cancellation has been requested. In this case, it should report any remaining TestDescriptors as skipped but not report any events for their descendants. It may report already started TestDescriptors as aborted in case they have not been executed completely. If a TestEngine supports cancellation, it should clean up any resources that it has created just like if execution had finished regularly.

Reporting Discovery Issues

Test engines should report discovery issues if they encounter any problems or potential misconfigurations during test discovery. This is especially important if the issue could lead to tests not being executed at all or only partially.

In order to report a DiscoveryIssue, a test engine should call the issueEncountered() method on the EngineDiscoveryListener available via the EngineDiscoveryRequest passed to its discover() method. Rather than passing the listener around, the DiscoveryIssueReporter interface should be used. It also provides a way to create a Condition that reports a discovery issue if its check fails and may be used as a Predicate or Consumer. Please refer to the implementations of the test engines provided by JUnit for examples.

Moreover, Engine Test Kit provides a way to write tests for reported discovery issues.