Writing tests

Gherkin#

@smoke
Scenario Outline: Adding two numbers
	Given I have entered <left> into the calculator
	And I have entered <right> into the calculator
	When I press add
	Then the result should be <total> on the screen

	Examples:
		| left | right | total |
		| 1    | 2     | 3     |
		| -5   | 5     | 0     |

Step definitions are ordinary Reqnroll [Binding] classes. Everything Reqnroll supports — data tables, ScenarioContext, constructor injection, hooks — works unchanged.

Configuring Reqnroll#

Reqnroll is configured from the Reqnroll section of precept.json. There is no reqnroll.json to write: Precept.Reqnroll registers itself as the unit test provider on both sides, and the section carries what that file would otherwise have held.

{
  "Reqnroll": {
    "MissingOrPendingStepsOutcome": "Error",
    "TraceTimings": true,
    "AddNonParallelizableMarkerForTags": [ "serial" ]
  }
}

Every key defaults to what Reqnroll itself defaults to, so a project with no section runs exactly as one with no reqnroll.json did. Every setting lists the keys, and the section is overlaid and overridden like any other — with one distinction worth knowing:

• Runtime keys — the binding culture, StopAtFirstError, the outcome of a missing step, the trace settings, BindingAssemblies — are read when the first scenario starts, out of the run's real configuration, so precept.{environment}.json and PRECEPT_REQNROLL__* apply to them.
• Build-time keys — FeatureLanguage, AllowRowTests, AddNonParallelizableMarkerForTags, AllowDebugGeneratedFiles, DisableFriendlyTestNames — shape the generated code, so they are read from the project's precept.json while the feature files compile. An overlay or a variable cannot reach them: the code-behind is compiled once, before any environment is chosen. Editing the file regenerates it.

A project that still has a reqnroll.json keeps working. The keys the section models are taken from precept.json whether or not the file sets them; only what the section does not model — dependencies, and the Cucumber Messages formatters — is still read from reqnroll.json, which is the one reason to keep one. To override a key in code, register the settings the way you would your own, and the plugin reads that instance first:

services.AddPreceptSettings<PreceptReqnrollSettings>(configure: r => r.StopAtFirstError = true);

Plain C##

[TestSuite("Checkout")]
[TestCategory("smoke")]
public class CheckoutTests
{
    [Test("A customer can check out")]
    [Retry(2, DelayMilliseconds = 200)]
    public async Task Checkout()
    {
        var response = await Rest.Post("/orders")
            .WithJsonBody(new { sku = "ABC", quantity = 2 })
            .SendAsync();

        await Assert.That(response).ToHaveStatusAsync(HttpStatusCode.Created);
        await Assert.That(response).ToHaveJsonValueAsync("status", "created");
    }
}

Attributes#

Discovery is strict about what it finds, and fails the session rather than running what it could see. A hook with the wrong shape, a test taking parameters with no [TestCase] to supply them, and a test assembly some of whose types cannot be loaded — a referenced assembly missing from the output directory, or present at a different version from the one the project was compiled against — all stop the run with the cause on the console. A suite that lost tests to a bad restore would otherwise report green with fewer results and nothing to say why.

Outcomes#

Throw PreceptIgnoreException to skip at runtime, PreceptPendingException for unimplemented work, PreceptInconclusiveException when the environment cannot produce a verdict, and PreceptAssertionException for assertion failures — which is what assertions throw.

Test explorer navigation#

Every discovered test reports the file and line it was written at, so "Go to test" in Visual Studio, Rider or the VS Code Testing pane opens the source — and for a Gherkin test that source is the .feature file, at the Scenario: line, not the generated code-behind. An Examples row navigates to its own line in the table.

Nothing needs to be configured for this. Precept reads the test assembly's portable PDB and takes the first sequence point of each test method; Reqnroll's code-behind carries #line directives back into the feature file, so that sequence point already names the Gherkin. Async and iterator methods are followed onto their state machine, where the compiler actually put the body.

The generated classes also carry [TestSource("Features/Checkout.feature", 12)], which takes precedence and keeps navigation working for a build without symbols. The path is project-relative so generated code stays free of machine-specific directories; the runner puts the directory back from the PDB, or by searching upwards from the test binary.

Two things turn navigation off:

• <DebugType>none</DebugType>, or any build that does not put a .pdb beside the test binary. Embedded PDBs (<DebugType>embedded</DebugType>) are read too. Plain C# tests then have no location at all; Gherkin tests fall back to the attribute.
• A deterministic build (ContinuousIntegrationBuild=true) rewrites source paths to a placeholder root. Precept checks that a path from the PDB exists before believing it, so Gherkin tests still resolve through the attribute — but plain C# tests will not resolve on the build agent. This does not affect a developer machine.

Apply Precept.Generated.TestSourceAttribute by hand when a test is generated by something other than Reqnroll and should point at whatever produced it. It is public for that reason and no other: it sits in its own namespace, out of IntelliSense, so it does not crowd the attributes tests are written with.