Writing tests
A Precept test is a C# method with a [Test] attribute, in a class with a [TestSuite] attribute.
You can write those classes by hand, or let Precept.Reqnroll generate them from .feature files.
Both kinds can live in one project.
Gherkin#
@smoke
Scenario Outline: Adding two numbers
Given I have entered <left> into the calculator
And I have entered <right> into the calculator
When I press add
Then the result should be <total> on the screen
Examples:
| left | right | total |
| 1 | 2 | 3 |
| -5 | 5 | 0 |
Step definitions are ordinary Reqnroll [Binding] classes. Everything Reqnroll supports works
unchanged: data tables, ScenarioContext, constructor injection, hooks.
Configuring Reqnroll#
Reqnroll is configured from the Reqnroll section of precept.json. There is no reqnroll.json
to write: Precept.Reqnroll registers itself as the unit test provider, and the section carries
what that file would otherwise hold.
{
"Reqnroll": {
"MissingOrPendingStepsOutcome": "Error",
"TraceTimings": true,
"AddNonParallelizableMarkerForTags": [ "serial" ]
}
}
Every key defaults to Reqnroll's own default, so a project with no section runs exactly as one with
no reqnroll.json. Every setting lists the keys. The section
is overlaid and overridden like any other, with one distinction:
- Runtime keys are read when the first scenario starts, from the run's real configuration. The
binding culture,
StopAtFirstError, the outcome of a missing step, the trace settings andBindingAssembliesare runtime keys, soprecept.{environment}.jsonandPRECEPT_REQNROLL__*apply to them. - Build-time keys shape the generated code, so they are read from the project's
precept.jsonwhile the feature files compile.FeatureLanguage,AllowRowTests,AddNonParallelizableMarkerForTags,AllowDebugGeneratedFilesandDisableFriendlyTestNamesare build-time keys. An overlay or a variable cannot reach them, because the code-behind is compiled once, before any environment is chosen. Editing the file regenerates it.
A project that still has a reqnroll.json keeps working. The keys the section models are always
taken from precept.json. Only dependencies and the Cucumber Messages formatters are still read
from reqnroll.json, which is the one reason to keep that file.
To override a key in code, register the settings the way you would your own. The plugin reads that instance first:
services.AddPreceptSettings<PreceptReqnrollSettings>(configure: r => r.StopAtFirstError = true);
Plain C##
[TestSuite("Checkout")]
[TestCategory("smoke")]
public class CheckoutTests
{
[Test("A customer can check out")]
[Retry(2, DelayMilliseconds = 200)]
public async Task Checkout()
{
var response = await Rest.Post("/orders")
.WithJsonBody(new { sku = "ABC", quantity = 2 })
.SendAsync();
await Assert.That(response).ToHaveStatusAsync(HttpStatusCode.Created);
await Assert.That(response).ToHaveJsonValueAsync("status", "created");
}
}
Attributes#
| Attribute | Effect |
|---|---|
[TestSuite] |
Marks a class as holding tests. Discovery skips a class without it. |
[Test] |
Marks a method as a test. Optional when [TestCase] is present. |
[TestCase(...)] |
One test case per row; values are converted to the parameter types. Implies [Test]. |
[TestCategory] |
A tag for filtering and reporting. |
[Ignore] |
Report as skipped without running. |
[Retry(n)] |
Rerun on failure up to n more times. [Retry(1)] gives a failing test one more go. |
[Timeout(ms)] |
Fail if the test overruns. Not enforced under a debugger. See Retries, timeouts and reporting. |
[NonParallelizable] |
Run alone, with nothing else in flight. |
[TestSource(file, line)] |
Where the test was written, for test explorer navigation. Emitted by the Reqnroll generator; lives in Precept.Generated and is rarely written by hand. |
[BeforeSuite] / [AfterSuite] |
Static, once per class. |
[BeforeTest] / [AfterTest] |
Per test. The after-hook runs even on failure. |
Discovery is strict. If something is wrong with the assembly, the session fails with the cause on
the console rather than running the tests it could see. That includes a hook with the wrong shape, a
test taking parameters with no [TestCase] to supply them, and an assembly some of whose types
cannot be loaded, for example because a referenced assembly is missing from the output directory
or present at a different version. Without this, a suite that lost tests to a bad restore would
report green with fewer results and nothing to say why.
Outcomes#
A test can report an outcome other than pass or fail by throwing one of these:
| Exception | Outcome |
|---|---|
PreceptIgnoreException |
Skipped at runtime. |
PreceptPendingException |
Pending: the work is not implemented yet. |
PreceptInconclusiveException |
Inconclusive: the environment could not produce a verdict. |
PreceptAssertionException |
Failed. This is what assertions throw. |
Test explorer navigation#
Every discovered test reports the file and line it was written at, so "Go to test" in Visual
Studio, Rider or the VS Code Testing pane opens the source. For a Gherkin test the source is the
.feature file, at the Scenario: line, not the generated code-behind. An Examples row navigates
to its own line in the table.
Nothing needs to be configured. Precept reads the test assembly's portable PDB and takes the first
sequence point of each test method. Reqnroll's code-behind carries #line directives back into the
feature file, so that sequence point already names the Gherkin. Async and iterator methods are
followed onto their state machine, where the compiler put the body.
The generated classes also carry [TestSource("Features/Checkout.feature", 12)]. It takes
precedence over the PDB and keeps navigation working for a build without symbols. The path is
project-relative, so generated code stays free of machine-specific directories; the runner works out
the directory from the PDB, or by searching upwards from the test binary.
Two things turn navigation off:
<DebugType>none</DebugType>, or any build that does not put a.pdbbeside the test binary. Embedded PDBs (<DebugType>embedded</DebugType>) are read too. Plain C# tests then have no location at all; Gherkin tests fall back to the attribute.- A deterministic build (
ContinuousIntegrationBuild=true) rewrites source paths to a placeholder root. Precept checks that a path from the PDB exists before using it, so Gherkin tests still resolve through the attribute, but plain C# tests will not resolve on the build agent. A developer machine is unaffected.
Apply Precept.Generated.TestSourceAttribute by hand when a test is generated by something other
than Reqnroll and should point at whatever produced it. It is public for that reason only. It sits in
its own namespace, out of IntelliSense, so it does not crowd the attributes tests are written with.