Execution model

Tests are grouped by declaring class, and MaxParallelism bounds how many units of work are in flight at once. ParallelScope decides what a unit is:

Anything marked [NonParallelizable] runs afterwards, one class at a time, whichever scope is set — a single non-parallelizable test takes its whole class into that phase, so its [BeforeSuite]/[AfterSuite] pair stays in one piece.

Why a high MaxParallelism can look ignored#

Because the default unit is a class, and the Reqnroll generator emits one class per .feature file, a suite of two feature files never puts more than two units in flight — "MaxParallelism": 8 leaves six slots empty and the run is as slow as its longest feature. Precept says so at the start of a run when the gap is large enough to matter.

Setting "ParallelScope": "Test" schedules every scenario individually, so few-but-long feature files can use the whole limit. What you give up is ordering within a feature:

• Scenarios of one feature no longer see each other's leftovers in a fixed order — a static field, a row a previous scenario inserted, a value stashed in a step-definition field.
• Reqnroll takes a pooled runner per scenario, so its own [BeforeFeature]/[AfterFeature] hooks run once per runner rather than once per feature. This is how Reqnroll behaves under every parallel-capable provider.
• Precept's own [BeforeSuite] still completes before the class's first test starts, and [AfterSuite] still runs after its last one finishes.

Put [NonParallelizable] on the few classes that genuinely need the old ordering and leave the rest test-scoped, rather than holding the whole run back for them.

Retries, timeouts and reporting#

[Retry(n)] counts reruns, not runs: a test always runs once, and n is how many further goes it gets while it keeps failing. [Retry(1)] runs a failing test twice, [Retry(0)] is the same as no attribute at all. "DefaultRetries" in precept.json says the same for tests carrying no attribute, and defaults to 0.

Retry only reruns hard failures — a skip or an inconclusive verdict will not change on a rerun. Each result carries timing information, category metadata and artifact paths through to the platform's reporters.

A single test has no time limit by default. An end-to-end test takes as long as the system under test makes it take, and a limit chosen in advance for every test is either too short for the slowest or too long to mean anything. What a run needs protecting from is never ending — a step waiting on a socket that will not answer, a loop that never exits — and one ceiling on the run is enough for that: "RunTimeoutMinutes" in precept.json, 12 hours (720) unless the file says otherwise, 0 for no limit at all. When it fires, every test still running is cut off and reported as failed with the reason, every test not yet started is reported as failed without running, and the runner writes one line saying the run was over time. The TRX and the reporters get a verdict for everything the run selected, which a build agent killing the process would never have given them.

A test that wants its own limit can have one — [Timeout(ms)] on the test or its class, or "DefaultTimeoutMilliseconds" for every test carrying no attribute, which defaults to 0. Either limit is enforced in two steps. When the time is up, PreceptTestContext.Current.CancellationToken is cancelled, and a body that stops on it fails with a TimeoutException there and then. A body that never looks at the token — every Reqnroll step, and most tests — is abandoned instead: the verdict is the same TimeoutException, reported at the timeout rather than whenever the body would have finished, and the [AfterTest] hook, the module verifications and disposal all run as they do for any failure, so a browser test that overran still gets its screenshot and trace. The abandoned body keeps running in the background until what it holds is disposed under it, which is usually at once; anything it throws after that is logged to the test, not raised. Observe the token where a test can — a wait, an HTTP call, a database command — and it ends cleanly instead.

No limit is enforced while a debugger is attached — not the run's and not a test's. The clock keeps running at a breakpoint, and a limit that fires there fails the test being looked at and closes its browser under the person looking. Debug for as long as the investigation takes; the limits return on the next run without a debugger.

A test that was actually rerun keeps every attempt in its output, each block headed by the attempt it belongs to and how that attempt ended:

[Attempt 1 of 3] Failed: Expected status 200 but received 503.
Given the catalogue service is reachable
When the client requests the catalogue
[Attempt 2 of 3] Passed
Given the catalogue service is reachable
When the client requests the catalogue

Without it a scenario that failed twice and passed on the third reported one clean green run, with nothing to say what had gone wrong. The header appears only when a rerun actually happened, so a test that passes first time reads as it always did. A test's duration covers every attempt it made, including the DelayMilliseconds between them.

The same goes for what the attempts wrote. A rerun gets a directory of its own under the test's — attempt-2, attempt-3 — rather than writing its failure.png and trace.zip over the first attempt's, and the result attaches every attempt's files, each labelled [Attempt n of m] the way the log is. A test that passed on its rerun therefore still carries the screenshot and trace of the failure it retried past, which is the thing a flaky test has to show.