Artifacts and logging
PreceptTestContext.Current is the running test's context. It is ambient and flows across await, so any code the test calls can reach it. It has three jobs:
Log(...)writes a line to the test's output.AttachAsync(...)records a file against the test.RegisterForDisposal(...)ties a resource to the test. It is disposed after the after-scenario hook, so hooks can still capture diagnostics from it.
Failing a test after its last line#
RegisterVerification(...) registers a check that runs after the test body and its after-scenario hook. If the check fails, the test fails, even if every assertion in the body passed:
PreceptTestContext.Current?.RegisterVerification(() => session.VerifyNoErrorsAsync());
Use it for a verdict a module reaches on the test's behalf: a console error the page logged after the last assertion, or a request that failed after the test was already done. Precept.Web uses it for FailOnConsoleErrors and FailOnPageErrors.
Checks run in the order they were registered, before anything registered for disposal is disposed, so a check still has the page or connection it is checking. The outcome is settled before disposal, so a failure decided here still gets the module's failure screenshot and trace. A test that already failed keeps its own failure, and the check's failure is logged instead. The first thing to go wrong is the one worth reporting.
What a test logged is reported as the platform's StandardOutputProperty. It reaches a TRX as <StdOut>, a test explorer's output pane, and every reporter, whether the test passed or failed. A failure also repeats the log under its message, so a failed test explains itself even where output is suppressed.
Seeing the Gherkin steps a scenario ran#
Reqnroll traces each step as it runs. Precept routes that trace into the running test's log rather than to the process console. That is what puts the steps in the TRX, in a test explorer, and on a ReportPortal test item. It is also why two scenarios running in parallel do not interleave their steps.
Whether the steps also appear on the console is the platform's decision, and it takes two flags. The Normal output level does not list passing tests at all, so there is nowhere for their output to go:
dotnet run --project tests/Acme.Tests -- --output Detailed --show-stdout All
passed Adding two numbers (105ms)
Standard output
Given a clean calculator
-> done: CalculatorSteps.GivenACleanCalculator() (0.0s)
When I press add
add(50 + 70) = 120
-> done: CalculatorSteps.WhenIPressAdd() (0.0s)
Anything a step logs itself lands in the right place in that sequence. Under dotnet test the same flags work for failing and skipped tests, but the SDK's own logger never lists passing ones, so a passing scenario's steps are only in the TRX and the reporters. Whether successful steps are traced at all is Reqnroll's setting, TraceSuccessfulSteps in the Reqnroll section of precept.json.
Where files go#
Each test writes into its own directory under ArtifactDirectory. The directory is named after the test but kept short on purpose: Windows caps a full path at 260 characters, and past that an artifact is still written but the TRX writer's copy of it fails with "attachment not found".
The directory is created the first time something asks for a path inside it. A module that records on failure does that before the test runs, so a passing test is left with an empty directory, and that is removed once the test is over. What survives a run is one directory per test that produced a file, which is what a pipeline publishing the root should ship. A retried test's reruns write into attempt-2, attempt-3 and so on under the test's directory, so the first attempt's files survive.
What reaches the TRX#
Recorded files are reported to the platform as file artifacts of the test that produced them. With --report-trx they are copied next to the .trx and listed as <ResultFile> entries of that test's result, which is what Azure DevOps' PublishTestResults task uploads as attachments. The TRX copies every file whatever its size. The reporters that upload files apply a size cap of their own; see what gets attached.
- task: PublishTestResults@2
inputs:
testResultsFormat: VSTest
testResultsFiles: '**/*.trx'
publishRunAttachments: true
Precept implements the platform's ITrxReportCapability, so a TRX also carries the declaring class name, the test's categories as <TestCategory> entries, the lines it logged as <StdOut>, and its failure split into <ErrorInfo><Message> and <StackTrace>.
The run record#
Every local run also writes a small record of itself, in PreceptRuns next to the built test app, for
example bin/Debug/net10.0/PreceptRuns. Each run gets a folder there named after its run ID.
| File | What it holds |
|---|---|
run.json |
The run: its environment, start and end time, and how many tests passed, failed and were skipped. |
results.jsonl |
One line per finished test: its name, outcome, duration, attempts, failure, log and the paths of its files. |
The record is for tools that show runs. You do not need to read it or configure it. The dashboard copies a finished run's results into its own history, so a record that a clean build removes is not the end of the run.
- It is not written on a build agent, or when the
Dashboardsetting isNever. See Turning it off. - Listing tests with
--list-testswrites none. - A run keeps the newest 20 records in the folder and deletes older ones when it starts.
- A test's log is trimmed to its last 5,000 lines in the record. The TRX has the whole log.
- If the record cannot be written, the run prints one warning and carries on.