Precept.Reporting.ReportPortal

ReportPortal reporting for Precept: a launch per run, a suite per feature, a test item per scenario, with logs and artifacts attached.

dotnet add package Precept.Reporting.ReportPortal

public sealed class Startup : IPreceptStartup
{
    public void ConfigureServices(IServiceCollection services, PreceptSettings settings) =>
        services.AddPreceptReportPortalReporting();
}

{
  "Reporting": {
    "ReportPortal": {
      "Endpoint": "https://rp.example.com",
      "Project": "acme_regression",
      "LaunchName": "Checkout regression",
      "Attributes": { "suite": "regression" }
    }
  }
}

Supply the API key from CI as PRECEPT_REPORTING__REPORTPORTAL__APIKEY; a key in a committed precept.json is a key anyone with the repository can post launches with.

Launches are opened from a build agent only. A developer's run reports nothing and asks for nothing — the reporter is never started, so the API key that machine does not have is not an error there. Add { "Reporting": { "CiOnly": false } } to precept.local.json to report from your own machine, and see CI and local runs for how a run is recognised as CI.

Each .feature becomes a suite, named by its feature title rather than by the class Reqnroll generated from it and described by the narrative under that title, and each scenario a test item tagged with the scenario's own Gherkin tags. Whatever the test logged goes up as log entries, a failure as an error with its stack trace, and the files a failing test captured — Playwright screenshots, traces, request logs — as attachments on the item, so a failure in the dashboard opens with the screenshot already there. Only a failing test's files are attached unless AttachOnSuccess says otherwise, and only files up to MaxAttachmentBytes — 8 MB by default, which a Playwright trace often exceeds; a larger file is skipped without an error. See what gets attached. Skipped tests are closed as NOT_ISSUE so nobody has to triage a test that was never meant to run. A scenario that passed only after a retry is one test item carrying its attempt count, not a chain of them: ReportPortal's own retry flag nests an item under an earlier one and leaves it out of the launch's statistics, and the earlier one is a report Precept never sends.

Set "LaunchMode": "DEBUG" alongside "CiOnly": false to keep the launches a developer's machine does send out of the project's statistics.

A run on an Azure DevOps agent describes its launch with a link back to the build, from the variables the agent already sets (BUILD_BUILDID, SYSTEM_TEAMFOUNDATIONCOLLECTIONURI, SYSTEM_TEAMPROJECT) — so a launch somebody opens from a chat message leads back to the pipeline run that produced it. It is appended to your Description rather than replacing it, and a run anywhere else — a developer's machine, another CI — has nothing appended.

The link goes the other way too: the launch's own page is posted to the run's PreceptRunLinks as soon as it is opened, so a run that also reports to Teams gets an Open ReportPortal button on its card. Nothing to configure — a launch id is assigned by the server, so it could not have been. See linking a run's reports to each other.

Reporting runs on a queue of its own, so a slow ReportPortal deployment costs the run nothing until the very end, where it gets Reporting:ShutdownTimeoutSeconds to finish sending.

Part of Precept, a .NET 10 test automation framework. Full documentation: reporting · configuration · artifacts