Concepts
Everything in Performance belongs to a workspace. Switch workspace from the header before you create or open scenarios. You only see data for workspaces you belong to. Organization operators may browse more broadly for support; day-to-day create/edit stays inside your membership role. Org settings writes still need org owner / admin — see Access & secrets.
Workspace boundary
Building blocks
| Term | Meaning | Typical home in the app |
|---|---|---|
| Environment | Named settings and variables (base URLs, secrets) used by scenarios | Workspace → Environments |
| Scenario | A k6 script plus options, usually linked to an environment | Performance → Scenarios |
| Scenario version | Snapshot of a script/options at a point in time | Scenario editor → Metadata → version history |
| Run | One execution of a scenario | Performance → Runs |
| Collection | Ordered list of scenario jobs you can run together | Performance → Collections |
| Collection run | One planned/executed pass of a collection (pending → running → finished) | Hub · Collections · Matrix |
| Matrix | Table of results across a single collection run (exportable) | Performance → Matrix |
| Reports (collection window) | Runs for all scenarios in a collection within a date range; highlights gaps | Performance → Reports → One collection |
| Reports (bundle) | Same evidence model across multiple collections with rollup KPIs | Performance → Reports → Bundle |
| Saved report / snapshot | Frozen JSON snapshot of a window (single or bundle); viewer, PDF, share | Performance → Reports → Saved |
| Trends | Daily rollups per scenario + environment | Performance → Trends |
| Connection | Link to an external Grafana or Prometheus-style system | Performance → Connections |
Matrix, Reports, and Trends — which view?
These three surfaces share a similar evidence grid layout, but they answer different questions. Pick by what you need to prove or monitor — not by which tab looks familiar.
| Matrix | Reports → Collection window | Trends | |
|---|---|---|---|
| Question | “Did this gate batch pass?” | “What ran for this collection (or bundle) in this period?” | “How is health drifting over time?” |
| Scope | One collection run (single batch execution) | One collection or bundle (up to 20 collections) + date range | Scenario × environment × day (UTC rollups) |
| Each row is… | One job in that batch (latest run for that scenario in the run) | One finished run that matches the window | One daily aggregate (many runs rolled up) |
| Typical trigger | spanline test collection-run --id … after Start run |
Separate test run --sync jobs, cron-style smokes, mixed schedules |
Ongoing runs over days/weeks |
| Key filters | Collection run (?collectionRunId=) |
Collection, From, To, optional Status | Scenario, Environment, From, To |
| Per-run links | Yes — open run, Compare as A, metrics | Yes — same evidence columns as Matrix | No — summary metrics only |
| Gaps | Missing jobs in this batch | Scenarios in the collection with no run in the window | N/A (empty day = no rollup row) |
| Export / audit | Export CSV (in app); JSON via CLI | Export CSV + Save snapshot (Saved tab) | Rebuild rollups (admin); no CSV yet |
| Best for | Release gate sign-off for one planned execution | Compliance / “what ran this week?” when jobs are not one batch | Spotting pass-rate or p95 drift, env comparisons |
Decision guide
Did you run ONE collection-run plan (Start run → CLI collection-run)?
└─ Yes → Matrix
└─ No → Did you need every scenario in a collection over a calendar window?
└─ Yes → Reports → Collection window
└─ No → Do you care about day-by-day aggregates, not individual runs?
└─ Yes → Trends
└─ No → Runs (history) or Compare (two runs)
Matrix vs Reports (most common confusion): Matrix is bound to one collectionRunId. If smoke A ran at 08:00 and smoke B at 09:00 as separate runs, Matrix for a batch that never happened will be empty or incomplete — use Reports → Collection window with the collection and dates instead.
Reports vs Trends: Reports lists individual runs (with run ids and evidence links). Trends aggregates finished runs into daily KPIs (pass rate, p95, RPS, error rate) — use it for monitoring, not for gate evidence on a specific release.
Compare is separate: exactly two runs, side-by-side deltas — not a collection or time-series view.
| Artifact | File attached to a run (logs, zip, and similar) | Run detail → Artifacts |
| Project (optional) | Durable product identity for filtering lists | Scenario / collection editor in the product |
App vs runner
| Surface | Role |
|---|---|
| Spanline app | Catalog scenarios, start collection plans, observe runs, compare, export matrix CSV, manage connections |
| Runner (CLI / CI) | Execute k6 and upload live metrics + summary (+ optional artifacts) |
The browser does not run k6 for you. Overview empty states remind you to create a scenario, then run it with spanline.
Lifecycle (happy path)
- Prepare — Environment variables → Scenario script & options
- Execute — CLI/CI starts a run (or executes a collection-run plan)
- Observe — Hub / Runs stream live samples; abort if needed
- Analyze — Compare two runs; open Matrix for one collection run, or Reports → Collection window when runs span different times
- Share — Export CSV (app) or JSON (CLI); download artifacts; optional external Connection
Collection-run states
| State | Meaning |
|---|---|
| pending | Plan created (Start run or webhook); waiting for a runner |
| running | CLI/CI is executing jobs |
| finished (passed/failed/…) | Jobs completed; open Matrix for evidence |
Roles (workspace)
| Role | Typical access |
|---|---|
| Viewer | Read scenarios, runs, matrix; reveal env values when allowed |
| Member | Create and edit scenarios/collections; start collection-run plans |
| Admin / owner | Member rights plus delete and collection webhook management |
Exact labels follow your workspace membership. If a control is disabled, ask an admin — it is usually a role or plan limit, not a product bug.
Overview KPIs
| KPI | Meaning |
|---|---|
| Total runs | How many runs exist in the workspace |
| Pass rate | Finished runs that passed |
| Inventory | Envs, scenarios, collections — and how many look ready for a gate |
| Running now | Live streaming runs |
Inventory can be non-zero while Total runs is still zero (catalog without executions yet).
Compare
Open two runs in the same workspace as Baseline (A) and Candidate (B) to see throughput, latency, and check differences (deltas are B − A). Saving named comparison bookmarks is not available yet.
Quotas and plans
Your organization plan may limit how many runs you can create in a period. If a run is rejected for quota, contact a workspace or org admin.
Coming later
Rail tabs marked Soon: Scripts, Thresholds only. Collection cron is stored as metadata only — Spanline does not schedule the job yet. For Matrix / Reports / Trends, see which view? above.