Guides

Day-to-day flows in the Spanline app. For the first smoke test, see Getting started. For runner commands, see Run tests.

Prepare an environment

Environments live under the workspace, not inside the Performance rail.

  1. Open Workspace → Environments (/workspaces/environments).
  2. Create an environment and add variables your scripts need (BASE_URL, tokens, and similar).
  3. Mark sensitive values as secrets. When your organization uses sealed secrets, the app encrypts those values so they are not stored in plain text — see Access & secrets.
Workspace Environments list
Workspace → Environments — shared variable sets for scenarios.

Create a scenario

  1. Open Performance → Scenarios.
  2. Choose New scenario, or use a template from the empty state (Smoke, Steady, Spike, Soak) when the catalog is empty.
  3. Set a clear name, paste your k6 script (or keep the template), configure options/stages/thresholds as needed, and attach an environment.
  4. Optionally link a Project in the editor so the toolbar project filter can narrow lists.

Sync from disk (CI / local-first) is done with the CLI (--sync --name), not from a button in the editor.

Scenarios catalog
Scenarios catalog — create, search, filter, and copy a CLI snippet per row.

Catalog tips

Control What it does
Search / Env / Template / Owner Narrow the list
Project · All (toolbar) Personal filter for this browser; not a global header setting
Copy CLI (row) Copies spanline test run --scenario-id …
Checkbox + Add to collection Bulk-add selected scenarios into a collection
After save “Next in the chain” offers copy CLI, add to collection, or open last run

Scenario editor tabs

Open a scenario (Edit) to configure it across tabs:

Tab What you set
Metadata Name, project, tags, notes, version history
Load Executor, stages (ramp/hold/down), graceful stop, runtime toggles
Gates Thresholds / check gates for pass-fail
JavaScript k6 script and options
Environment Linked environment, variable preview, override mode
Validate Client-side dry-run checks before you save
Scenario editor Metadata tab
Metadata — identity, project link, tags, and version history.
Scenario editor Load tab with stages
Load — ramping-vus stages, RPM estimate, and runtime flags.
Scenario editor Gates tab
Gates — thresholds that decide pass/fail.
Scenario editor JavaScript tab with k6 script
JavaScript — the k6 script the runner executes.
Scenario editor Environment tab
Environment — attach Workspace environments and review variables.
Scenario editor Validate tab
Validate — dry-run checks before you commit the scenario.

Version history

On Metadata, the version history table lists script snapshots. Restore loads an older snapshot into the editor draft; Save writes a new version. Pin a version from the CLI with --version-id when you need a known script for a release gate.

Scenario version history on Metadata tab
Version history — restore a previous script snapshot, then save.

After save, copy the scenario id from the UI (or use --name / --sync from the CLI) when you run from a laptop or CI.

Run from the CLI

Use the steps in Getting started or Run tests. After the run starts, open it from Overview, Scenarios, or Runs to watch live metrics and the final summary.

Tip from Overview: the hub shows a ready-to-copy snippet with spanline test run --scenario-id <id>.

Watch and abort a run

  1. Open Performance → Runs (or jump from Overview → recent runs).
  2. Use search and filters (status, source, scenario) when the list is long.
  3. Open a run to follow live samples (VUs, RPS, latency, errors) as they stream in.
  4. Use Abort if you need to stop early — the runner picks up the abort on its next check.
Runs history list with statuses and p95
Runs — filterable history with status, p95, and duration.
Live run detail with VUs RPS p95 and progress
Run detail — live summary while the CLI streams metrics (Abort stays available).
Finished run detail with passed summary
Finished run — Summary with thresholds, checks, and downloads.

Run detail tabs (finished runs)

While a run is live you mainly use the Summary view. After it finishes, extra tabs appear:

Tab / control What you get
Summary Metrics strip, progress, thresholds, checks, console tail
Metrics Latency charts and related observation panels
Artifacts Files uploaded with --upload-artifacts (download each file)
Infra Infrastructure snapshots (for example Kite) when collectors recorded them
Download (header) Downloads the summary JSON — separate from the Artifacts tab
Compare Opens Compare with this run as a starting point
Run detail Metrics tab
Metrics — charts after the run finishes.
Run detail Artifacts tab
Artifacts — download uploaded bundles and logs.
Run detail Infra tab with Kite deployment snapshots
Infra — before/after deployment readiness, pod table, and resource usage (demo fixture data from seed).

Compare two runs

  1. Open Performance → Compare.
  2. Pick Baseline (A) and Candidate (B) in the same workspace (deep links also accept ?a= / ?b=).
  3. Review throughput, latency, and check deltas (shown as B − A).

Other entry points:

Compare two runs screen
Compare — Baseline (A) vs Candidate (B) with side-by-side deltas.

All three use an evidence-style table, but they are not interchangeable. Full comparison: Concepts → which view?.

You need… Open
Sign-off for one collection-run batch Matrix
Every scenario in a collection across separate run times / dates Reports → Collection window
Daily health rollups (pass rate, p95 drift) Trends

If you are unsure between Matrix and Reports: Matrix needs a collection run id from Start run + test collection-run. Reports needs only a collection and from/to dates — use it when jobs did not share one batch plan.

Collections and matrix

Collections are ordered scenario packs for a release-style gate.

  1. Open Performance → Collections and create a collection (or open a row to edit).
  2. In the editor, use Metadata (name, project, notes, owners, trigger) and Scenarios (ordered jobs).
  3. Click Start run — that creates a pending collection-run plan and opens the execution drawer.
  4. Copy the collection-run id and execute with the CLI (spanline test collection-run --id …) or your pipeline — see Run tests.
  5. Watch job progress in the drawer. Use Abort remaining to stop unfinished jobs.
  6. When finished, open Performance → Matrix (or the drawer’s matrix link, optionally ?collectionRunId=) and use Export CSV.

Reminder: Matrix is tied to one collection run. Separate schedules → Reports → Collection window. Long-term drift → Trends.

Collections list
Collections — ordered jobs for gate-style batch runs.
Collection editor drawer
Collection editor — Metadata and Scenarios tabs; Start run creates a pending plan.
Collection run execution drawer
Collection run — Collections page with the execution drawer open (progress, Abort remaining, CLI snippet, Open matrix).
Matrix evidence view
Matrix — exportable evidence across a single collection run (Export CSV in the app).

Reports {#reports}

Use Performance → Reports for calendar-window evidence when Matrix’s single collection-run batch is not the right scope.

Mode When
One collection One collection + From / To — gaps for missing scenarios
Bundle Up to 20 collections (often via Project filter) — rollup KPI + accordion per collection

Quick steps: pick scope → collection(s) → dates → review KPIs and grid → Export CSV or Save snapshotSaved tab → Open for the full-page viewer, PDF, workspace embed, or public unlock link.

Full detail (viewer page, Charts layout, 24h embed tokens, URL table, roles): Reports & sharing.

Reports collection window with runs and gaps
Reports → Collection window — evidence grid with KPIs and gaps.
Reports Saved tab with stored snapshots
Reports → Saved — persisted snapshots; **Open** opens the viewer page.
Full-page report viewer
Saved snapshot viewer — toolbar for PDF and Embed & share.

Performance → Trends is for monitoring, not gate evidence. It shows daily rollups per scenario and environment: pass rate, p95 (best/worst), RPS, and error rate — one row per day, not one row per run.

Performance Trends daily rollups
Trends — daily rollups in the same evidence-grid layout as Matrix; pass-rate badges and Rebuild rollups.

Collection webhooks

On the collection editor Metadata area, Webhooks lets an external system start a collection-run plan.

  1. Save the collection first (webhooks need an id).
  2. Create a webhook; copy the secret when it is shown (shown once).
  3. Use the inbound curl example from the UI (header includes the webhook secret).
  4. Have a runner pick up the plan with spanline test watch --once (or your team’s equivalent).
  5. Rotate or delete the webhook if the secret may have leaked.

Managing webhooks typically requires admin / owner. Keep secrets out of tickets and screenshots — see Access & secrets.

Triggers (what ships today)

Trigger Meaning
manual You click Start run in the app
webhook Inbound secret creates a plan; a runner executes it
cron Schedule fields are metadata only — Spanline does not fire the cron yet. Use external CI or test watch until registered runners ship

Connections

Add a Connection when you want Spanline to know about an external Grafana, Prometheus-style, or related endpoint.

  1. Open Performance → Connections.
  2. Create a connection and fill endpoint details.
  3. Test the connection from the app (Test all when you have several).
  4. Keep tokens in sealed secret fields when available.
Performance Connections screen
Connections — external dashboard / metrics endpoints.

Project filter

On Scenarios and Collections, the toolbar control labeled Project · All filters lists for your browser session. Link a project when you create or edit a scenario/collection so rows participate in that filter.

Overview KPIs (quick read)

KPI Meaning
Total runs Runs recorded in this workspace
Pass rate Share of finished runs that passed
Inventory Counts of envs / scenarios / collections (and “ready for gate”)
Running now Live runs currently streaming

Empty states (No runs yet, No collection runs yet) mean no runner has uploaded yet — not that the product is broken. Chain reminder on the hub: Scenarios → Collections (plan) → CLI collection-run → Matrix / Compare, or ad-hoc runs → Reports (collection window) / Trends.

Coming later (not documented as available)

Product rail Soon tabs — Scripts, Thresholds — are placeholders. Named saved comparison links and in-app Matrix JSON export are not available yet (use CLI test matrix --format json for JSON).