A leader asks whether engineering quality is improving or getting worse. The honest answer, in most organisations, is “I think so?”, followed by an anecdote about a recent incident, or a coverage percentage pulled from whichever tool happened to be open. Neither is evidence. Both are guesses dressed up to sound confident in a meeting.
Why CI logs aren’t a reporting tool
Test results genuinely do contain the answer to that question, but they live in CI logs that nobody can query, compare, or chart over time. A single build log tells you what happened on one run. It says nothing about the trend across the last month, and it certainly doesn’t survive long enough to compare against a build from a quarter ago.
That leaves sprint and release retrospectives without dated evidence to point to. “Are we getting more stable” becomes a matter of who in the room shouts loudest about their most recent memorable incident, rather than something anyone can look up and settle with data. Ask the same question again next quarter and the comparison is even harder, because whatever evidence existed for this quarter has usually already rotated out of the build system by then.
Dated, auditable evidence: dashboards, trends, and exports
Obvyr’s project dashboard turns raw execution data into something a leader can actually read: headline metrics (total tests, total executions, pass rate, flakiness rate), time-series trend charts covering unique tests run, execution volume, and pass/fail/skip rates over time, a date-range selector so you’re never stuck looking at an all-time average by default, and rankings of the most-failed, flakiest, and slowest tests so attention goes to the right place first. None of it requires someone to manually assemble a report from CI logs.
For point-in-time reporting (a board update, a retrospective, a compliance request), the same data exports directly to PDF, giving you a dated snapshot you can archive, share, or compare against a later one. You don’t even have to remember to generate one: Obvyr automatically produces a monthly and quarterly report for every project (the same snapshot, trend charts, and PDF export as an on-demand report), sitting ready in your reports list whenever the retrospective or leadership update comes around.
What this doesn’t do yet: milestones (bookmarking a specific sprint or release so every observation captured within that window is automatically tagged) are on our roadmap, not shipped today. For now, the dashboard’s date-range selector and the automatic monthly/quarterly reports are the way to isolate how a specific period looked.
Why a fixed cadence beats a manual one
The reports most likely to matter are exactly the ones most likely to get skipped: the ones due during a crunch, a reorg, or the week everyone’s underwater on a release. A workflow that depends on someone remembering to generate a snapshot tends to produce evidence when things are calm and gaps when they aren’t, which is backwards: a difficult quarter is precisely when leadership wants dated proof of what actually happened, not a shrug.
A fixed monthly and quarterly cadence sidesteps that failure mode entirely. The report exists whether or not anyone thought to ask for it, which means a retrospective always has something to point to, and a compliance request always has a paper trail that predates the request itself rather than being assembled in response to it. It’s a small mechanical difference (automatic versus on-demand), but it’s the difference between evidence that survives a busy quarter and evidence that quietly stopped existing right when someone needed it.
Even without milestone bookmarking, the shift from “I think so?” to “here’s the trend chart, and here’s last quarter’s report” is the difference between reporting quality by anecdote and reporting it by evidence.
For a leader responsible for more than one project, the same dashboards feed directly into comparing test health across your whole portfolio.