Ask five engineering teams what “87% pass rate” means and you’ll get five different answers: different suites, different definitions of what counts as a test, different tolerance for flakiness baked invisibly into the number. Multiply that across a portfolio of a dozen projects and there’s no honest way to compare them, because nobody’s actually comparing like with like.
No shared vocabulary for quality
This isn’t a data problem so much as a translation problem. Every project has its own pipeline, its own test framework, its own history of decisions about what gets tested and how. A pass rate or coverage number from one project doesn’t mean the same thing as the nominally identical number from another, and there’s rarely a single view that shows every project’s numbers side by side using one consistent definition.
For a leader trying to scan an entire engineering estate and identify where attention is most needed, that gap is a real problem. Without a shared vocabulary, “which projects need help right now” is answered by whoever happens to raise their hand in a status meeting, not by looking at the data.
One portfolio view, one set of definitions
Obvyr’s account dashboard gives you the aggregate view: total and active projects, combined test counts across all of them, an overall pass rate, and recent test runs spanning every project in the account. Sitting alongside it, the home page renders a card for every individual project, and each one is computed the same way: the same flakiness rate calculation, the same health profile logic, the same daily execution metrics, regardless of which team owns the project or what framework it uses. Tag-based filtering works the same way across every project too, so a label like “backend” or “release-candidate” means the same thing everywhere it’s applied.
That consistency is the actual point, more than any single dashboard. A “72% pass rate” on one project’s card and an “89% pass rate” on another’s are directly comparable precisely because both numbers came from the same calculation running against the same shape of data, not because either team configured their own version of what “pass rate” means.
What this doesn’t do yet: the account dashboard itself shows an aggregated total across all your projects, not a per-project breakdown. That comes from the individual project cards on the home page instead. Today, comparing projects means scanning those cards yourself, with the reassurance that every one of them uses the exact same definitions, rather than an automatic ranked leaderboard sorted by whichever metric you care about most. A more structured cross-project ranking view is on our roadmap, not shipped today.
Even without an automatic ranking, a leader scanning the estate today gets something they didn’t have before: every project’s numbers, computed consistently, rather than a dozen locally-invented interpretations of what “good” looks like.
What this looks like for a portfolio of ten projects
In practice, that means a VP responsible for ten projects can open the account home page and read ten cards, each showing a pass rate, a flakiness rate, and a health status, and trust that a low number on card four means the same thing as a low number on card seven. Today, spotting the two projects that need attention is still a scan-and-compare exercise rather than a sorted list, but it’s a scan across ten numbers that all mean the same thing, not ten numbers that each need translating first. That’s a meaningfully smaller problem than the one most engineering leaders start with, even before an automated ranking view exists.
That consistency starts further upstream, with tying every execution to the agent, user, and environment that produced it so the underlying data is trustworthy in the first place, and it’s what makes reporting quality to leadership meaningful across an entire portfolio, not just one project.