A single open-findings number tells you nothing about direction. Five hundred open findings could be a program that just cleaned up a backlog of two thousand, or one that is quietly drifting upward week after week. The number is the same. The trajectory is opposite. Security leaders who report only absolute counts to their boards are handing executives a photograph when the executives asked for a film.
This is the gap PMAP’s Comparison Analytics closes. Instead of reading one entity in isolation, it places two of them next to each other, computes the delta between them, and renders the difference as the headline. The question stops being “how many findings do we have” and becomes “are we better than last quarter, better than the other subsidiary, better than before the pentest wave.” Those are the questions that actually drive budget and accountability.
This article walks through every comparison mode PMAP supports, explains how each one is computed, and shows where the boundaries sit so you know exactly what the platform measures. Every behavior described here is grounded in how the analytics domain and report comparison endpoint actually work. For the single-entity KPIs that feed these comparisons, read the vulnerability KPI and SLA dashboards article. For the broader program view, the vulnerability risk analytics and reporting pillar ties the whole reporting layer together.
Why Raw Counts Cannot Answer “Are We Improving”
A vulnerability management program produces a continuous stream of numbers. Open findings, critical counts, SLA breaches, average remediation days. Each of these is a fact about a single moment. The problem is that security is not a moment. It is a slope.
Consider three failure modes of count-only reporting. First, the missing baseline. When you tell an executive that you have forty-seven critical findings, you have given them nothing to compare against. Is forty-seven good? It depends entirely on whether last month was thirty or ninety. Without the prior value, the current value is unanchored. Second, the apples-to-oranges trap. A large subsidiary with ten thousand assets will always have more raw findings than a small one with two hundred. Comparing their absolute counts punishes scale and rewards smallness, which is exactly backwards. What matters is closure rate and SLA discipline, both of which normalize out size. Third, the silent drift. A program can hold a steady open-finding count while its critical share creeps upward and its remediation speed degrades. The headline number looks stable while the underlying posture rots.
Comparison Analytics is designed around the premise that the delta is the signal. PMAP’s analytics domain is a read-only intelligence layer that aggregates findings, assets, SLA timers, taxonomy codes, and team activity into the counts, trends, and rankings that power the platform. Comparison sits on top of that layer and does one extra thing: it subtracts. It takes two aggregates and produces the difference between them as a first-class result, complete with directional coloring so a reader knows in one glance whether the movement is good or bad.
That subtraction is what turns data into a story an auditor or a board can follow. The rest of this article is about how PMAP performs it across six distinct comparison contexts.
One Standardised Metric Shape for Any Two Entities
The architectural decision that makes comparison coherent is a single metric shape called SubjectMetrics. Whether the subject is a company or a project, PMAP describes it with the same fields: total, open, and closed findings, a severity split, SLA breach count, closure rate, SLA breach rate, average remediation days, risk score, and retest success rate. Because both sides of every comparison speak this identical vocabulary, the platform can lay them out in matching panels and compute a meaningful delta field by field.
When you run a comparison, PMAP returns a CompareResponse containing the left subject, the right subject, and a ComparisonDelta. The delta is not a vague summary. It is computed as right minus left for every numeric field that matters: total findings, open findings, closure rate, SLA breach rate, risk score, and retest success rate. This is a deliberate, predictable rule. If the right entity has a higher closure rate, the delta is positive. If it has a higher SLA breach rate, the delta is also positive, but as you will see, the platform knows that a positive movement on breach rate is bad news and colors it accordingly.
The standardised shape is what lets the rest of the feature scale. Adding a new comparison context does not require inventing a new metric language. The Company vs Company tab, the Project vs Project tab, and the multi-project comparison all reuse SubjectMetrics, so an analyst who learns to read one view can read all of them. Consistency here is not cosmetic. It is what keeps a benchmarking workspace from becoming a pile of incompatible reports.
Company vs Company and Project vs Project
The most direct comparison is two entities of the same type placed side-by-side. In the Company vs Company and Project vs Project tabs, you pick a left entity and a right entity from two dropdowns, then trigger the comparison. PMAP renders two subject cards, each showing eight KPI tiles, a delta table with four columns of metric, left value, right value, and delta badge, and a grouped bar chart that draws two bars per category across Total, Open, Urgent, Critical, and High.
This is the view a security manager opens to benchmark subsidiaries. Two business units sit next to each other on identical metrics. One has a closure rate of eighty-one percent, the other sixty-three. One closed within SLA on most criticals, the other breached repeatedly. The delta table makes the gap explicit instead of leaving it for the reader to compute. Because closure rate and SLA breach rate are ratios rather than raw counts, the comparison stays fair even when the two companies differ wildly in size.
Project vs Project works identically, swapping companies for projects. A pentest lead can place two engagements next to each other and ask which one is remediating faster. Both left_id and right_id are required; if either is missing, the server returns HTTP 400 rather than guessing. The comparison type defaults to company and can be switched to project explicitly.
When the comparison involves a whole portfolio rather than a pair, PMAP supports multi-project comparison. The endpoint accepts repeated project parameters and returns the same SubjectMetrics shape for each project supplied, so a manager can rank an entire batch of engagements on one screen. There is no enforced upper bound on how many projects you submit, which means an analyst can compare a large portfolio in one pass, with the practical caveat that query time grows in proportion to the number of projects.
Inverted-Delta Metrics Where Lower Is Better
Not every positive delta is good news, and PMAP refuses to pretend otherwise. Some metrics improve when they go down. SLA breach rate and open findings are the clear examples. A higher breach rate is worse, not better, so a positive delta on that metric represents a regression.
The platform encodes this with inverted-delta coloring. For standard metrics like closure rate, a positive delta renders green because more closure is better. For inverted metrics like SLA breach rate and open findings, a positive delta renders red because the right entity is performing worse. The badge color is therefore a direct, honest signal of direction. A reader does not have to remember which metrics are “good high” and which are “good low.” The interface carries that knowledge so the human does not have to.
This small rule has an outsized effect on how quickly a comparison can be read. A row of green and red badges communicates the verdict before anyone parses a single number. The numbers are there for the detail, but the color tells the story. This is the kind of design that lets a benchmarking view land in an executive meeting without a five-minute explanation of how to interpret it. The same KPI definitions that drive these badges are explained in depth in the KPI and SLA dashboards article.
Vendor Performance Scorecards for Procurement
Multi-vendor security programs run on the work of consulting firms, and those firms are not interchangeable. Some close findings quickly and pass retests on the first attempt. Some leave a long tail of open items and miss SLA windows. Until you measure them on the same scale, you are renewing contracts on reputation rather than evidence.
PMAP’s Vendor Performance tab produces a per-firm scorecard. For each consulting firm in scope it reports projects completed, total and closed findings, average closure time in days, average critical count, retest pass rate, and SLA breach rate. The tab requires no selection. It loads every in-scope firm as soon as you open it, drawing a bar chart of the top firms by projects completed and closure rate, with a full table of all firms beneath it. Firm names are truncated in the chart for legibility, while the table carries the complete roster.
This is procurement intelligence, not vanity metrics. A platform administrator preparing for a contract review can see at a glance which firm delivers the fastest average closure time, which one passes retests most reliably, and which one is quietly accumulating SLA breaches. Those four dimensions, closure time, retest pass rate, critical density, and breach rate, are precisely the levers a procurement decision should turn on. The scorecard turns a subjective sense of “firm A feels slower” into a defensible row of numbers you can put in front of a vendor at renewal.
Because the scorecard is built from the same scoped analytics aggregates as the rest of the platform, it only ever shows firms whose work falls inside the viewer’s scope. There is no pagination on the result; the top performers populate the chart while the table holds every firm in scope.
Year-Over-Year Posture Trends
Quarterly and annual reviews live or die on the long view. A board does not want to know what happened last week. It wants to know whether this year is better than last year. That requires a time series that spans calendar boundaries, and most count-based reporting simply cannot draw one.
The Year-over-Year tab renders a multi-year series of created, closed, open, critical, and high findings. A granularity toggle switches the series between monthly and quarterly resolution. The chart draws created, closed, and critical lines so a reviewer can see the relationship between intake and remediation over time. When the data spans at least two calendar years, PMAP adds a per-year summary bar chart beneath the line chart, giving a clean year-on-year rollup alongside the granular trend.
The granularity parameter is forgiving. It accepts monthly as the default or quarterly, and any other value is silently coerced to monthly rather than rejected. This means a malformed deep-link never breaks the view. It degrades gracefully to the default. The trade-off is that the coercion is silent, so a caller who passes an unexpected granularity gets a valid monthly series back without a warning. For an interactive dashboard this is the right behavior; the view always renders something useful.
Year-over-year is where a posture improvement becomes presentable. A security leader can stand in front of a board and show that critical findings dropped quarter over quarter across two years, that closures consistently outpaced new intake, and that the open backlog trended down. That is a narrative executives can act on, and it is exactly the kind of evidence that risk quantification frameworks like the FAIR model and measurement standards such as NIST SP 800-55 ask security programs to produce: posture expressed as a measured trend, not a snapshot.
Diffing Two Scanner or Assessment Runs
After a pentest wave or a fresh scan, the operational question is sharp and immediate. What is new, what got fixed, and what is still here. Reading two long finding lists by hand to answer that is slow and error-prone. PMAP’s Scan Comparison tab does the diff for you.
You select a baseline run and a target run from two dropdowns. PMAP fetches the findings for each side and computes the difference. The diff is keyed on each finding’s fingerprint, falling back to the finding identifier when no fingerprint is present. The result is three expandable cards: new findings, resolved findings, and persistent findings. A finding present in the target but absent in the baseline is new. One present in the baseline but gone from the target is resolved. One present in both is persistent. The cards carry distinct visual markers so the three categories never blur together.
There is a deliberate bound here worth understanding. Each side fetches up to five hundred findings, and the fingerprint match runs client-side. Findings beyond five hundred on a side are not represented in the diff. This keeps the comparison fast and responsive for the typical scan or engagement, and it is a known, documented limit rather than a silent truncation surprise. There is also a precise edge in the matching logic: a finding with no fingerprint falls back to its identifier, which means findings lacking fingerprints can never match across sides. Each one shows up as new or resolved, never as persistent. That is correct behavior; without a stable fingerprint there is no reliable way to assert that two findings are the same issue.
The same tab supports an assessment-run mode reached through a deep-link. When two run identifiers are supplied in the URL, PMAP fetches findings by assessment run and hides the selector entirely, loading the comparison straight from the link. This makes a run-to-run diff shareable as a single URL, which is useful when a pentest lead wants to hand a colleague the exact comparison they are looking at. The assessment-run metadata such as name and date is fetched separately, and if that record is not accessible the comparison still runs on the findings alone.
Report Delta as Compliance Evidence
The most audit-relevant comparison is between two completed reports. Compliance reviews routinely ask the same question: between this audit period and the last one, what changed. PMAP’s Report Delta tab answers it at the finding level and produces an evidence-grade record.
You select two completed reports from dropdowns that are pre-filtered to completed status only. The comparison is posted to a dedicated report comparison endpoint, and PMAP categorizes every finding into new, resolved, or persisting. The matching key is a composite of the lowercased title and lowercased asset name, which makes the match resilient to casing differences between report snapshots. A finding in the head report but not the base is new. One in the base but not the head is resolved. One in both is persisting.
The persisting set carries extra intelligence. Within findings that appear in both reports, PMAP detects severity changes and tallies them as escalated or improved counts derived from a numeric severity weight map. So the delta does not just say “this issue is still open.” It says “this issue is still open and its severity climbed from high to critical,” and it surfaces that with an old-to-new severity arrow on the affected row. The result view presents three summary stat cards for new, resolved, and persisting; clicking a card switches the active finding list so a reviewer can drill into any category.
This is exactly the artifact an auditor wants. It is a finding-level, period-over-period delta that documents what was remediated, what newly appeared, and what persisted with its severity trajectory. It maps cleanly to the measurement and analysis expectations in standards like ISO/IEC 27004, which call for monitoring security performance over time with evidence. The report delta is the comparison angle on report data. The production side of reporting, how those reports are generated and formatted in the first place, is covered in the async security report generation article.
Scope Isolation in Every Picker and Result
A benchmarking workspace is only safe if it can never show a viewer data from a tenant they should not see. PMAP enforces this at the source. Every analytics aggregate applies a scope filter drawn from the authenticated context, so cross-tenant data never surfaces in any comparison result.
The isolation extends to the pickers themselves, which is the subtle and important part. The company dropdown returns only companies within the viewer’s scope. The project dropdown returns only in-scope projects. The report picker is filtered to completed reports that are in scope. This means an analyst scoped to one company cannot even select another tenant’s entity to compare against, because the other tenant never appears in the dropdown in the first place. The boundary is enforced before the comparison runs, not patched afterward.
This matters most in exactly the scenario Comparison Analytics is built for: a managed security provider or a large enterprise running many subsidiaries on one platform. The whole point of the feature is to compare entities, and the whole risk is that a comparison could leak across a tenant line. PMAP closes that risk by making scope a property of every query and every picker. There is no comparison mode that bypasses it. The multi-tenant enforcement model is part of the broader configurable dashboards and analytics architecture, where the same scope filter governs every widget and every metric.
Six Comparison Modes in One Workspace
Everything above lives in a single screen. The Compare workspace presents six tabs in a pill-style switcher: Company vs Company, Project vs Project, Vendor Performance, Year-over-Year, Scan Comparison, and Report Delta. An analyst moves between benchmarking contexts without leaving the page or learning six different tools.
The active tab can be set from a deep-link query parameter, so a specific comparison mode is shareable as a URL. Open the workspace with the scan comparison type and the right tab is already active. This is the same pattern that makes the assessment-run diff shareable: the URL carries the intent. Comparison results are cached briefly, holding for sixty seconds, which keeps repeated views snappy. The trade-off is that data changes within that window require a manual refresh to appear, a reasonable exchange for a responsive workspace that is not re-querying the database on every interaction.
Consolidating six modes into one workspace is what turns comparison from an occasional report-building exercise into a daily-use surface. A manager benchmarking subsidiaries, a procurement lead scoring vendors, an executive preparing a year-over-year board slide, and a pentest lead diffing a fresh wave all work from the same screen with the same scope guarantees. They are not exporting data into spreadsheets and reconciling it by hand. They are reading the gap directly.
How PMAP Measures the Gap, Not Just the Count
The throughline of Comparison Analytics is a single idea. A count describes a state; a delta describes a direction; and direction is what a security program is actually managed against. PMAP builds the delta into the result rather than leaving it for a human to compute, and it does so consistently across companies, projects, vendors, calendar years, scan runs, and report snapshots.
Because it sits on the analytics read layer, the feature inherits that layer’s discipline. It writes nothing, it scopes everything, and it speaks one standardised metric vocabulary so any two comparable entities can be laid side by side. The inverted-delta coloring keeps the verdict honest, the report delta produces audit-grade evidence, and the scoped pickers make tenant leakage structurally impossible.
The result is a benchmarking workspace that answers the questions count-only reporting cannot. Are we better than last quarter. Is this subsidiary outperforming that one. Did the pentest wave actually reduce our exposure. Which vendor earns the renewal. Those answers are the difference between a dashboard that decorates a meeting and one that decides a budget.
To put these comparisons in front of your own stakeholders, read the risk analytics datasheet and benchmark your subsidiaries, projects and scans in one workspace.
Frequently Asked Questions
What can PMAP’s Comparison Analytics actually compare?
Six modes in one workspace: two companies side-by-side, two projects, a portfolio of projects at once, consulting-firm vendor scorecards, year-over-year posture trends, a diff between two scanner or assessment runs, and a finding-level delta between two completed reports. Companies and projects are compared on a single standardised metric shape, and every mode applies scope isolation so only in-scope entities appear.
How is the delta between two entities calculated?
For company and project comparisons, PMAP computes the delta as right minus left for every numeric field: total findings, open findings, closure rate, SLA breach rate, risk score, and retest success rate. The delta badge is colored by direction. Standard metrics like closure rate show green for a positive delta, while inverted metrics like SLA breach rate and open findings show red for a positive delta because higher is worse.
Does scan comparison have a finding limit?
Yes. Each side of a scan or assessment-run comparison fetches up to five hundred findings, and the diff is computed client-side by fingerprint, falling back to the finding identifier when no fingerprint exists. Findings beyond five hundred on a side are not included in the diff. A finding with no fingerprint can only appear as new or resolved, never as persistent, because there is no stable key to match it across sides.
How does report delta classify findings?
PMAP matches findings between two completed reports using a composite key of the lowercased title and lowercased asset name. A finding in the head report but not the base is new, one in the base but not the head is resolved, and one in both is persisting. Within the persisting set, severity changes are tallied as escalated or improved counts and shown with an old-to-new severity arrow on the affected row.
Can year-over-year comparison span multiple calendar years?
Yes. The Year-over-Year tab renders a monthly or quarterly series of created, closed, open, critical, and high findings across multiple calendar years. When the data spans at least two calendar years, PMAP adds a per-year summary bar chart beneath the line chart. The granularity parameter accepts monthly or quarterly, and any other value is silently coerced to monthly so the view always renders.
Can a viewer accidentally compare data from another tenant?
No. Every analytics aggregate applies a scope filter from the authenticated context, and the pickers themselves return only in-scope entities. The company, project, and report dropdowns never list entities outside the viewer’s scope, so a cross-tenant comparison cannot even be selected, let alone run. Scope is enforced before the comparison executes, not corrected afterward.
Is Comparison Analytics a separate report to generate?
No. All six modes live in one interactive workspace, and the active mode can be set from a deep-link query parameter so a specific comparison is shareable as a URL. Results are cached for sixty seconds for responsiveness, which means data changes within that window need a manual refresh. The report delta consumes already-generated reports rather than producing them; report generation and formatting are handled separately.
