Most security teams do not lack assessment work. They lack a place to put it. A penetration test lives in a folder of PDFs. A vulnerability scan campaign lives in a scanner console nobody else can read. A retest lives in an email thread. The consulting firm that ran the engagement tracks its own man-days in a spreadsheet, and the client tracks remediation in a third tool. Every assessment produces value, yet the value scatters the moment the engagement ends.
This is the difference between running individual tests and managing a security assessment program. A program needs structure that outlives any single test. It needs a way to define what is in scope, who is doing the work, how many days are budgeted, what methodology is being followed, and how findings move from one wave of testing to the next. When that structure exists, an assessment is no longer a one-time event. It becomes a comparable, repeatable, auditable unit of work.
PMAP treats the project as that unit. A project is the container for a security engagement, and every finding, scan, assessment run, and piece of evidence lives inside one. Around that container, PMAP adds the commercial and operational layers a real program needs: consulting firm directories, framework agreements with man-day accounting, methodology checklists, sequential assessment runs, scan campaigns, and remediation tracking. This article walks through how those pieces fit together, and how they let a team move from ad-hoc tests to managed engagements at scale.
If you are new to the broader process that assessments feed into, the vulnerability management lifecycle pillar covers how a single finding travels from intake to closure. This article focuses one level up, on how the engagements that produce those findings are planned, scoped, and run.
From Ad-Hoc Pentests to Managed Engagements
An ad-hoc pentest is a transaction. A firm is engaged, a test is run, a report is delivered, and the relationship resets. There is nothing inherently wrong with this for a single point-in-time need. The problem appears the second time, and every time after that. Without a persistent record of what was tested, by whom, against which scope, and with what result, each new engagement starts from zero. Comparisons across time are impossible. Coverage gaps are invisible. The history that auditors and executives ask for does not exist in any usable form.
A managed engagement inverts this. The project becomes the long-lived record. In PMAP, a project represents a bounded assessment effort scoped to one customer company and owned by one or more consulting firms. It can model a penetration test, a vulnerability scan campaign, or a recurring security review. Because the project persists, the work inside it accumulates. The second penetration test against the same scope can be compared against the first. A retest can reference the exact findings it is re-validating. A quarterly review can show whether the attack surface grew or shrank since last quarter.
The shift from ad-hoc to managed is not about adding bureaucracy. It is about giving the work a place to live so that the next engagement inherits the context of the last one. The rest of this article describes the building blocks PMAP provides for that, starting with how a project is structured.
Structuring an Assessment Project
A project in PMAP is more than a name and a date. It carries the relationships and constraints that define what the engagement actually is. Each project belongs to one company, the tenant the assessment is being performed for. It optionally links to a primary consulting firm and to a framework agreement that governs the commercial terms. It has a project type and a status, and it tracks both planned and actual effort in man-days.
Project creation enforces real rules rather than accepting any input. A project name must be unique within its company. Two projects in the same tenant cannot share a name, and an attempt to create a duplicate is rejected at write time. This keeps the project list unambiguous when a program runs dozens of concurrent engagements. Creation is also blocked for any project under a deactivated company, so paused or offboarded tenants cannot accumulate new work by accident.
When a project links to a framework agreement, PMAP validates the link instead of trusting it. The agreement’s company must match the project’s company. If the project names a consulting firm, the agreement’s firm must match too. A mismatch is rejected rather than silently stored. This guards against the common error of charging man-days for one client against an agreement signed with another, an error that is expensive to untangle after the fact.
Effort is tracked with two numbers. Planned man-days capture what was budgeted, and actual man-days capture what was spent. Whenever a project that touches a framework agreement is created, updated, or deleted, PMAP recalculates the consumed man-days on that agreement automatically. The contracted capacity pool stays accurate without anyone updating a spreadsheet by hand.
Three-Way Scope: Assets, Groups and Selectors
Scope is where most assessment programs lose precision. A penetration test against “the production environment” means nothing if the environment is not defined in concrete, machine-readable terms. PMAP gives a project three independent ways to define what is in scope, and they can be combined.
The first is individual assets. A tester can pin specific assets to the project scope by adding them one at a time. This is the precise mode for a targeted test against a known set of systems.
The second is asset groups. Instead of pinning hundreds of assets, the project can pin an asset group, and the scope inherits everything in that group. Each pinned group carries its asset count so the team can see how large the scope actually is.
The third is attribute-based selectors. A selector defines scope by rule rather than by enumeration. It matches assets by type, tag, or label, and it reports both how many assets currently match and how many do not. This is the mode for a scope that changes over time. As new assets that match the selector come into inventory, they fall into scope without anyone editing the project.
Each of the three modes is managed independently. A project can use one, two, or all three at once. The result is a scope definition that is both explicit enough to audit and flexible enough to survive a changing environment. For more depth on how the underlying inventory is built and grouped, the attack surface and asset inventory pillar covers asset truth as the foundation of the whole program.
Assessment Runs and Sequential Wave Numbering
A project is a long-lived container, but the work inside it happens in bounded executions. PMAP models each of those as an assessment run. Where the project is the engagement, an assessment run is one wave or milestone within it: a named, time-boxed campaign that can aggregate multiple underlying scans and the findings they produce.
Every run receives an automatically assigned sequential number within its project. The first wave is run one, the next is run two, and so on. This numbering is allocated through an advisory-locked database function that guarantees the sequence is monotonic and gap-free, even when two runs are created at the same moment. The benefit is a stable reference. When a report or a conversation says “the findings from run three,” everyone knows exactly which wave is meant, and the number never shifts.
Runs are also classified by type. A run can be an assessment, a retest, an ad-hoc test, a scheduled run, or a manual pentest. The type makes the intent of each wave explicit. A retest run is structurally distinct from the original assessment it re-validates, which matters when a team wants to see how a fix held up rather than treating every wave the same.
Each run carries a four-state status. It begins as planned, moves to in progress, and ends as either completed or cancelled. The status is user-controlled, so a team advances a run when the work genuinely reaches that stage rather than having the platform guess. A run also has an optional owner, the user responsible for it, resolved to a display name on read so the run list shows who owns each wave at a glance.
Read responses for a run are enriched in a single query. The owner name, the creator name, the project name, and the company name all come back together, along with a live scan count and a live findings count. The findings count is computed by unioning findings attributed through scans with findings attached manually, then deduplicating, so a run shows its true finding total whether the work came from a scanner or a tester’s keyboard.
Scan Campaigns Launched From a Run
A run can be more than a label on existing work. It can launch the work itself. When a run is created in campaign mode, a single API call atomically creates the run and spawns one scan per selected integration. The team chooses which scanners to fire, optionally targets specific assets, and optionally filters by severity, and PMAP creates the run together with its full set of scans in one action.
This is what turns an assessment run into an orchestration point. Instead of opening each scanner console separately, kicking off a Tenable scan here and a Qualys scan there and a DAST run somewhere else, the team launches the whole wave from one place. The run becomes the boundary that holds all of those scans together, and the findings they produce flow back into the same run for unified comparison.
The launch is resilient by design. If a campaign targets several integrations and one of them fails to start, the failure is logged but does not roll back the run. The other scans still launch, and the run row is returned immediately. The team gets the wave they asked for rather than an all-or-nothing failure because a single connector was unavailable. For the deeper mechanics of how PMAP drives many scanners through one pipeline, the multi-vendor scan orchestration pillar covers the orchestration layer in full.
Scans are not locked to the run that created them. An existing scan can be attached to a run after the fact, and a scan can be moved between runs. When a run is deleted, its scans are not deleted with it. The database nulls the link and preserves the scan and finding history. Detaching or reorganizing runs never destroys the underlying data, which is exactly the behavior a program needs when it reshapes how past work is grouped.
Multi-Firm Engagement Management
Real engagements are rarely run by a single party. A large penetration test might use a primary firm for the core testing, a subcontractor for a specialized component, and an independent auditor for quality assurance. PMAP models this directly. A project can engage multiple consulting firms at once, each with its own role, its own man-day allocation, and its own date range.
The role assigned to each firm makes the structure of the engagement explicit. A firm can be primary, secondary, a subcontractor, an auditor, or a QA party. These are not cosmetic labels. They record who is accountable for what across a multi-party engagement, so the project answers the question “who did which part of this test” without anyone reconstructing it from memory.
Each firm engagement also carries its own man-day allocation and its own active date range. This separates the contribution of each party. When a project involves three firms, the project does not blur their work into one effort. It tracks what each firm was allocated and when each firm was active, which is the level of detail that procurement and finance ask for when an engagement spans multiple vendors.
The firms themselves come from a shared directory. PMAP maintains a global catalog of external consulting and penetration-testing vendors that every company in the deployment draws on. A firm that appears across many engagements is a single, consistently maintained record rather than a duplicate created per tenant. The directory stores each firm’s contact details, an active or inactive flag for retiring a vendor without deleting its history, and a record of the firm’s qualifications. We return to those qualifications later, because they double as a compliance input.
Framework Agreements and Man-Day Budgets
Behind a managed engagement sits a commercial reality: someone agreed on how much work the firm would deliver, and someone is tracking how much of that budget remains. PMAP captures this in the framework agreement. A framework agreement defines the pool of man-days, or the project-based engagement scope, that a consulting firm may draw on when delivering security services for a company.
PMAP supports two contract models. A framework agreement is a shared man-day pool that multiple projects consume from over time. A project-based agreement is a one-off engagement scoped to a single piece of work. The two models cover the common procurement patterns: an ongoing retainer with a firm, and a discrete statement of work for one named project.
The accounting is live rather than manual. An agreement tracks its total man-days, its used man-days, and a remaining figure computed as total minus used. The remaining value is never stored. It is calculated at query time, so it cannot drift out of sync with the underlying numbers. For framework-type agreements, used man-days are recomputed by summing the effort of every linked project, preferring actual days where they exist and falling back to estimated days where they do not.
The recalculation fires automatically. Every time a project linked to an agreement is created, updated, or deleted, PMAP recomputes that agreement’s consumed man-days as part of the same operation. A program manager watching contracted capacity sees the true remaining budget without reconciling spreadsheets at the end of the month. When a framework’s pool is nearly exhausted, that fact is visible from the agreement record itself, in time to negotiate more capacity before work stalls.
Checklists and Methodology Coverage
A finding tells you what is wrong. A methodology tells you whether you looked everywhere. The two are different kinds of assurance, and a mature assessment program needs both. PMAP provides methodology coverage through project checklists.
A checklist is a structured, trackable task list that lives inside a project. It drives an assessment through a defined methodology, with each step recorded as an item the analyst can mark complete. The checklist answers a question a findings list cannot: not “what did we find” but “did we test everything we said we would test.”
Checklists come from a template library. PMAP ships system templates and lets teams author custom ones. Templates are typed, with built-in values for OWASP web testing, PCI DSS, internal pentest methodologies, and a custom catch-all. Because the type is stored as a free string, a team can add a new standard without waiting for a schema change. When a tester starts a project, they instantiate a checklist from a template in one action, and the platform copies the template’s items into the project.
The copy matters. Once a checklist is instantiated on a project, it is independent. Later edits to the source template do not propagate into the running engagement. The work in progress is insulated from changes to the master methodology, so a checklist a tester is halfway through does not shift under them.
Progress is calculated for the team rather than maintained by it. Every checklist item carries a done flag, and on every update PMAP computes a progress percentage server-side as the ratio of completed items to total items. The number is stored and returned on every read, so a project manager sees methodology completeness as a percentage without counting checkboxes. Items can also carry a category and free-text notes, so an analyst records observations against a specific test step rather than in a separate document. For the full picture of how methodology coverage maps to the work, see the checklist methodology coverage datasheet.
Remediation Campaigns From Draft to Done
Assessments find problems. Remediation campaigns coordinate the fixing. Where a project organizes the testing, a remediation campaign organizes the response. A campaign is a named, time-bounded grouping of findings assigned to a responsible owner, and it lets a security team track progress toward closure at the program level rather than chasing one finding at a time.
A campaign moves through an enforced lifecycle. It starts as a draft, becomes active, enters review, and finishes as either completed or cancelled. The transitions are validated against a state machine. A campaign cannot jump from draft straight to completed, and an invalid transition is rejected rather than quietly accepted. This keeps the status meaningful, because a campaign in review genuinely passed through being active first.
Findings are linked into a campaign by reference. A team adds an arbitrary set of findings to a campaign, and PMAP enforces that only findings belonging to the campaign’s company are included. The findings remain the same records they always were. The campaign is a coordination layer over them, not a copy. A linked-findings view shows each one enriched with its title, severity, status, asset, and current assignee, so the campaign owner sees the state of the whole remediation effort in one list.
Assignment can happen in bulk. With one action, a campaign owner assigns a single user to every finding in the campaign, or to a chosen subset. Each assignment emits a notification event, so the people picking up the work are told the moment it lands on them. The campaign also computes its own progress metrics: the total number of findings, how many are closed, how many are overdue, and a closure rate. Those numbers turn a remediation effort into something a manager can report on rather than guess at.
The Wave Timeline and Run Deltas
Comparison over time is the payoff of running assessments as a structured program. PMAP delivers it through the wave timeline. The timeline presents a project’s scan waves in chronological order, and for each wave it shows what changed: how many findings are new, how many persisted from the prior wave, how many were resolved, and how many reopened. Severity breakdowns accompany each wave, and assessment runs that have no scans attached are surfaced separately so manual pentest waves are not lost in a scan-centric view.
The delta is computed when it matters most, at completion. When an assessment run transitions to completed, PMAP identifies the immediately preceding run on the same project and computes the difference between them. New findings are those present in the current run but not the previous one. Persisting findings appear in both. Resolved findings were in the previous run and are absent from the current one. Reopened findings are those that show a reopened status-history entry within the current run’s window.
This is the number that answers the question every program leader eventually asks: is the security posture getting better or worse? A run delta turns two waves of testing into a measurable trend. Over many waves, the timeline shows whether remediation is outpacing new discovery or falling behind it. That trend is the difference between reporting activity and reporting progress.
When a run completes, PMAP also emits a completion event carrying the delta, which downstream notification and real-time subscribers can act on. The wave-over-wave comparison is not just a screen a person has to open. It is a signal the platform broadcasts the moment a wave closes.
Qualification and Vendor Performance Tracking
A managed assessment program eventually has to answer for the firms it engages. Two questions recur. Was the firm qualified to do the work? And did the firm actually deliver? PMAP supports both.
Qualification tracking lives on the consulting firm record. Each firm can hold a set of qualification records, and each one captures a certification scheme such as CREST, CHECK, or ISO 27001, along with a level or class, a scope, an issuer, and a validity window. A qualification carries a status that is active, expired, or pending renewal, plus explicit valid-from and valid-until dates. Before a firm is engaged on a project, a program manager can verify that the firm holds the certifications the engagement requires and that those certifications are still in date. The qualification record is also a compliance input, because it provides documented evidence that the chosen vendor was credentialed for the work.
Vendor performance is the other half. Because every project records which firms were engaged, in which roles, with which man-day allocations, the program accumulates a record of what each firm was responsible for over time. Combined with the findings and run history those engagements produced, this gives a program leader the basis to compare vendors on real delivery rather than reputation. The firm directory can be exported to CSV or XLSX for procurement and compliance reporting, so the catalog and its qualifications feed the formal vendor governance process without manual re-keying.
How PMAP Manages Assessments End to End
Pull the pieces together and a single shape emerges. The project is the durable container for an engagement, scoped to a company, owned by one or more firms, governed by a framework agreement, and defined by a three-way scope of assets, groups, and selectors. Inside the project, sequentially numbered assessment runs mark each wave of testing, and a run can launch a multi-scanner campaign in one action or aggregate work attached after the fact.
Methodology coverage rides alongside the findings through project checklists, so a program proves it tested what it intended to test. Remediation campaigns coordinate the response, moving findings through a validated lifecycle toward measurable closure. The wave timeline and run deltas turn consecutive waves into a trend a leader can report. And the consulting firm directory, with its qualification records and man-day accounting, keeps the commercial and compliance side honest.
None of these layers exists in isolation. A finding produced by a scan in a campaign launched from a run inside a project links back, through the project, to the firm that ran it and the agreement that funded it. That connected structure is what lets a team scale from one penetration test to a portfolio of concurrent engagements without the work scattering across tools again. To see the engagement model laid out as a reference, read the assessment projects and engagement datasheet, and to structure your next engagement, start with a project in PMAP.
Frequently Asked Questions
What is the difference between a project and an assessment run in PMAP?
A project is the long-lived container for a security engagement, scoped to one company and owned by one or more consulting firms. An assessment run is a single bounded execution inside that project, such as one wave of testing or one milestone. A project can hold many runs over time. Each run is numbered sequentially within the project, carries its own type and status, and aggregates the scans and findings produced during that wave. The project persists across runs so that consecutive waves can be compared against one another.
How does PMAP launch scans across multiple scanners at once?
When an assessment run is created in campaign mode, a single API call creates the run and spawns one scan per selected integration. The team chooses which scanners to fire, optionally targets specific assets, and optionally filters by severity. PMAP creates the run together with all its scans in one action. If one integration fails to start, the failure is logged and the remaining scans still launch, so a single unavailable connector does not block the whole wave. The findings from every scan flow back into the same run for unified comparison.
How are man-days tracked against a framework agreement?
A framework agreement defines a pool of man-days a consulting firm can draw on for a company. Each agreement tracks total man-days, used man-days, and a remaining figure computed as total minus used at query time. For framework-type agreements, used man-days are recomputed by summing the effort of every linked project, preferring actual days over estimated days. The recalculation runs automatically whenever a linked project is created, updated, or deleted, so the remaining capacity stays accurate without any manual spreadsheet work.
Can a single project involve more than one consulting firm?
Yes. A project can engage multiple consulting firms at once, each with its own role, man-day allocation, and date range. The available roles are primary, secondary, subcontractor, auditor, and QA, which record who is accountable for which part of a multi-party engagement. The firms come from a shared global directory, so a vendor that appears across many engagements is a single maintained record rather than a per-tenant duplicate.
How do checklists help with assessment methodology?
A project checklist is a structured task list that drives an assessment through a defined methodology, with each step recorded as an item the analyst marks complete. Checklists are instantiated from a template library that includes built-in types for OWASP web testing, PCI DSS, and internal pentest methodologies, plus custom templates. When a checklist is created from a template, the items are copied so the running checklist is independent of later template edits. PMAP computes a progress percentage automatically, giving project managers a clear measure of methodology completeness.
What does a run delta show, and when is it calculated?
A run delta shows what changed between one assessment run and the one before it on the same project. It reports new findings, persisting findings, resolved findings, and reopened findings. The delta is computed when a run transitions to completed: PMAP identifies the immediately preceding run by its sequential number and compares the two finding sets. The result is surfaced in the project’s wave timeline and emitted as a completion event, turning consecutive waves of testing into a measurable trend that shows whether the security posture is improving or declining.
How does PMAP track whether a consulting firm is qualified?
Each consulting firm in the directory can hold qualification records, each capturing a certification scheme such as CREST, CHECK, or ISO 27001, along with a level, scope, issuer, and validity window. A qualification carries an active, expired, or pending-renewal status with explicit valid-from and valid-until dates. Before engaging a firm on a project, a program manager can confirm the firm holds the required certifications and that they are still in date. The record also serves as documented compliance evidence that the chosen vendor was credentialed for the work.
