Most teams treat the audit trail as a single table. Something happens, a row gets written, an auditor reads it later. That model works until you ask it to do two jobs at once. A compliance reviewer wants a clean, ordered ledger of high-consequence security actions. A platform operator wants a complete record of every entity change across a busy system that never slows down the request path. Force both demands onto one writer and one of them loses. The compliance ledger picks up noise and ordering gaps, or the activity record starts dropping rows under load, or the whole thing adds latency to the operations it is supposed to observe.
A dual audit trail design solves this by accepting that these are different problems with different priorities. One trail is built for correctness and ordering. The other is built for throughput. This article walks through how that split works in practice, using PMAP’s two-trail implementation as a concrete reference. You will see why security events are written synchronously while activity logs are batched, how a typed event vocabulary keeps the compliance trail honest, and why neither trail loses a row even when the system is shutting down or running hot.
If you want the wider context of how audit fits into a vulnerability management compliance program, the pillar guide on vulnerability management compliance and audit covers the full picture. This piece zooms into one design decision: why two trails beat one.
One Audit Log Is Not Enough
The instinct to keep audit simple is reasonable. Fewer tables, one write path, one query surface. The problem is that audit serves two readers whose needs pull in opposite directions.
The first reader is the auditor. During a SOC 2 examination or an ISO/IEC 27001 review, someone needs to confirm that access was granted to the right people, that privileged roles were created and removed under control, and that destructive actions were attributable to a named actor at a known time. This reader cares about a small number of high-consequence events and cares intensely about their integrity. A missing access-revoke event is a finding. An out-of-order sequence of grant and revoke is a question that takes hours to answer.
The second reader is the operator. When a finding’s status changes unexpectedly, when an asset disappears from inventory, or when a bulk import rewrites a thousand records, someone needs to reconstruct what happened. This reader cares about volume and breadth. Every entity mutation across the platform should be recorded, and the recording must never become the reason a request is slow.
These two readers do not want the same table. The security reader wants a short, strictly ordered, typed ledger. The operations reader wants a wide, high-throughput stream. PMAP’s audit domain provides exactly two complementary trails for this reason. The security trail lives in access_audit_events and is owned by audit.Service. The activity trail lives in activity_logs and is owned by admin.AuditQueue. Each is tuned for its own reader, and the design treats them as separate concerns rather than one log wearing two hats.
The rest of this article takes each trail in turn, then compares them side by side.
Trail One, the Security Event Ledger
The security audit trail is the compliance-grade record. It captures high-consequence security actions and writes them synchronously to PostgreSQL on the request path. Synchronous means the row is committed before the handler returns. There is no background queue between the event and its durable storage, and that is a deliberate choice.
The events this trail captures are narrow by design. They include login success and failure, access grant, access revoke, access expiry, role creation, update and deletion, sensitive views, and destructive deletes of findings, projects and assets. These are the actions an auditor asks about first. They are also the actions where ordering and attribution matter most, because the difference between “access granted then revoked” and “access revoked then granted” is the difference between a controlled environment and an open one.
This trail maps cleanly onto the audit and accountability expectations described in the NIST SP 800-53 AU control family, which calls for a defined set of auditable events, attribution to individual actors, and protection of audit content. By keeping the security trail focused on a curated event set, PMAP keeps the ledger readable for exactly the review that frameworks like SOC 2 and ISO/IEC 27001 require.
Typed Event Kinds
The security trail does not accept free-form event names. It uses a fixed vocabulary of twelve named event-type constants: login_success, login_failed, access_granted, access_revoked, access_expired, role_created, role_updated, role_deleted, sensitive_view, finding_deleted, project_deleted, and asset_deleted. Each constant is defined in code, and the database enforces the same set through a CHECK constraint on the event_type column.
This double enforcement matters more than it first appears. A typo in an event name is caught at compile time because the constant has to exist in the code. If a value somehow reaches the database that the CHECK constraint does not recognize, the write fails at the database layer rather than silently storing a malformed event type. Adding a new event kind is therefore an intentional act. It requires both a new constant in the service and a paired migration that extends the CHECK constraint. The vocabulary cannot drift by accident, which is precisely the property you want in a compliance ledger that an auditor will read literally.
A controlled vocabulary also makes the trail queryable. Because every grant uses the same access_granted constant, an auditor can filter the entire history of access changes with a single exact-match query rather than guessing at every variant a developer might have typed. The security audit endpoint supports filters for actor, target, event type and time range for exactly this reason.
Why Ordering Must Be Strict
The security trail is written synchronously because order is part of the evidence. If access events were handed to a background goroutine for later insertion, two events emitted close together could land in the database in the wrong sequence depending on scheduling. For most logs that is harmless. For a compliance ledger it is corrosive, because the narrative of who had access when depends on the rows arriving in the order the actions occurred.
By inserting directly on the request path, the security trail guarantees that the database write order matches the action order. There is no goroutine scheduling reordering between the event and its row. The cost is that the write happens inline, but the security event set is small and low volume, so this cost is acceptable. Correctness wins here because the events are rare and consequential. That trade-off flips for the second trail.
Trail Two, the Activity Log
The activity trail is the operations record. Where the security trail is narrow and synchronous, the activity trail is wide and asynchronous. It captures every entity-level action across the platform: finding create, update and delete, bulk import, asset mutation, scan events, SLA breaches and escalations, report generation, and team changes. This is high-volume traffic. A single bulk import can generate hundreds of activity rows, and the system processes many such operations concurrently.
Writing each of these synchronously on the request path would tax the database connection pool and add latency to ordinary work. So the activity trail inverts the priority of the security trail. It is fire-and-forget on the hot path. A LogActivity call enqueues an event and returns without waiting for the database. A background worker does the actual writing.
The activity trail is what the operations reader uses to reconstruct an incident. It records user_id for the actor, an action string such as finding.created or bulk_import_findings, the entity_type and entity_id of the target, a JSON details payload, and the client IP. The read surface for this trail is the admin activity log endpoint, separate from the security event endpoint, which keeps the operations stream from cluttering the compliance ledger.
Batched, Fire-and-Forget Writes
The activity trail’s throughput comes from batching. The AuditQueue runs a single background worker that drains an in-memory channel and writes events in batched round-trips using pgx.Batch. Instead of one database round-trip per event, the worker accumulates events and flushes them together. The default behavior flushes when the batch reaches two hundred events or when a two-second ticker fires, whichever comes first.
This design keeps database connection pressure low even at high request rates. A burst of activity becomes a handful of batched inserts rather than a flood of individual ones. The two-second flush ceiling means events do not sit in the buffer indefinitely during quiet periods, so the activity log stays close to real time without paying per-event cost during busy ones. The buffer itself is sized generously by default to absorb bursts before any fallback behavior is needed.
The Synchronous Fallback
Fire-and-forget raises an obvious worry. What happens if the buffer fills faster than the worker can drain it? A naive fire-and-forget design would simply drop the event, and a dropped audit row is a silent gap in the record.
PMAP’s activity trail refuses that outcome. When the channel buffer is full at enqueue time, LogActivity does not discard the event. It falls back to a synchronous single-row insert on the caller’s context, writing the row immediately rather than queuing it. The system also logs a warning so operators can see that the buffer is under pressure and tune the queue parameters if the condition persists.
The effect is a graceful degradation. Under normal load the activity trail is fast and batched. Under extreme load it gets slower for the affected calls but never loses a row. This is the design principle that distinguishes a real audit log from a best-effort metrics stream. Completeness is non-negotiable even when performance has to give.
Comparing the Two Trails Side by Side
It helps to see the two trails against each other, because the contrast is the point. They are not redundant copies. They make opposite trade-offs on purpose.
| Property | Security audit | Activity audit | |—|—|—| | Table | access_audit_events | activity_logs | | Writer | audit.Service (Log / LogFromRequest) | admin.AuditQueue (LogActivity) | | Volume | Low, security events only | High, all entity CRUD | | Write path | Synchronous, on the request path | Batched background worker | | Ordering | Strict, matches action order | Best-effort within the flush window | | Vocabulary | Twelve typed constants, DB CHECK constraint | Free-form action strings | | Optimised for | Correctness and attribution | Throughput and breadth |
Read the table as a set of intentional choices rather than a feature list. The security trail accepts inline write cost to buy strict ordering, because its events are rare and consequential. The activity trail accepts best-effort ordering within a flush window to buy throughput, because its events are frequent and its job is reconstruction rather than legal sequence. Neither trail is a compromise of the other. Each is the right answer to a different question.
A compliance reviewer reads the security trail and asks: who was granted what, when, and by whom. An operator reads the activity trail and asks: what changed across the system around the time this went wrong. Two questions, two trails, no contention between them.
Completeness Over Latency
The governing principle behind both trails is the same, even though they reach it from opposite directions: audit completeness ranks above request latency. The audit system must never become a second failure mode for the operations it records.
This shows up in how write errors are handled. When the security service writes a security event, an error from that write is logged but never propagated back to the calling handler. A database hiccup while recording an access change cannot break the access change itself. The audit write fails quietly into the operator’s logs rather than failing the user-facing request. The same philosophy drives the activity trail’s synchronous fallback. Both trails would rather slow down or log an internal error than drop a row or break a request.
This is a subtle but important stance. A heavy-handed audit layer that blocks or fails requests when its database is slow effectively punishes the system for trying to keep records. PMAP’s audit domain is deliberately minimal for this reason. It is described as pure cross-cutting infrastructure injected into every domain that performs security-relevant mutations, and it is designed not to add a second point of failure to those operations. Completeness is achieved through resilient write paths, not through blocking the application.
Graceful Drain on Shutdown
The most common moment to lose audit rows is shutdown. A process receives a termination signal, stops accepting work, and exits, leaving anything still buffered unwritten. For a batched audit queue this is a real risk, because events accepted moments before shutdown may still be sitting in the channel.
PMAP closes this gap with an ordered shutdown sequence. The activity queue’s Stop method is called after the HTTP listener has closed and all in-flight requests have completed. By that point no new LogActivity calls can arrive, and every call made during the life of the now-finished requests has been enqueued. Stop then drains the channel and flushes the remaining events before the process exits. The result is that no audit row is lost on a SIGTERM. The ordering of the shutdown steps is the safeguard: stop accepting work first, then drain what was accepted.
Together, the synchronous fallback and the graceful drain close the two windows where a fire-and-forget design would normally leak. Buffer pressure during operation is handled by the fallback. Buffered events at shutdown are handled by the drain. Between them, the activity trail keeps the completeness guarantee that its asynchronous nature might otherwise threaten.
Evidence That Survives a Delete
A deletion is the hardest event for an audit trail to record well, because the thing the audit row points at no longer exists. If a finding is deleted and the audit row only stores the finding’s ID, then later, when an auditor reads the trail, that ID resolves to nothing. The row says a finding was deleted but cannot say which one in human terms.
The security trail handles this by capturing identifying detail at the moment of emission. For finding_deleted, project_deleted and asset_deleted events, the joined entity row will be gone by the time anyone reads the trail. So the event’s payload captures the entity’s name at emit time. When the audit screen renders the event and finds no live entity to join against, it falls back to the name stored in the payload. The deletion remains legible long after the entity is gone.
This is what tamper-evident, audit-ready evidence looks like in practice. The record of a destructive action does not degrade just because the action succeeded. An auditor reviewing deletions months later sees named entities, attributed actors, and exact timestamps rather than a list of orphaned identifiers. That property maps directly to the kind of accountability that frameworks expect from an audit record, where a logged event must remain meaningful and attributable over time.
It also reinforces the case for a typed, controlled security trail. Because deletes are first-class typed events with payload capture, they cannot be lost in a sea of generic activity rows. The high-consequence action gets the high-consequence treatment.
How Auditors Read Both Trails
A trail is only as useful as it is readable. Raw audit data is full of UUIDs: actor IDs, target IDs, role IDs, scope IDs. An auditor confronted with a screen of opaque identifiers cannot do the job, and forcing them to cross-reference IDs by hand turns a review into an ordeal.
The security trail resolves identifiers into names at query time. Its list query joins against the relevant tables to produce actor name and email, target user name, role name, scope name, and entity name, all resolved server-side. The audit screen never renders a raw UUID where a human-readable name exists. Scope names are resolved through a correlated subquery that branches on the scope type, so a company-scoped event shows the company name and a project-scoped event shows the project name. For deleted entities, the payload fallback described above fills the gap.
This name resolution is what makes the dual-trail design usable rather than merely correct. The compliance reviewer reads the security trail as a narrative of named people taking named actions on named things at known times. The operator reads the activity trail as a stream of entity changes attributed to actors. Both surfaces are gated for the right audience. The security event log is restricted to platform administrators, while the activity log is admin-gated through the admin router. The separation of read surfaces mirrors the separation of trails. Each reader gets the trail built for their question.
Many of the events that feed the security trail originate in the access-control system. Grants, revocations, expiries and role changes are exactly the actions a multi-tenant deployment needs to keep attributable, which is why the trail is a natural companion to the access model described in the multi-tenant vulnerability management guide. And because four-eyes approval produces a separate, parallel record of who authorised sensitive changes, it complements rather than duplicates the audit trail. The companion piece on four-eyes approval in vulnerability workflows covers that control in depth.
To see the full data model behind both trails, including the event vocabulary, payload structure and read endpoints, the audit trail and compliance evidence datasheet lays it out alongside the controls it supports. It maps the dual-trail design to the evidence expectations of SOC 2, ISO/IEC 27001 and the NIST audit and accountability family.
Frequently Asked Questions
What is the difference between a security event and an activity log?
A security event is a high-consequence, low-volume record written synchronously on the request path into the access_audit_events table. It covers actions like access grants, role changes and destructive deletes, and its ordering is strict. An activity log is a high-volume, best-effort record written by a batched background worker into the activity_logs table. It covers every entity-level change across the platform, such as finding edits, bulk imports and asset mutations. The first trail is optimized for correctness and attribution. The second is optimized for throughput and breadth. They are complementary trails serving two different readers, not redundant copies of one log.
Why are security events written synchronously?
Security events are written synchronously because their ordering is part of the evidence. If access events were handed to a background queue, two events emitted close together could be inserted in the wrong sequence depending on goroutine scheduling. For a compliance ledger, the order of grant and revoke is meaningful, so the trail writes directly on the request path to guarantee that the database write order matches the action order. The event set is small and low volume, so the inline write cost is acceptable in exchange for strict ordering.
Can an audit row be lost under load or on shutdown?
No. The activity trail protects against both windows where a fire-and-forget design might leak. Under load, if the queue buffer is full, the activity logger falls back to a synchronous single-row insert rather than dropping the event, and it warns operators so they can tune the queue. At shutdown, the queue’s drain method runs after the HTTP listener has closed and all in-flight requests have completed, flushing every buffered event before the process exits. The synchronous fallback and the graceful drain together keep the completeness guarantee intact.
How is a deleted entity still shown in the audit log?
For delete events such as finding_deleted, project_deleted and asset_deleted, the entity will be gone by the time anyone reads the trail, so a join against the live table returns nothing. To keep the record legible, the event’s payload captures the entity’s name at the moment of emission. When the audit screen cannot join against a live entity, it falls back to the name stored in the payload. The deletion stays meaningful long after the entity itself is removed.
What are typed event kinds and why do they matter?
Typed event kinds are a fixed vocabulary of twelve named event-type constants used by the security trail, such as login_success, access_granted, access_revoked and finding_deleted. They matter because they are enforced in two places. The code requires the constant to exist, which catches typos at compile time, and the database enforces the same set through a CHECK constraint, which rejects any unrecognized value at the storage layer. Adding a new event kind therefore requires both a code change and a paired migration. This prevents the compliance vocabulary from drifting by accident and keeps the trail cleanly queryable by exact event type.
Why does the audit system never break the request it records?
Because audit must not become a second failure mode for the operations it observes. When the security service write fails, the error is logged but never returned to the calling handler, so a database hiccup while recording an access change cannot break the access change itself. The activity trail follows the same principle through its synchronous fallback. The audit domain is deliberately minimal and treated as cross-cutting infrastructure, designed to record events resiliently rather than to block or fail the application when its own writes struggle.
How does the dual-trail design support compliance frameworks?
The split maps directly onto what audit-focused frameworks expect. The narrow, typed, strictly ordered security trail provides the curated, attributable event record that SOC 2 examinations and ISO/IEC 27001 reviews ask for, aligning with the auditable-event and attribution expectations in the NIST SP 800-53 AU control family. The broad activity trail provides the operational completeness needed to reconstruct what changed and when. Name resolution keeps both trails human-readable, payload capture keeps deletions legible, and the completeness-over-latency stance keeps the record whole. The result is audit-ready evidence that a reviewer can read as a narrative rather than a pile of identifiers.
