A 15-Question VulnerabilityOps Audit for Client Engagements

You know the moment in a discovery call when the client says, “We scan everything, so we’re fine”? It is the cue to pull out a checklist. Over the past year the question I find most revealing is not “what do you scan with?” but “what happens to a finding after the scan?” That is the territory of VulnerabilityOps, and this post is a friendly, usable audit you can run in a half-day workshop with any engineering or security lead. Print it, tick it, and you will know where the real gaps are.

A quick bit of context so you can explain it to a sceptical sponsor. The Ops Atlas, which collects names for operating disciplines, describes VulnerabilityOps as a flow-based way of managing software weaknesses and notes that no standards body owns the term; the underlying practice goes back to long-standing NIST and CIS guidance. In other words, you are not selling a fad, you are tightening something the client should already be doing.

Part A: Know what you are looking at

1. Is there an inventory, and does it cover everything that is reachable? Ask who adds a new system to the list and how long that takes. Gaps in inventory are gaps in scanning.
2. Is every asset tagged for exposure? Public, internal, or unknown. The Atlas offers a sensible local rule: an untagged asset is treated as public until someone says otherwise, which forces the tagging conversation.
3. What share of hosts are scanned with credentials, and how recently? Ask for the coverage figure over the last seven days. The Atlas example shows 1,820 of 1,900 hosts at 95.8 percent, with the missing 80 explained by credentials lost during a key rotation. If the client cannot tell you the number, that is finding number one.
4. Who guards the scanner accounts? They hold wide access. Treat them like privileged secrets, with rotation and monitoring.

A practical tip for Part A: bring a recent change record and trace one new server from request to first scan. Count the days. If the answer is “it depends”, ask what it depended on. Most of the time you will uncover a manual step that nobody owns, and that step is where blind spots breed.

Part B: Prioritise on evidence

5. Is “fix every nine and above” the entire policy? If so, say gently that severity labels do not prescribe actions. The Atlas names this as one of the classic anti-patterns.
6. Is a list of known exploited flaws checked every day? The US agency’s catalogue had 1,748 entries in mid-October 2026, and daily matching against it is an easy early win.
7. Is anyone multiplying EPSS by CVSS? Please ask. The standards body behind the probability model calls that “Score Laundering”, and it hides which input did the work.
8. Are the tiers connected to dated clocks? A tier with no deadline is a label, not a policy. A good starting shape is a few days for exposed and exploited items, a couple of months for routine items, and “fix on upgrade” for low-exposure ones.

Part C: Move the work and prove the result

9. Does each finding land in a queue with a named owner and a date? Ask to see a real example from last week, not a diagram.
10. What counts as done? The answer you want is evidence: a rescan run with credentials, a settings check, a rebuilt image carrying a new digest, or a dispute that was formally resolved. The answer to worry about is “the ticket was closed”. The Atlas also stresses a lesson from a 2023 remote-access appliance flaw: the definition of fixed must include ending sessions that were already hijacked, not only installing the update.
11. Do clocks reset when a rescan runs? They should not. Findings should be identified by asset, weakness, and component so one issue stays one issue.
12. Who may demote a priority? The Atlas pattern is that analysts can raise a tier but never lower one on their own.

When you reach question ten, expect some defensiveness. Teams have usually closed tickets in good faith, and being told that status proves nothing can feel like an accusation. Frame it as a measurement problem instead: a status field records intent, whereas a rescan records the world. Offer to sample five closed items from last month and check them together. The result is usually enough to turn the room.

Another useful probe is recurrence. If an item that was closed shows up again within ninety days, the fix was either incomplete or got overwritten by a rebuild from an old image. Asking for that rate often reveals more about release habits than any pipeline diagram.

Part D: Read the numbers honestly

13. Do they track arrivals divided by verified closures per tier? A value above one means the queue is growing. In the Atlas worked example a routine tier sat at 1.15, which translated into roughly two months of backlog against a sixty-day promise.
14. Do they report age without saturation? If so, they may be congratulating themselves for a number that fell only because new arrivals diluted the old ones.
15. Is there a dispute rate and an exception ledger? Unfixed-by-design items should carry an owner and an expiry, and the total should be visible. Otherwise exceptions become a silent second backlog.

How to run the workshop

Keep it to ninety minutes with three people in the room: someone who runs scanning, someone who fixes things, and someone who can approve exceptions. Walk the fifteen questions in order and mark each green, amber, or red. Resist the urge to solve problems during the session; the job is to find them. If the client wants more on how consulting work frames this kind of exercise, our overview of where consulting fits into managing delivery risk sets out the usual structure.

Afterwards, group the reds into three buckets: visibility (inventory, coverage, credentials), decision rules (tiers, clocks, signals), and proof (closure, recurrence). Most engagements find the heaviest cluster in proof. Teams are often good at finding things and surprisingly casual about showing they were fixed.

A starter plan you can offer on the spot

For the first thirty days the Atlas leans on housekeeping: close inventory gaps, add exposure tags, begin the daily exploited-flaw match, and capture a baseline. The next thirty days introduce a first policy, a hand-off record between finder and fixer, closure by rescan, and a way to request exceptions. By day ninety you automate one high-volume class of fix, hold the first monthly review of flow numbers, and add an alert when coverage drops. If the client already works in a pipeline-first model, link the plan to the way security checks are built into delivery pipelines so the two efforts share ownership.

Dates your client may care about

Two items make good conversation starters. A US directive from June 2026 sets computed deadlines for federal civilian agencies, and a FedRAMP alignment date of 7 December 2026 follows. In Europe, the Cyber Resilience Act’s manufacturer reporting duty, with a 24-hour early warning, has applied since 11 September 2026. Neither applies to every client, so confirm scope before alarming anyone, and recommend they take proper advice where obligations are involved.

Wrapping up

The nicest compliment you can get after this workshop is “I did not know we could not prove that.” The audit works because it replaces opinions about tooling with questions about evidence. Run it once, hand back the colour-coded sheet, and you will have a credible scope for the next phase of work.

Kabir Malhotra