How I implemented continuous defense.

From the article to the implementation

In Continuous Defense: Start Small. Build Confidence. Accelerate the Scale., I set out the approach. This page records what we actually built, what the pilot showed, and what remains to be done.

The risk we wanted to address

An application can become vulnerable when a new issue is disclosed in an unchanged dependency. We wanted a maintained understanding of our application that could connect fresh advisory and exploitation evidence to the exact software we run.

This helps us react to disclosed risks, including vulnerabilities previously exploited as zero-days. Advisory matching cannot establish that undisclosed flaws are absent.

What the Architect built

Security Skills Architect's application-defence workflow produced defend-example-application: a custom skill with application context, selected code references, an evidence profile, checks, finding rules and an operating runbook. I use the anonymised name here; the private package retains the application-specific identity. The example application handles uploaded documents and private account data, so PDF processing and ownership boundaries matter to its review.

The skill defines how to interpret evidence and what needs verification. A separate executor supplies collection, durable state and recurrence. Codex helped build these pieces; an existing local Qwen3:30B model runs the recurring interpretation on a local host.

How the recurring check works

1. Identify the running API container and collect its installed Python package versions.
2. Query OSV for affected-version matches and retrieve the advisory details.
3. Group aliases and cross-reference CVEs with CISA's known-exploited vulnerability catalogue.
4. Record new or changed findings, feed freshness, vendor-listed fixes and missing evidence.
5. Ask the local model to explain the evidence using the application-specific skill.
6. Retain the report and require fresh retesting to verify a dependency fix.

OSV receives package names and versions. Source code and model prompts remain local. The model has no command tools or patching authority. The timer is configured for 15-minute reviews; the separate same-host watchdog reports stale or failed coverage.

What the pilot showed us

The PDF findings are relevant to the application's purpose. Its reviewed import code opens PDFs and inspects their pages; its sharing code opens PDFs and parses page content. Both files matched the deployed versions. This establishes PDF-library use, while authentication, scanner availability and each advisory's triggering conditions still need review.

Maintainer advisories describe an image-loading issue fixed in pypdf 6.21.1 and a malformed-PDF resource-consumption issue fixed in 6.21.2. The deployed version observed during the pilot fell within their affected ranges.

What we need to do next

1. Prioritise a compatible pypdf update and regression tests for PDF import, conversion and sharing.
2. Review the pip matches against build and runtime installation behaviour, then prepare the appropriate image update.
3. Verify affected code paths and preconditions for each material finding; keep unknown reachability explicit.
4. Rebuild and retest the deployed application before recording a verified fix.
5. Connect the findings and delivery-health feedback to krnali Ops.

These are recommendations and verification work arising from the pilot. The application dependency fixes have not been implemented by this exercise.

What the test changed

The first implementation repeated six container-setting checks. Reviewing it against the emerging-vulnerability objective showed that this was insufficient. We added deployed-package inventory, current advisory queries, known-exploitation enrichment and finding reconciliation.

The review also exposed defects in state-loss handling, capability checks, model-output validation and the separation of instructions from application data. Six targeted regression probes now pass after corrections, alongside the initial 16 runtime unit tests. A subsequent separate-agent security review examined the runtime and deployed service configuration. Further state, exploitation-history, temporary-file and watcher corrections now have 29 passing runtime tests, independently re-run, and the bounded pilot update completed a staged run before installation. The model also incorrectly inferred non-use from unassessed reachability; stronger instructions and a regression guard address that demonstrated mistake. Its explanations still require repeated quality evaluation.

The security review and release gates

The scoped AI-assisted review is recorded in a sanitised security follow-up. It is evidence for a bounded pilot rather than external certification. Collector authority, independently trusted inventory, transactional recovery, resource limits and independent delivery remain production acceptance work.

We need to exercise a real finding-to-remediation loop, compare model results against held-out cases and deterministic baselines, and observe sustained failure and recovery behaviour. More passing unit tests alone cannot close those operational gates.

The protection and its limits

The pilot gives us recurring evidence about disclosed dependency risks relevant to our application, together with an actionable review backlog. A feed outage preserves open findings and reports degraded coverage. A model explanation cannot close an issue.

The 17 matches do not establish 17 exploitable application bugs, confirmed zero-days or compromise. Coverage currently centres on Python packages in the API container and selected code evidence. Worker/client components, native and OS dependencies, complete source analysis, runtime exploit detection, independent external notification and live Ops delivery remain gaps. Operational acceptance is conditional.