Skip to content
Auto-CTI

Auto-CTI-Swarm

How the system works

Auto-CTI-Swarm is an autonomous multi-agent cyber-threat-intelligence pipeline, built for SMEs without a dedicated security team. A seven-stage Python pipeline ingests threat sources daily, triages and enriches them with Claude, classifies threat actors, scores deterministically, checks itself by consensus and writes everything to Cloudflare R2. This website is the read-only frontend: no database, rendered straight from R2.

Architecture

Architecture cti-agents pipeline (7 stages) writes to Cloudflare R2; this Astro site reads it via Pages Functions and an R2 binding. Quellen / Sources ▸ RSS (4 Feed-Typen) ▸ NVD CVE API v2 ▸ CISA KEV Catalog ▸ FIRST.org EPSS ▸ Jina Reader (Scrape) ▸ MITRE ATT&CK Matrix + Claude Haiku · Sonnet + DeepSeek (Vote · RAG) + Jina Search (Akteure) kein LLM-Key? → Fallbacks cti-agents (Python) 7 Stufen · Cron / Manuell 1 · Ingestion 2 · Enrichment (Triage) 3 · Risk Scoring 4 · Deep Dive + TTPs 5 · Threat-Actor Intel 6 · Consensus + Vote 7 · Reports + Alerts writes Cloudflare R2 index/ · enriched/ · risk/ ttp/ · reports/ · meta/ S3-kompatibel · Free 10 GB R2-Binding Cloudflare Pages Astro 5 SSR · Vanilla JS Edge-Cache 60 s 👤 Browser

The pipeline (cti-agents) writes to R2; this Astro site reads it at the edge through an R2 binding, with no database of its own.

The pipeline in detail

Seven stages, from an RSS line to the delivered report:

  1. Ingestion

    RSS (alerts, news, research, strategic), NVD CVE API, CISA KEV (incremental watermark) and FIRST.org EPSS. News and research are fetched in full via Jina Reader, with ~80–90 % of irrelevant news pre-filtered. Guardian dedup (processed_hashes, FIFO 5000) plus a regex guard against prompt injection.

  2. Enrichment

    Claude Haiku triages every item for relevance to the company profile (parallel, 8 workers). The correlator collapses CVE clusters, the enricher pulls CVSS/EPSS/KEV, and a deterministic vendor-consistency check flags title/vendor contradictions.

  3. Risk scoring

    Pure deterministic formula: CVSS + KEV bonus + EPSS + industry weight + freshness decay, with fixed floors (e.g. KEV + CVSS ≥ 9 → at least 75). No LLM, fully reproducible. Runs before the deep dive because the model router needs the score.

  4. Deep dive + TTPs

    Only for high-scoring alerts (score ≥ 65 or KEV): the model router picks the tier, Claude produces impact + mitigation, and the TTP mapper assigns MITRE ATT&CK techniques, validated against the official matrix, with a mandatory procedure per technique.

  5. Threat-actor intelligence

    Named actors are classified through an ordered resolver chain (curated table → naming decoder → optional RAG inference with cross-provider consensus → unknown). Each actor is then enriched deterministically: company relevance, activity status, CVE severity, MITRE playbook + mitigations and related actors. Everything flows into a cumulative registry (meta/threat_actors.json).

  6. Consensus + voting

    Two-stage quality gate: 7 deterministic regex rules catch contradictions (e.g. KEV listing vs. "no exploitation"), then a cross-provider vote (Sonnet vs. DeepSeek) for the top alerts. Disagreement only ever lowers the score.

  7. Output → R2

    Executive, technical and weekly report as JSON + PDF (HTML→PDF), validated before release. Delivery via Telegram + email for critical/KEV alerts and re-escalations. Everything is written to Cloudflare R2 and linked from the daily index index/{date}.json.

Consensus & voting

Before a report ships, the pipeline checks itself: deterministic first, then an optional model vote. Both stages can only lower a score, never raise it. Plausible-sounding but self-contradictory items don't survive.

Stage 1: Deterministic rules

Seven pre-compiled regex rules (EN/DE) look for contradictions between the hard facts (KEV, CVSS, EPSS, vendor, CVE, TTPs) and the prose. Each rule carries a fixed weight:

Rule Trigger Penalty
KEV_DENIAL Text contradicts the KEV listing (no exploitation reported) −15
CVSS_LOW_RISK High CVSS despite low-risk wording −8
TTP_MISMATCH ATT&CK technique in the text is absent from the mapping −8
EPSS_UNLIKELY High EPSS despite "exploitation unlikely" wording −6
VENDOR_MISMATCH Vendor not found in the title −5
CVE_ABSENT Title-claimed CVE missing from the description −3
TTP_SKIPPED TTP mapping skipped (placeholder / aggregation article) −3

Total per alert capped at −30 points. From −8 the alert is flagged for manual review and excluded from the LLM vote.

Stage 2: Cross-provider voting

Only the top-5 alerts per day with score ≥ 85 that passed stage 1 cleanly enter the vote. Anthropic Sonnet and DeepSeek Pro (two different providers) answer the same five weighted fact questions (KEV exploitation, field contradiction, risk tier, vendor, relevant TTPs). Each disagreement subtracts points (max −15). Votes are cached in R2 and idempotent: same alert, no second LLM call.

Anthropic · Sonnet vs. DeepSeek · Pro

Threat-actor intelligence

Every threat actor named in an item runs through an ordered resolver chain: first hit wins. Each result carries its provenance and a confidence (0–1), so a bare LLM guess never appears as confirmed attribution. The deterministic resolvers (1–2) always run, with no network and no API key.

Classification: resolver chain

  1. Curated Table

    provenance=table conf 1.0

    31 hand-curated actors (APT28, Lazarus, LockBit …) with country, type and motivation. Authoritative, append-only, no network or LLM.

  2. Naming Decoder

    provenance=decoder conf 0.5–0.9

    Decodes documented vendor schemes: Microsoft weather (Typhoon/Blizzard), CrowdStrike animals (Bear/Panda), Palo Alto CL-/TGR-, Microsoft Storm-####. Purely deterministic.

  3. RAG Inference

    provenance=rag conf ≤ 0.85

    Opt-in: evidence from MITRE and vendor pages via Jina, then a 2-call cross-provider consensus (e.g. Anthropic + DeepSeek). Inference only, never overrides 1–2.

  4. Unknown

    provenance=unknown conf 0

    No resolver hits → type stays open and is honestly marked unknown.

Enrichment: per actor

Beyond attribution, the pipeline enriches every actor deterministically, the web only renders it:

Company relevance

Deterministic overlap with company_profile.yaml (tech stack, threat focus, sector/region) with traceable reasons. "Affects your stack" only on a direct product hit.

Activity status

Active (≤ 30 days), dormant (≤ 365) or historical, computed from last_seen at render time, so never stale.

CVE severity

CVSS, EPSS and KEV status per named CVE, sorted by danger.

MITRE playbook + mitigations

Techniques grouped by kill-chain tactic; mitigations come deterministically from the ATT&CK matrix, no LLM guessing.

Related actors

Linked via shared CVE, ATT&CK technique or malware.

Attribution is authoritative only from the table/decoder; RAG inference (opt-in, cross-provider consensus) runs only for open actors, never overrides 1–2 and is flagged "inferred" in the UI. Resolver 1–2 and all enrichment are deterministic and run without an API key; without one, an actor degrades to "unknown" plus deterministic enrichment. The cumulative registry (meta/threat_actors.json) feeds this site's Actors page.

Output & delivery

Each run produces validated reports and makes them available over three channels:

Reports (PDF + JSON)

Executive, technical and weekly report as JSON and PDF (HTML→PDF via WeasyPrint + Jinja2), deterministically checked for top-threats consistency before release. On a breach a template fallback kicks in instead of a hard fail.

Telegram + email

Two optional push channels (the pipeline runs without them too): urgent text alerts on critical/KEV hits and re-escalations (newly KEV-listed, CVSS/EPSS jump), plus the report PDFs as attachments. Bilingual: DE channels by default, separate EN channels. Email via Resend (SMTP fallback), with a 30-day CVE dedup against spam.

This website

The live frontend (Astro 5 on Cloudflare Pages) reads R2 through an R2 binding and renders exactly one precomputed read-model per page, instead of reading per alert: threats/recent.json, meta/threat_actors.json and the daily index. That keeps every page to a single GET, under the edge subrequest limit. 60s edge cache, bilingual (DE/EN), with RSS and JSON endpoints.

Tech stack

Pipeline (cti-agents)

  • · Python 3.12
  • · Pydantic v2 + pydantic-settings
  • · httpx + tenacity (Retries)
  • · Anthropic SDK + OpenAI-SDK (DeepSeek-kompatibel)
  • · boto3 (Cloudflare R2 / S3-API)
  • · feedparser (RSS/Atom)
  • · structlog (JSON Logs)
  • · WeasyPrint + Jinja2 (PDF)
  • · Matplotlib (Charts, Agg-Backend)
  • · pytest + respx (41 Suites · 800+ Tests)

Web (cti-swarm-web)

  • · Astro 5 + @astrojs/cloudflare (output: server)
  • · Vanilla JS Toggles · kein Framework
  • · Tailwind CSS v4 + @tailwindcss/vite
  • · Zod (R2-Schema-Validierung)
  • · Cloudflare Pages Functions + R2-Binding
  • · Playwright + axe (E2E + a11y)
  • · Lighthouse (Self-Review-Loop)

Costs · everything on free tier

The whole stack runs on free-tier quotas. The only paid component: Claude API calls (~90% cheaper with prompt caching).

Service Free-tier limit Current
Cloudflare R2 (storage) 10 GB ·
Cloudflare R2 (Class A ops, MTD) 1 000 000 / month ·
Cloudflare R2 (Class B ops, MTD) 10 000 000 / month ·
Cloudflare Pages unlimited requests · 500 builds/month ·
Anthropic Claude API paid (~90% caching discount) ·
DeepSeek API (consensus vote) paid (top-5 / day only) ·
Resend (Email) 3 000 / month ·
Telegram Bot API free (bot rate limit) ·
Jina Reader / Search 1M tokens/month ·
NVD · CISA KEV · FIRST.org EPSS free (public) ·

Live usage appears once the pipeline writes meta/usage_summary.json.

Source code

Both repositories are private. Architecture, prompts, consensus rules and tests live entirely in code, no hidden logic in closed-source services.

ESC

…