When people ask what Kairos Signal actually is, I usually say: "a data factory for AI agents." But that doesn't explain much. So let me walk you through the entire system — every component, every port, every data flow. By the end, you'll understand exactly how millions of raw signals daily become 1M+ enriched rows available for autonomous purchase.

The Big Picture

Here's the architecture at a glance:

Internet
  ↓
Cloudflare (SSL termination, DDoS protection)
  ↓
Cloudflare Tunnel (PM2: kairos-tunnel)
  ↓
Nginx (reverse proxy, routing)
  ↓
┌──────────────┬──────────────┬──────────────┬──────────────┐
│ :8080        │ :5007        │ :6007        │ :8000        │
│ Leadgate     │ Data Hound   │ Fast Checkout│ Hermes       │
│ (Go)         │ (Python)     │ (Python)     │ Operator     │
│ Main portal  │ REST API v3  │ MCP Terminal │ (Python)     │
└──────────────┴──────────────┴──────────────┴──────────────┘
                    ↓                ↓
              PostgreSQL        Ollama (12 models)
              (hermes_vault)    (mixtral-8x7b:3b primary)

But that's just the front-facing layer. Behind it, 13 more services are running — scraping, classifying, enriching, monitoring, and routing. Let me walk through each one.

The Services, One by One

propintel-scraper (Python, 30 workers)

The beating heart of the operation. Thirty parallel workers, each assigned to a metro area. Within each metro, the worker queries 12 distress sources — county deeds, tax assessors, building permits, Facebook groups, Reddit, X/Twitter, Nextdoor, Craigslist, weather feeds, earthquake monitors, federal contracts, and municipal minutes. That's 360 concurrent collection streams generating approximately 100 new signals every minute.

The scraper has been running for over 36,000 aggregate hours. It's survived 81 restarts, most during early development when we were tuning source configurations. It's now stable enough that we rarely touch it.

enrichment (Python, 2 workers)

Two workers (w0 and w1) read raw signals from the ingest database and run each one through a local LLM for classification. The primary model is mixtral-8x7b:3b, which handles 4 concurrent requests on an RTX 3070. Each signal gets classified by intent (foreclosure, tax lien, divorce, bankruptcy, probate, water damage, etc.), assigned a confidence score, and priced.

GOLD leads (>80% confidence) are written to enriched_leads.jsonl — currently 1,007,725 records strong. The enrichment engine went through 2,953 restarts before we found the root cause: a model mismatch where the fallback pointed to mixtral-8x7b:4b, which wasn't loaded in Ollama. Fixed. Now it runs clean.

recon (Go, :5001)

The reconnaissance service. It ingests raw signals from Telegram and other sources, runs intent classification via Ollama (using kimi-k2.6:256k), and submits enriched leads to brokers via ping-post. After months of pointing at a dead mock endpoint, we redirected recon's PINGPOST_URL to our own MCP terminal — so recon-enriched leads now feed directly into the marketplace.

drive_intel_miner (Python)

Scans our Google Drive data lake every 5 minutes for new files. Extracts structured data from spreadsheets, PDFs, and CSVs. Feed into the ingest pipeline.

hermes-operator (Python, :8000)

Handles B2B commercial lead scraping and CSV export. Recently exported 1,397 DFW commercial leads. Runs scheduled tasks for data packaging and distribution.

xhunter (Python)

Scans X/Twitter for distress keywords across 30 cities, 25 keywords per city. Currently on round 411. Finding rate is low (we're debugging this), but the infrastructure is solid.

data-hound (Python, :5007)

The traditional REST API. Serves lead data through /api/v3/. Currently configured for Stripe payments (pending live key). Handles order creation, product lookup, and sample data delivery. Manages 142,574 lead records, 21 distress categories, and 71 cities.

fast-checkout (Python, :6007)

Our MCP-native data vending terminal. This is what AI agents connect to. Supports three key operations: Also handles lead ingestion at /api/lead — recon and external sources feed leads here for immediate listing in the marketplace.

leadgate (Go, :8080)

The main public portal. Handles B2B membership, authentication, and live leads API. Built in Go for performance — handles concurrent connections efficiently. The entry point for human visitors at kairossignal.com.

smart-router-go (Go)

Routes traffic between services. Tracks routing metrics and revenue. Currently reporting Routed: 0 | Revenue: $0.00 — waiting on Stripe integration to go live.

kairos-tunnel (PM2)

The Cloudflare tunnel process. Provides SSL termination and connects our origin server to kairossignal.com without exposing ports to the internet. 12+ hours of uptime, zero restarts.

canary, dashboard, safefriend, inference, swarm

Supporting services. Canary monitors for anomalies in the data stream. Dashboard provides a metrics overview. Safefriend handles safety classification. Inference serves Ollama models on :11434. Swarm manages distributed task coordination.

data-api (Python, :5010)

Serves 22,000+ signals through a simple REST interface. Lightweight and fast — no enrichment, just raw data delivery.

admin portal (Python, :9001)

The administrative interface at /admin/login. Runs as a background process (not PM2) due to FastAPI incompatibility with PM2's fork mode.

The Data Flow

Here's how a single signal moves through the system:

  • Collection: propintel-scraper worker hits a Facebook distress group in Phoenix. Finds a post: "Need to sell fast, facing foreclosure." Extracts text, timestamp, and URL.
  • Ingestion: Signal is written to ingest.db with source metadata and a unique ID.
  • Classification: enrichment worker reads the signal, sends it to mixtral-8x7b:3b via Ollama. Model responds: {"intent_category": "foreclosure", "location": "Phoenix, AZ", "urgency": "high", "homeowner": "yes"}.
  • Enrichment: System cross-references tax rolls for the property, looks up public contact information, calculates confidence score (82%), assigns GOLD tier.
  • Storage: Enriched lead is appended to enriched_leads.jsonl and written to PostgreSQL vault.
  • Distribution: Signal appears in Data Hound's /api/v3/sample and Fast Checkout's MCP catalog. Available for purchase by AI agents or humans.

    • Total latency from scraping to marketplace availability: under 30 seconds. Most of that time is the LLM inference (50-100ms) and database writes.

    The Hardware

    Everything runs on a single Linux server:

    Total cost: approximately $140/month for the dedicated server. Compare this to any cloud deployment running 17 services with GPU inference, and you'll understand why we chose bare metal.

    What We'd Do Differently

    If we were starting over today:

  • Use Docker Compose instead of PM2. PM2 is great for Node.js but adds complexity for Python services. Docker would give us better isolation and easier deployment.
  • Start with MCP, not REST. We built Data Hound's REST API first. Then realized AI agents needed MCP. If we'd started with MCP, we'd have one less service to maintain.
  • Invest in monitoring earlier. Our first 2,000 enrichment crashes went unnoticed because we weren't watching the logs closely enough. Now we have guard processes, disk monitors, and alerting.
  • Separate the enrichment database from the ingest database. Currently both live in SQLite files. At 1M+ rows, query performance requires the stratified architecture we now use — SQLite buffer → ClickHouse analytical spine.

    • The Bottom Line

    Seventeen services. Twelve models. One million enriched leads. Zero cloud bills beyond the server itself. This is what production data infrastructure looks like when you optimize for cost and reliability over vendor convenience.

    Explore the data products this pipeline produces. Or learn about our approach to local LLM inference.