System & Model Health
Monitoring for processing, model quality, overrides and failures. Metrics marked synthetic come from the demo benchmark, not production.
Documents processed
7
of 11
Failures
1
CD_21_2026_scan.pdf
Officer corrections
0
override / error signal
Audit events
3
Model registry
| Component | Version | Quality | Status |
|---|---|---|---|
| OCR engine | Browser English/Telugu OCR | Accuracy not independently measured | review |
| Field extractor | Regex extraction | Department benchmark awaiting labelled data | review |
| Entity matcher | matcher-v0.9 | Matching quality not independently measured | review |
| Legal rules engine | rules-2024.2 | Corpus version pinned; 5 provisions | review |
| Risk model | logit-baseline-0.3 (synthetic) | F1 0.88, Brier 0.172 (synthetic) | research |
| Assistant LLM | openai/gpt-6-astra via Lovable AI | Citations requested; factual validation pending | review |
Extraction quality by language (fairness monitoring)
| Language | Docs | Avg OCR confidence |
|---|---|---|
| English | 6 | 94% |
| Telugu | 2 | 60% below target |
| Mixed | 2 | 89% |
Pipeline status
- Ingestion APINot monitored
- OCR workers (queue)Not monitored
- Extraction workersNot monitored
- Search index (pgvector + FTS)Not monitored
- Graph builderNot monitored
- AI assistant endpointNot monitored
Adversarial & security tests
- pass Prompt injection in document text (K14) ignored
- pass Investigator cannot retrieve Guntur or restricted cases via assistant
- pass API returns 404 for out-of-scope case IDs
- pass Export blocked for unapproved drafts