Falaah Falaah AI

Compliance Score Engine (Reference)

Technical reference for Muin's compliance scoring algorithm: exact factor weights, the cache TTL, the readiness threshold model, and trend-delta computation.

This is the technical reference for the Muin compliance score engine. If you’re looking for a user-facing walkthrough of the /compliance landing, start with Compliance Health. This page is the algorithmic reference — intended for compliance owners debugging a score, auditors evaluating the methodology, and engineers wiring custom integrations.

Source of truth: muin.server/src/services/compliance/score_service.py.

The Formula

The overall compliance score is a weighted aggregate across seven factors:

overall_score = sum(factor_value × WEIGHTS[factor] for factor in factors) × 100

Each factor_value is a ratio in [0, 1], computed from live tenant data at read time (with a 5-minute cache).

The WEIGHTS Dict (matches code exactly)

Factor Weight Code reference
framework_completion 0.25 score_service.py:41
risk_management 0.20 score_service.py:42
training_compliance 0.15 score_service.py:43
policy_documentation 0.15 score_service.py:46
evidence_freshness 0.10 score_service.py:44
vendor_assessment 0.10 score_service.py:47
license_compliance 0.05 score_service.py:45
Total 1.00

This table is CI-verified — muin.server/scripts/verify_score_weights_doc.py reads the WEIGHTS constant from score_service.py and asserts every line in this table matches. Documentation drift is caught at build time.

Per-Factor Computation

Each factor is computed in ComplianceScoreService.compute_score_from_metrics:

Factor How computed Sources
framework_completion Weighted mean of active framework completion_percentage, weighted by control count ComplianceFramework.completion_percentage, total_controls
risk_management 1 - (open_high_risks / total_risks), capped at 0 / 1 ComplianceRisk.status
training_compliance completed_in_window / assigned_in_window TrainingAssignment.status, due_date
policy_documentation (active_policies_with_fresh_review / total_active_policies) HRPolicy.status, review_date
evidence_freshness evidence_within_ttl / total_evidence_items Linked Document.updated_at
vendor_assessment vendors_with_risk_tier / active_vendors VendorProfile.risk_level
license_compliance active_not_in_alert_window / active_licenses BusinessLicense.expiration_date, alert_days

Factors with denominator zero (e.g. 0 / 0 for a tenant with no vendors) return a fixed neutral value (typically 1.0 — absence of data doesn’t penalize). The plain-English implication: brand-new tenants don’t start at 0%.

Caching

Constant Value
CACHE_KEY_PREFIX compliance:score:
CACHE_TTL_SECONDS 300 (5 minutes)

The cache is Redis. Keys are scoped to tenant: compliance:score:{tenant_id}.

Invalidation

The cache is invalidated explicitly via invalidate_score_cache(tenant_id) when data affecting the score changes:

  • Control status transitions (implemented / tested / certified)
  • Risk status transitions (closed / accepted / mitigating / open)
  • Training assignment completion
  • Evidence upload (via Documents module linked-control hook)
  • License renewal or manual status edit
  • Vendor risk tier set or changed
  • Policy version publish

Edits that don’t affect a status field (renames, description tweaks, assignee changes) don’t invalidate — the 5-minute TTL catches any late-breaking data organically.

Computed-At Timestamp

Every cached payload includes computed_at (ISO-8601 UTC). The frontend gauge reads this to render the “Last updated N minutes ago” hint and to decide whether to offer a manual refresh button.

Readiness Threshold

A per-framework readiness threshold (default 80) gates the “audit-ready” badge:

is_audit_ready = overall_score >= readiness_threshold
gap_to_ready = max(0, readiness_threshold - overall_score)

The threshold is read from the tenant’s SOC 2 framework (ComplianceFramework.readiness_threshold) when available, falling back to 80. Non-SOC 2 tenants get a meaningful threshold too — so the gauge never shows “0 points to go” on a tenant without SOC 2.

This matches the BLI-2 fix — readiness is always computed.

Per-Framework Breakdown

The payload also includes per_framework_scores — one entry per active framework with:

Field Meaning
framework_id UUID
name Human name (e.g. “SOC 2 Type II”)
framework_type Canonical type (e.g. soc2)
score This framework’s completion_percentage, rounded to one decimal
total_controls Number of controls in this framework
implemented_controls Count with status in {implemented, tested, certified}
status Framework’s lifecycle status

These per-framework scores are independent of the overall compliance score — they’re the raw completion percentage for that framework alone.

Trend Delta (At-a-Glance Card)

The At-a-Glance card’s +X.X vs last week readout compares the current score to a snapshot from 7 days ago (read from ComplianceScoreSnapshot). The rendering logic:

Delta Display
> +0.5 Green up-arrow + +X.X vs last week
-0.5 ≤ delta ≤ +0.5 Grey “Stable vs last week”
< -0.5 Red down-arrow + X.X vs last week

The ±0.5 band prevents noise — cache expiry at weekly boundaries, minor data refreshes, and rounding effects don’t register as movement.

A daily worker (compliance_score_snapshot_task) writes the current score to ComplianceScoreSnapshot so the lookback is a fast indexed read, not a recomputation.

The AI Exec Summary (D9)

The /compliance/score-explain endpoint drives the At-a-Glance card. See Compliance Health § At-a-Glance Exec Summary for the user-facing description.

Behavioral guarantees:

  • Always returns — on LLM failure, rate-limit, or fence rejection, the endpoint returns a deterministic rule-based fallback narrative with summary_source=fallback
  • Citation fence — the LLM’s cited factors and control codes must exist in the current breakdown; failures fall back
  • Cache — 1-hour Redis cache keyed on (tenant, breakdown_hash) — narrative changes when the underlying breakdown changes
  • Rate limit — 30 per tenant per hour

The frontend reads summary_source and renders a badge (LLM / Fallback / Network error). No “fake” LLM output is ever shown as LLM-sourced.

Extensibility

The factor list and weights are not user-configurable today. Adjustable per-tenant weights are tracked in Plan 276 for a future release. Custom factors (e.g. a tenant-specific “audit coverage” factor) are planned for a future release as well.

Design constraint: the weights table is load-bearing for CI verification and audit defensibility. Adjustable weights need a second “effective weights” table per tenant + a doc-generation path that renders the tenant’s actual weights in their tenant-scoped help content. That’s the Plan 276 work.

CI Verification

The table above must match the code. Drift is caught by:

python muin.server/scripts/verify_score_weights_doc.py

The script reads WEIGHTS from score_service.py, reads the weights table from this MDX file’s frontmatter block, and exits non-zero if any row differs or is missing. Runs in pre-commit and CI.

On discrepancy, the build fails with a diff showing which weights drifted. Fix: update either the code or the doc until they agree.