Judge
Model Scope & Limits
What can this system reliably do?
States what the system was built and validated for, and flags clearly when a case falls outside that.
- Evidence: research-based (Amershi et al. 2019, Mitchell et al. 2019)
- Tested: concept only, mock data
Overview
Why it matters
The people who built it know its limits. The people using it don’t.
AI systems are built and validated for specific contexts. A credit model validated on employed applicants may fail self-employed ones. These limits are known to the builders and almost never visible in the interface.
Confidence & Uncertainty describes how sure the system is within its scope. This pattern describes where it applies at all. Highly confident and out of scope is one of the most dangerous combinations.
What does it look like?
Out-of-scope detection during processing
Out-of-scope detection during processing
- New application submitted.
- System detects that the applicant is a gig worker with variable income, outside the training distribution.
- Interface flags: “⚠ This application type falls outside the standard validation scope. Automated scoring may be less accurate.”
- Case is automatically routed to a specialist review queue.
- Specialist reviews manually, with the out-of-scope flag visible in the case record.
Operator evaluating deployment in a new market
Operator evaluating deployment in a new market
- Product manager considers deploying the system in a new geographic market.
- Opens the scope and limits documentation.
- Sees: “System validated on applications from Germany and Austria only. Performance in other markets has not been assessed.”
- Decision: commission a local validation study before deployment.
User asks about limitations directly
User asks about limitations directly
- Applicant asks: “Is this system fair for people like me?”
- System shows the relevant scope limitations in plain language: “This system was designed for applications in standard employment. If your situation is different: self-employment, freelance, non-standard income: the assessment may be less accurate.”
- Link to Model Transparency Cards for full documentation.
When does it fit?
Use it when
- The system was validated for a specific population, domain or context
- Some cases are known to be handled less reliably
- Teams decide whether to use the system in a new context
Skip it when
- The scope is broad and well validated, so warnings would never trigger
- The limits are too technical to explain to this user: restrict use instead
Design
How do I apply it?
-
Treat limits as information.
Showing limits openly builds trust. Hiding them destroys it in a single incident.
-
Show scope where it’s used.
A limitations page helps. An alert when this case falls outside the validated range is essential.
-
Make “outside scope” a normal state.
It isn’t an error. Route the case to review, don’t show a generic error message.
-
Be specific about this case.
“Not validated for self-employed applicants, and this applicant is self-employed” is useful. “Has limitations” is not.
-
Pair every warning with a next step.
Human review, a specialist or another tool. A warning alone only creates worry.
-
Update scope with every model version.
Scope can grow or shrink with each release. The display has to follow.
How can I show it?
Scope statement
One plain sentence on what the system was built for.
A plain-language description of what the system is designed for, shown prominently in the interface: “This system was validated for salaried employees with at least 12 months of employment history.”
Use with caution tier
Within scope, edge of scope, outside scope.
A three-tier system: - ✓ Within scope: use normally - ⚠ Edge of scope: use with additional review - ✗ Outside scope: do not rely on this output alone
Out-of-scope indicator
An alert when this case falls outside the validated range.
A visible alert triggered when the current case or query has characteristics outside the validated scope: “This applicant is self-employed. Results may be less accurate: specialist review recommended.”
Version and training date indicator
Model version and training cutoff next to the scope.
Scope changes across model versions. Showing the model version and training data cutoff date alongside the scope indicator helps users assess whether the current version is appropriate for their situation.
Limitations panel
A collapsible list of known limitations.
A collapsible section in the interface (also linked to the Model Transparency Cards) listing known limitations: - Population coverage - Time range of training data - Geographic or linguistic scope - Known edge cases or failure modes
Scope coverage map / visualization
A visual of where the model has been validated.
For systems with geographic, demographic, or domain scope, a visual representation of where the model has been validated. Particularly useful for analysts and operators evaluating whether to deploy in a new context.
Further left for everyday users, further right for operators deciding where to deploy.
What does it cost?
Transparency vs. over-warning
A warning on every case gets ignored. Show it only where the limit really applies.
Disclosure vs. abandonment
Visible limits can make people distrust the whole system. Present scope as a precise boundary.
Accuracy vs. comprehension
A precise description of training data can be unreadable. Find plain words that are still accurate.
Connections
Which patterns work with it?
Confidence & Uncertainty
How certain is the system?
Uncertainty is within scope. Scope & Limits defines where scope ends. Both are needed for a complete picture of system reliability.
Model Transparency Cards
How was this system built and tested?
Model cards are the canonical documentation of scope. This pattern surfaces key scope information at the point of use, with a link to the full card for detail.
Human-in-the-Loop
Who can review or override this decision?
Out-of-scope cases are the primary trigger for mandatory human review. The two patterns should be directly linked.
Grounding & Hallucination Indicators
Can I trust what the system generated?
For LLMs, “out of scope” manifests as hallucination risk. Scope warnings and grounding indicators address the same underlying problem in different AI system architectures.
Sources (4)
Sources (4)
Amershi et al. (2019) — Guidelines for Human-AI Interaction
Guideline G4 (“show contextually relevant information”) and G17 (“be transparent about the system’s limitations”) directly support the design approach of this pattern
Chalmers & Galani (2004) — Seamful Interweaving: Heterogeneity in the Theory and Design of Interactive Systems
introduces “seamful design”: the deliberate surfacing of system limitations and seams as useful information rather than embarrassing failures. Foundational for the design philosophy of this pattern
Mitchell et al. (2019) — Model Cards for Model Reporting
the primary framework for documenting model scope, intended use, and out-of-scope use cases. Model cards are the documentation layer; this pattern is the runtime disclosure layer
Chouldechova (2017) — Fair prediction with disparate impact: A study of bias in recidivism prediction instruments
documents how AI systems validated on one population can produce discriminatory outcomes when applied to another. The empirical basis for why scope and population coverage limits matter for fairness, and why out-of-distribution use constitutes a genuine harm risk
Your model is now used in a context it wasn’t built for.
A new market, a new user group, a new kind of data, and the drop in quality stays invisible until something goes wrong. That’s where this pattern starts. Let’s look at your interface together.
Work with me