Understand
Inspection Dialogue
Can I ask the system follow-up questions?
Lets users interrogate a result directly, probing single factors and testing what-ifs in their own words.
- Evidence: research-based (Lim et al. 2009, Miller 2019)
- Tested: concept only, mock data
Overview
Why it matters
Static explanations answer the designer’s question, not the user’s.
A caseworker wants to know: “What would this score be if I corrected the employment dates?” A fixed chart can’t answer that. This pattern lets people ask follow-up questions about one decision, test what-ifs and get answers with a visible basis.
As AI interfaces become conversational, this pattern becomes both more natural and more important.
What does it look like?
Applicant follow-up question
Applicant follow-up question
- Applicant sees attribution showing “employment gaps” as a top negative factor.
- A suggested question appears: “Why does employment history matter for this decision?”
- Applicant selects it.
- System responds: “Consistent employment history reduces repayment risk. Gaps in the past 24 months were flagged because they suggest income instability.” Linked to the attribution factor for context.
- Applicant understands the reasoning and moves to recourse options.
Caseworker hypothetical investigation
Caseworker hypothetical investigation
- Caseworker suspects the employment data was entered incorrectly.
- Opens inspection panel and types: “What would the score be if the employment gap were 3 months instead of 12?”
- System calculates and responds: “With a 3-month gap, the score would increase from 42 to 61: above the referral threshold.”
- Caseworker requests corrected documentation and updates the case.
Out-of-scope question
Out-of-scope question
- Applicant asks: “Is this system biased against people like me?”
- System responds: “I can only answer questions about the specific factors that contributed to your decision. For information about how the system was validated for fairness, see [Model Transparency Card].”
- User is redirected to the appropriate resource rather than receiving an unreliable generative answer.
When does it fit?
Use it when
- People have varied follow-up questions no fixed explanation can cover
- Experts need to investigate edge cases
- The interface is already conversational
Skip it when
- Speed matters more than depth
- The system can’t answer follow-ups reliably
- The real question is “what can I change?”: use Actionable Recourse
Design
How do I apply it?
-
Scope it to this decision.
“I can answer questions about this recommendation” is safer than an open chat.
-
Offer questions, not a blank field.
Most people don’t know what to ask. Suggest questions the system can answer well.
-
Show the basis of every answer.
An answer without a visible basis can’t be checked.
-
Say when a question is out of scope.
Better “I can’t answer that reliably” than a plausible guess.
-
Log the questions.
If a caseworker overrides after a what-if, that’s part of the record.
For your technical team
Research on questioning context-aware systems identifies recurring question types (why, why not, what if, how to) that a guided question set should cover (Lim et al., 2009). Generated answers about a decision must be grounded in the model’s actual outputs; pair free-text questions with grounding indicators.
How can I show it?
Guided question set
Suggested follow-up questions to pick from.
A set of pre-defined, contextually relevant follow-up questions presented as quick-select options. Research on how users question context-aware systems identifies four recurring question types that inspection dialogue must support: - Why: “Why did employment history matter so much?” (explaining the factors) - Why not: “Why wasn’t my income enough on its own?” (contrastive, ruling out alternatives) - What if: “Would a higher income have changed the result?” (hypothetical / counterfactual) - How to: “What would I need to do to get a different outcome?” (recourse-seeking) Generating contextually relevant questions from each of these types, based on the actual factors in the decision, provides structured coverage of what users genuinely need to understand.
What if input
Change one value and see the effect.
A structured form allowing users to change one variable and see the hypothetical impact: “What if my income were €X?” This is inspection dialogue structured as a counterfactual query: a bridge between inspection and Counterfactual Explanation.
Follow-up answer card
Answers anchored to the part of the explanation they concern.
When a user asks a question, the answer appears as a new card anchored to the relevant part of the explanation, not as a free-floating response. This keeps the inspection grounded in the specific decision rather than abstractly conversational.
Inspection history
A record of the questions asked.
A record of the questions asked during an inspection session: useful for audit purposes and for caseworkers who need to document the reasoning behind an override decision.
Explanation chat panel
A chat limited to this one decision.
A side panel with a conversational interface specifically scoped to the current decision. The system can only answer questions about this decision, not general questions about the AI. Scoping prevents the conversation from drifting outside what the system can reliably answer.
Free-text question input
Open questions, paired with grounding signals.
An open text field allowing users to ask their own questions. Most powerful but requires the system to understand and answer reliably. Should be paired with grounding indicators to signal when the system is uncertain about its answer.
What does it cost?
Interactivity vs. reliability
The more open the dialogue, the more unreliable answers. Limit questions to what can be answered.
Depth vs. efficiency
It slows down high-volume work. Make it available, not mandatory.
Transparency vs. gaming
“What if I change this?” is the first step to gaming. Limit what-ifs where that’s a risk.
Connections
Which patterns work with it?
Explanation Layering
How much detail do I need?
Layering offers depth through structure. Inspection dialogue offers depth through interaction. For most users, layering is sufficient; inspection dialogue is for users who need to go further.
Counterfactual Explanation
What would have changed the outcome?
“What if” inspection is functionally a counterfactual query. The patterns share ground and can be combined: counterfactuals are a structured form of inspection dialogue.
Attribution
Why did the system make this decision?
Attribution provides the content that inspection dialogue allows users to probe. The two patterns work together: attribution answers the first question, inspection dialogue answers the follow-up.
Human-in-the-Loop
Who can review or override this decision?
For caseworkers, inspection dialogue is part of the override workflow. The investigation a caseworker conducts before an override should be logged as part of the decision record.
Grounding & Hallucination Indicators
Can I trust what the system generated?
In LLM-powered inspection interfaces, grounding indicators are essential. Users need to know when the system’s answer to a follow-up question is well-supported vs. potentially fabricated.
Sources (3)
Sources (3)
Lim et al. (2009) — Why and Why Not Explanations Improve the Intelligibility of Context-Aware Intelligent Systems
studies how users ask “why” questions of context-aware systems and what kinds of answers satisfy them. Foundational for understanding the natural follow-up question structure that inspection dialogue must support
Miller (2019) — Explanation in Artificial Intelligence: Insights from the Social Sciences
establishes that explanation is inherently interactive and conversational, not a static transfer of information. Directly motivates inspection dialogue as a pattern
Lakkaraju et al. (2022) — Rethinking Explainability as a Dialogue: A Practitioner’s Perspective
argues, based on interviews with practitioners, that explanations should be interactive dialogues rather than pre-packaged one-shot outputs. Direct support for the conversational structure of this pattern
Your experts keep asking questions your interface can’t answer.
Caseworkers, clinicians and analysts need to probe edge cases. That’s where this pattern starts. Let’s look at your interface together.
Work with me