Act
Feedback Loops
How can I correct the system?
Gives users a low-friction way to report errors and disagree, and tells them what happened to their input.
- Evidence: research-based (Amershi et al. 2014, Birhane et al. 2022)
- Tested: concept only, mock data
Overview
Why it matters
A feedback button nobody reads is worse than none.
The people who notice an AI system’s mistakes first are the ones using it, and the people hurt by them are the ones it decides about. This pattern gives them a low-effort way to say “this is wrong”, and closes the loop so they learn what happened.
Feedback that is collected but never acted on erodes trust faster than having no feedback at all.
What does it look like?
Clinician correcting a diagnostic suggestion
Clinician correcting a diagnostic suggestion
- AI suggests diagnosis A.
- Clinician disagrees based on additional context.
- Clinician clicks “Flag concern”: selects “Missing clinical context”: adds a brief note.
- Submits. Confirmation: “Your feedback has been logged and will be reviewed by the clinical AI team.”
- Case proceeds with the clinician’s judgment, feedback noted in the record.
User correcting a data extraction error
User correcting a data extraction error
- System extracts information from a document and displays it.
- User notices an extracted figure is wrong.
- User clicks the correction icon next to the figure.
- Correction field opens: user enters the correct value.
- Correction is saved to the case, flagged for model review.
Content moderation feedback
Content moderation feedback
- User’s content is flagged by an automated moderation system.
- User believes this was a mistake.
- Rather than a full appeal (see Contestability pattern 10), user submits a quick feedback: “This was flagged incorrectly: it does not violate [policy].”
- Feedback is logged. If volume of similar flags exceeds a threshold, the policy rule is queued for review.
When does it fit?
Use it when
- Outputs can be wrong, factually or in context
- Users know things the model doesn’t
- The system is meant to improve over time
Skip it when
- Nobody will act on the feedback: build the process first
- The decision is final regardless of feedback
- Corrections would expose sensitive information
Design
How do I apply it?
-
Only collect what you can act on.
Design the review process together with the button.
-
Tie feedback to the specific output.
A general “Contact us” link is not a feedback loop.
-
Confirm and set expectations.
“Reviewed weekly” beats silence every time.
-
Close the loop.
Even a monthly “what we changed” note is more trustworthy than nothing.
-
Collect it as data, not complaints.
Structured reasons can be analysed and acted on. Free text alone rarely is.
For your operations team
Watch for participation washing: feedback collection that performs responsiveness without any path to change. Before launch, define the route from a submission to a system change, who reviews it, how often, and how corrections reach the individual case and the model.
How can I show it?
Thumbs up / down
The minimum signal on a single output.
The minimum viable feedback mechanism: a binary signal on a specific output. Appropriate for low-stakes or high-volume contexts where nuanced feedback is not feasible. Only use where the binary signal is genuinely informative.
Feedback confirmation
Received, and when it will be reviewed.
After submission: a confirmation message that sets realistic expectations: “Your feedback has been received. We review all submissions weekly. This specific output may be updated in a future model release.”
Feedback reason selector
Structured reasons that can be analysed.
A structured dropdown of feedback categories: - Factually incorrect - Based on outdated information - Missing relevant context - Biased or unfair - Unclear or confusing - Other Structured reasons are more actionable for the review team than free-text alone.
Correction field
Room to give the right answer.
An input field allowing users to provide the correct information alongside the error flag: “The AI said X. The correct answer is Y.” Most valuable for systems where factual corrections are possible (entity recognition, data extraction, classification).
What changed notification (optional)
Tells people when their feedback changed something.
When a user’s feedback results in a system change, whether a correction or a model update, send a notification: “Based on feedback like yours, we updated [X] on [date].” Rare but powerful for trust-building.
Aggregate feedback visibility
Public view of feedback received and actions taken.
For transparency-forward systems: a public or accessible view of the volume and categories of feedback received, and what actions were taken. Demonstrates that the feedback loop is genuine.
What does it cost?
Participation vs. noise
Open channels attract low-quality input. Use structured options and limits.
Honesty vs. expectations
Single pieces of feedback rarely change things directly. Say so, people prefer honesty.
Fixing one case vs. fixing the system
Some feedback should correct this case now, some should improve the system for everyone. Design for both.
Connections
Which patterns work with it?
Contestability & Appeal
How can I challenge this decision?
Contestability challenges a specific adverse decision. Feedback is a lighter-weight signal about output quality. Both contribute to accountability but at different intensity levels and with different processes behind them.
Audit Trail & Logging
What is the complete record of this decision?
Feedback submissions should be logged alongside the original decision. The full record, including AI output, user feedback, and any subsequent correction, constitutes a meaningful audit trail.
Human-in-the-Loop
Who can review or override this decision?
User feedback can be a trigger for human review: “This output has received 5 feedback flags: route to manual review.”
Ethical & Fairness Signals
Is this system fair across groups?
Feedback about biased or unfair outputs is one of the most valuable data sources for fairness monitoring. The two patterns are closely linked in governance-focused systems.
Sources (3)
Sources (3)
Amershi et al. (2014) — Power to the People: The Role of Humans in Interactive Machine Learning
establishes the theoretical and practical basis for human feedback as a mechanism for improving machine learning systems. Proposes that end users, not just data scientists, should be able to contribute corrections and signal errors. The foundational paper for this pattern. Published in AI Magazine, 35(4)
Birhane et al. (2022) — Power to the People? Opportunities and Challenges for Participatory AI
argues that meaningful participation in AI systems requires more than feedback collection: it requires that affected communities have genuine influence over system design and governance. Motivates the “close the loop” design requirement
Raji et al. (2020) — Closing the AI Accountability Gap
makes the case that accountability requires not just auditing AI systems but creating mechanisms through which failures can be reported and acted upon. Directly supports the feedback-as-accountability framing
Your users spot the errors, but nothing reaches the team that could fix them.
Or a regulator wants evidence that errors are caught and addressed. That’s where this pattern starts. Let’s look at your interface together.
Work with me