Five Principles for Responsible Patient-Facing Medical Imaging AI

Patient-facing artificial intelligence can make medical imaging information more approachable, but accessibility alone is not enough. A clear answer can still be wrong, incomplete, poorly timed, or used outside the context for which it was designed.

Responsible design begins by asking a governance question rather than a technology question: What role should this system be allowed to play in a patient's care journey?

The following five principles offer a practical framework for organisations developing or deploying tools that explain radiology reports or related imaging information. They are not a substitute for legal, regulatory, clinical, privacy, or security review. They are a starting point for aligning product decisions with patient safety and trustworthy communication.

1. Define the intended use—and the boundary around it

A product should state precisely what it does. “Uses AI for medical imaging” is not an adequate intended use. A safer description distinguishes among several very different functions:

• Explaining general radiology terminology;
• Identifying sections of an existing report;
• Restating report text in plain language;
• Generating questions for a clinical appointment;
• Analysing medical images;
• Detecting, diagnosing, or predicting disease;
• Recommending treatment or follow-up.

Moving from education toward diagnosis or treatment changes the risk profile and may change the applicable regulatory obligations. The FDA notes that it regulates medical devices based on intended use and technological characteristics; AI-enabled functions intended for diagnosis, treatment, mitigation, or prevention may fall within the medical-device framework (Artificial Intelligence-Enabled Medical Devices).

The intended use should be reflected consistently in marketing, interface copy, onboarding, output language, internal testing, and customer support. A disclaimer cannot correct a product whose main experience encourages users to treat educational output as a diagnosis.

Practical controls

• Write a one-sentence intended-use statement.
• List prohibited or unsupported uses.
• Avoid diagnostic claims unless the product is appropriately validated and authorised for them.
• Prevent marketing language from exceeding the evidence.
• Reassess the intended use whenever features or models change.

2. Preserve the meaning of the source

Plain-language output should simplify vocabulary, not alter clinical meaning. Radiology language contains details that must survive summarisation:

• Negation;
• Uncertainty and probability;
• Laterality and anatomical location;
• Measurements and units;
• Comparison with prior studies;
• Recommendations and time frames;
• Technical limitations;
• Distinction between observations and conclusions.

An explanation system should make the original source available and, where practical, trace each explanation back to the relevant sentence. Users should be able to tell whether a statement came from the radiologist's report or was generated by the system.

This design supports error detection. It also reduces the temptation to interpret an elegant summary as a replacement for the original medical record.

Practical controls

• Display source text alongside explanations.
• Use automated checks for negation, laterality, measurements, and dates.
• Prohibit unsupported additions.
• Preserve uncertainty rather than replacing it with definitive language.
• Escalate ambiguous or internally inconsistent text instead of guessing.

3. Keep humans responsible for clinical decisions

The World Health Organization's guidance on AI for health identifies protection of human autonomy, safety, transparency, accountability, inclusiveness, and sustainability as core principles. Humans should remain in control of healthcare systems and medical decisions (WHO Ethics and Governance of Artificial Intelligence for Health).

For patient-facing imaging tools, “human oversight” should be concrete. A generic instruction to “consult a doctor” is insufficient if the product simultaneously delivers a confident diagnosis or discourages follow-up.

The system should help the user identify the appropriate next human contact: the ordering clinician, radiologist, specialist, primary care professional, or emergency service. It should also explain that the responsible professional will interpret the report alongside history, symptoms, examination, and other tests.

Practical controls

• Do not independently initiate, stop, or modify treatment.
• Do not replace a report's follow-up recommendation.
• Provide clear pathways for professional review.
• Use symptom-based emergency guidance that does not depend on AI reassurance.
• Make it easy to report incorrect or harmful output.

4. Treat privacy and security as clinical safety issues

Imaging reports may contain names, dates of birth, medical record numbers, examination dates, diagnoses, and other sensitive information. An organisation should collect only what is necessary for the stated purpose and explain how information is processed.

Privacy communication should answer practical questions:

• What information is uploaded or entered?
• Is it stored, and for how long?
• Is it used to train or improve models?
• Which processors or service providers receive it?
• Where is it processed?
• Can a user delete it?
• How are access, encryption, incident response, and retention managed?

Requirements vary across jurisdictions and business models. Compliance should be assessed by qualified privacy and legal professionals; organisations should not assume that a consumer-facing disclaimer resolves health-data obligations.

Security also affects clinical trust. An altered report, mixed patient record, or unauthorised disclosure can cause harm even when the explanation model itself performs correctly.

Practical controls

• Minimise data collection and retention.
• Encourage removal of unnecessary identifiers where appropriate.
• Separate user accounts and patient records reliably.
• Encrypt data in transit and at rest where applicable.
• Document subprocessors and data flows.
• Test deletion, access control, audit logging, and incident response.

5. Evaluate continuously in the real context of use

Model performance in a controlled test set does not establish safety in every population or workflow. Report styles differ among institutions, modalities, countries, and radiologists. Patients differ in health literacy, language, disability, age, access to care, and emotional state.

The FDA, Health Canada, and MHRA transparency principles emphasize the performance of the human-AI team, essential information for users, context, and human-centered design. Transparency should communicate intended use, performance, limitations, and the information that can affect risks or patient outcomes (Transparency for Machine Learning-Enabled Medical Devices).

Evaluation should therefore include more than a single accuracy score. It should ask:

• Does the system preserve clinical meaning?
• Do patients understand uncertainty?
• Can users distinguish education from diagnosis?
• Does the output improve the questions asked in a clinical appointment?
• Are certain report types or patient groups more likely to receive poor explanations?
• Do warnings appear at the right time and in understandable language?
• What happens when input is incomplete, contradictory, or outside scope?

Practical controls

• Maintain representative test sets across report types and populations.
• Include radiologists, clinicians, patients, safety specialists, and accessibility experts in evaluation.
• Monitor real-world errors and near misses.
• Version models, prompts, policies, and evaluation results.
• Define thresholds for rollback or suspension.
• Revalidate after material system changes.

Governance must be visible in the product

Responsible AI is not achieved by publishing principles that users never encounter. Governance should appear in the experience itself:

• A clear statement of purpose before upload;
• Consent and privacy information in readable language;
• Original text beside generated explanations;
• Visible uncertainty and limitations;
• A clear separation between report content and AI-generated content;
• Easy access to professional follow-up;
• Mechanisms to correct errors and delete data.

These features may make a product feel less effortless than one that simply produces a definitive answer. That friction can be protective. In healthcare, confidence should follow evidence rather than substitute for it.

A checklist for healthcare leaders

Before adopting a patient-facing medical imaging AI tool, an organisation should be able to answer:

1. What exact task is the system intended to perform?
2. What tasks are explicitly outside its scope?
3. What evidence supports its performance for the intended users?
4. How does it preserve uncertainty, negation, laterality, and measurements?
5. Can users trace explanations to the original report?
6. Who remains accountable for clinical decisions?
7. How are privacy, retention, deletion, and security handled?
8. How are errors reported, investigated, and corrected?
9. How is performance monitored across populations and over time?
10. What conditions trigger escalation, rollback, or suspension?

If these questions cannot be answered clearly, deployment is premature.

From helpful language to trustworthy systems

The promise of patient-facing imaging AI is not that it can remove clinicians from the process. Its value lies in helping patients navigate a difficult information environment, recognise the limits of automated explanation, and participate more effectively in conversations about their care.

Trustworthy systems set a narrow purpose, preserve the source, keep humans responsible, protect sensitive information, and continue testing after launch. Those practices are not obstacles to innovation. They are what make useful innovation sustainable.

References

• World Health Organization. Ethics and Governance of Artificial Intelligence for Health.
• World Health Organization. WHO Calls for Safe and Ethical AI for Health.
• U.S. Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices.
• U.S. Food and Drug Administration, Health Canada, and MHRA. Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles.
• RadiologyInfo.org. All About Your Radiology Report: What to Know.