The AI medical scribe privacy storm: Why we need clinician-controlled solutions

A perspective on the hidden dangers of corporate-controlled medical AI

---

We’re being sold a dream. Major AI medical scribe platforms promise to liberate us from pyjama time documentation, reduce burnout and help us actually connect with our patients again. The stats look compelling, 40-50% improvement in documentation time, 60% reduction in cognitive burden. Who wouldn’t want that?

But here’s what keeps me up at night: we’re trading our professional autonomy and our patients’ privacy for convenience. And the price we’ll pay down the line could be catastrophic.

The privacy powder keg

Let’s be honest about what’s really happening here. Every conversation you have with a patient is being captured, processed and stored by corporations whose primary loyalty isn’t to you or your patients, it’s to their shareholders. Major tech companies have already exposed hundreds of millions of customer records due to “misconfigurations” and security breaches. Healthcare saw over 100 million records compromised in 2024 alone.

These aren’t isolated incidents. They’re previews of what’s coming when we centralise the most sensitive conversations in medicine into corporate-controlled AI medical scribe systems.

But the hacking risk is just the tip of the iceberg. Here’s what really terrifies me: we’re not talking about exposing dry medical notes anymore. These AI medical scribes are capturing raw, unfiltered human vulnerability, your patient’s voice breaking as they describe how their chronic pain has destroyed their marriage, the shame when they admit they’ve been avoiding activities with their children because of their back injury, the frustration and tears when they confess they avoided telling their employer about how the workplace accident really happened.

Traditional medical notes were our professional interpretation of these moments, filtered through medical language and clinical judgment. But AI medical scribes are recording the actual desperation in a construction worker’s voice when they admit they can’t do their job anymore, the fear of an athlete discussing how their injury might end their career, the embarrassment of someone explaining how their pelvic floor dysfunction affects their intimate relationships.

What happens when insurance companies come knocking with subpoenas for that AI medical scribe data? When malpractice attorneys realise these systems capture everything, not just the sanitised notes we’ve traditionally written, but the raw emotional content, the actual hesitations, the diagnostic uncertainty, the off-hand comments that are a part of normal clinical conversations?

Picture this: An insurance adjuster reviewing your patient’s AI medical scribe transcript, listening to the actual recording of your patient honestly describing how their knee injury has been affecting them more lately, then using that candid discussion to reclassify them as “high-risk” and dramatically increase their health insurance premiums.

Or imagine a divorce attorney subpoenaing AI medical scribe records where your patient discussed how their back injury sometimes requires help lifting their child. Their honest admission about needing assistance, shared in the safety of your clinic, now weaponised to argue they’re an unfit parent in custody proceedings.

Meanwhile, hospital administration gains a powerful new weapon: when malpractice claims arise, they can comb through your AI medical scribe records looking for any deviation from company protocols to shift liability onto you personally. Did you skip mentioning a particular risk factor? Did you use slightly different wording than the approved patient education script? That’s all documented now.

Your career progression could depend on AI analysis of your consultation style, how closely you follow institutional guidelines, how efficiently you move through appointments, whether your patient interactions align with corporate communication standards. Recruitment, redundancies and promotions now informed by algorithmic review of every clinical conversation you’ve ever had.

The corporate surveillance problem

Here’s what really bothers me about the current AI medical scribe rollout: we’re not the customers, we’re the product. But more chilling is what the product actually contains. These systems aren’t just documenting medical facts; they’re creating comprehensive psychological profiles of our most vulnerable patients.

Every time a patient shares how their chronic pain has affected their relationships, discusses their fears about functional limitations, talks about how their condition impacts their confidence and personal life, or admits they sometimes need help with daily activities, that’s all captured in corporate databases. The AI doesn’t just hear “patient reports functional limitations”, it captures the actual voice breaking when they describe needing help from family members, the embarrassment when they explain how their pelvic floor dysfunction affects their relationships, the exact words they use when they’re worried about their physical capabilities as a parent. Every clinical decision, every patient interaction, every diagnostic pathway becomes training data for algorithms we don’t control.

These platforms aren’t just selling us convenience. They’re building comprehensive profiles of how physiotherapy is practiced, which they can then monetise in ways we can’t even imagine yet. Today it’s documentation. Tomorrow it’s “clinical decision support” that replaces clinical reasoning with algorithmic protocols, enabling healthcare systems to deskill our roles and automate away significant portions of what we do. Why pay for experienced clinicians when an AI can generate standardised treatment plans based on presenting symptoms? Why invest in clinical expertise when algorithms can guide junior staff through cookie-cutter care pathways?

What we actually need: clinician-controlled AI

Don’t get me wrong, I’m not anti-AI. The technology itself is remarkable and we desperately need help with documentation and clinical decision making. But we need solutions that put us, not corporations, in the driver’s seat.

Imagine instead: AI assistants that run locally on our devices or on platforms we can trust. Systems where we decide what gets recorded, what gets shared and with whom. Where patients can opt in or out of specific features without it affecting their care. Where the AI serves us and our patients, not some distant corporate board.

This isn’t just about privacy, it’s about professional autonomy. When major tech platforms or AI medical scribe companies control the AI that documents our practice, they effectively control how we practice physiotherapy. The algorithms they build will inevitably shape how we think about diagnosis, treatment and patient care.

The path forward

We need to demand better. We need to define strict standards for how medical AI systems gather and handle our data. We need solutions that:

• Give us granular control over what gets recorded and shared
• Allow patients to participate in deciding how their conversations are processed
• Keep the AI’s insights between us and our patients, not feeding corporate databases
• Provide transparency about how the AI makes its decisions

The current path could lead to a future where insurance companies have actual recordings of patients admitting they’re not following treatment protocols, where employers can subpoena AI medical scribe data to challenge workers’ compensation claims based on honest admissions about pre-existing conditions, where lawyers use patient recordings of frustration and despair as evidence of malingering and where our treatment rooms, once safe spaces for honest discussion about pain and function, become surveilled environments where patients hide their true struggles.

The bottom line

We got into physiotherapy to care for patients, not to feed corporate surveillance systems. The burnout crisis is real and we do need technological solutions. But we can’t let desperation drive us into digital contracts that compromise our professional judgment and our patients’ privacy.

Before you sign on for that AI medical scribe platform, ask yourself: Who really benefits from this arrangement? And what happens to all that data when the next healthcare cyberattack hits, or when the insurance companies come calling?.

---

A different path forward

While writing this piece, I kept thinking: what would clinician-controlled AI actually look like in practice? We’ve been working on that question by building PAI, a physiotherapy AI assistant from the team that brings you Physiopedia. Instead of listening to your patient conversations, you dictate questions and information to it when you choose. Need help with a clinical topic? Ask PAI. Want to explore the evidence base? No problem. You control exactly what information gets shared.

It’s not as convenient as an automated scribe but it represents something important: AI built by clinicians, for clinicians, without venture capital or corporate surveillance. You own your data, you control the interactions and your patients’ private moments stay private.

If you’re interested in trying a different approach, one where you remain in the driver’s seat, you can learn more at the PAI home page. Because the future of AI in healthcare doesn’t have to be one where we trade our patients’ safety and our professional autonomy for convenience.

Photo by NOAA on Unsplash