What agentic delivery changes in healthcare
The constraint in health software is not that changes are risky. It is that the data an agent would find most useful is the data it must never see, and that shapes the whole adoption path.
Every industry has a reason to be careful. Healthcare has a specific one that changes the shape of an agentic pipeline rather than just its pace: the material an agent would most benefit from reading is protected health information, and there is no framing under which casually sending it to a model provider is acceptable.
This is not the same as the risk conversation in payments, where the worry is what a change does. Here a large part of the worry is what the agent reads on the way to making the change, and that is a different problem with different fixes.
The exposure paths people miss
Ask a health software team whether agents touch patient data and the answer is no. Ask how their integration tests run and the picture usually changes.
The exposure is rarely a deliberate decision. It arrives through paths that were built years ago for good reasons and were never re-examined once tooling started reading everything continuously.
The remediation is unglamorous and mostly predates the problem. Synthetic test data. A staging environment that is not a production snapshot. Fixtures scanned for the patterns that matter. None of this is novel advice; what changed is that the cost of ignoring it moved from a latent risk to a continuous one, because tooling now reads these files all day rather than a developer reading them occasionally.
What HIPAA asks, without overclaiming
I want to be precise here, because this is exactly where vendor writing tends to promise things it cannot deliver. No tool makes an organisation HIPAA compliant. Compliance is a programme, assessed against your environment, involving policies and training and agreements that no software provides.
What is fair to say is that the rules ask questions an agentic pipeline has to be able to answer, and two of them get harder at volume.
The first is the minimum necessary principle: access to protected information should be limited to what is required. An agent with broad repository read access and a credential inherited from a senior engineer is a poor fit for that principle, and the fix is the least-privilege work described elsewhere in this series. Task-scoped, short-lived credentials are genuinely a better answer than the standing access a human holds, and this is one of the few places where the agentic setup can be stronger than the human baseline rather than a compensation for new risk.
The second is that a business associate handling protected information on your behalf needs an agreement in place. If a model provider processes anything covered, that is a question for your legal and privacy teams about the contract, not a question your engineering tooling answers. The honest engineering contribution is to make it unnecessary: if protected information never enters a prompt, the question does not arise for that path, and being able to demonstrate which endpoints were called with what scope is how you show it.
Clinical safety is a separate discipline
There is a category error worth naming, because it is common and it is dangerous.
Machine-checkable acceptance criteria are excellent at establishing that software does what a specification said. They say nothing about whether the specification was clinically correct. A dosage calculation that faithfully implements a stated formula is verified and can still be wrong in a way that harms someone, and no amount of criteria coverage catches it.
This matters more under agentic delivery for a reason that is easy to miss. When production was slow, clinical review was implicitly rate-limited by how much software existed to review. Remove that limit and the clinical safety function becomes the constraint, immediately, and it is usually staffed by a small number of people who are also doing several other jobs.
What criteria verify
- The code matches the specification
- Edge cases in the stated logic
- Behaviour under the conditions written down
- Nothing about clinical correctness
What clinical review verifies
- The specification is safe to implement
- The interaction is right for the workflow
- The failure mode is acceptable in a ward
- Rate-limited by people, not by tooling
Interoperability is where the real value sits
The most useful place to point agentic delivery in health software is also the least glamorous: the integration surface.
Health systems run an unusual amount of interface code. Message parsing, format translation, mapping one code system onto another, adapters for a hospital’s particular interpretation of a standard that was supposed to remove the need for particular interpretations. This work is voluminous, well-specified, tedious, and mostly not clinical. It is exactly the shape of work agents handle well.
It also has a natural verification story. Interface behaviour is testable against message fixtures, conformance is checkable, and the correctness question is usually “does this produce the right output for this input” rather than a clinical judgement. You can write criteria for it that genuinely settle the matter.
Where this breaks down
Synthetic data is harder than it sounds and the shortcuts defeat it. Realistic synthetic health data has to preserve the distributions and edge cases that make real data useful for testing, and teams that generate something too clean find their tests stop catching things. The usual compromise, a lightly obfuscated production extract, is not synthetic data and should not be described as such internally, because people will make decisions believing that it is.
De-identification is not a binary. Removing names and identifiers does not make a rare diagnosis in a small geography unidentifiable. Anyone treating a de-identified dataset as unconditionally safe to send outside the boundary should get advice from someone qualified rather than from an engineering blog.
A self-hosted model removes one question and not the discipline. Running inference inside your boundary eliminates the third-party processing question, which is real progress. It leaves the internal one: an agent is still assembling and moving sensitive material across trust boundaries inside your network, and least privilege still applies unchanged.
The clinical safety bottleneck has no tooling answer. Everything above routes around it rather than solving it. If your organisation’s constraint is that two clinicians review every patient-facing change, agentic delivery does not help with the thing that is actually limiting you, and pretending otherwise sets up a disappointing pilot.
And regulated medical device software is a different regime entirely. If your product is a device under a regulator’s software rules, the change control and documentation obligations are stricter than anything discussed here, and the relevant piece is the one on medtech rather than this one.
The takeaway
The distinguishing constraint in healthcare is not change risk, it is that the most useful context is the context an agent must not read. Fix the exposure paths first, because they were built years ago for good reasons and nobody re-examined them once tooling started reading continuously.
Point the throughput at interfaces and internal tooling, where the work is voluminous and the correctness question is answerable. Keep clinical judgement clearly separated from criteria verification, and expect the safety function to become your constraint the moment production stops being one.
If you take one thing into next week: find out whether your staging environment holds real patient records, and if it does, work out how many times tooling has read them this month. That number is currently unknown almost everywhere, and unlike most things in this domain it is straightforward to find out.