Compliance · HIPAA
AI observability for HIPAA compliance
When a clinical LLM reads a discharge note, the prompt contains PHI, and so does every log of it. Here is what the Security and Privacy Rules expect of AI telemetry, and how to keep it useful without creating a new disclosure risk.
Last reviewed by the Perimattic AI Suite team
In short
What does HIPAA require from AI observability?
If an AI system creates, receives or stores electronic PHI, the HIPAA Security Rule requires audit controls that record and examine activity in it (45 CFR 164.312(b)), access controls, and integrity protections. The Privacy Rule’s minimum-necessary standard limits how much PHI reaches prompts and logs. Any vendor that stores those logs is a business associate and needs a business associate agreement.
Logs are part of the system
Traces of a clinical LLM are electronic PHI if prompts or outputs contain patient details. They need the same safeguards as the EHR data they came from.
Redact before you store
Removing identifiers at collection, before telemetry leaves your environment, keeps traces useful for debugging while shrinking what an incident could expose.
Regulation overview
What HIPAA requires
HIPAA protects identifiable health information held by covered entities (providers, health plans, clearinghouses) and their business associates. The Security Rule sets administrative, physical and technical safeguards for electronic PHI. The Privacy Rule limits uses and disclosures. The Breach Notification Rule sets what happens when PHI is exposed.
- Authority
- US Department of Health and Human Services, Office for Civil Rights
- Legal basis
- Health Insurance Portability and Accountability Act of 1996, the HITECH Act of 2009, and the Privacy, Security and Breach Notification Rules (45 CFR Parts 160 and 164)
- Penalties
- Tiered civil money penalties by level of culpability, adjusted for inflation each year, with an annual cap per type of violation that now exceeds $2 million. Criminal penalties are enforced by the Department of Justice.
Capabilities
Observability built around PHI
Redaction at collection
Identifier patterns can be stripped from prompts, retrieved context and outputs in your OpenTelemetry Collector, before spans leave your network. Rules and defaults are agreed with your privacy team in the security review.
Signal it produces: Redaction rate per field, redaction misses flagged
Audit trail for every AI request
Who called the model, from which application, with what retrieved records and what came back, linked by one request ID.
Signal it produces: Activity record for 164.312(b) reviews
Access records for the telemetry itself
Who viewed or exported traces, and when, so the observability tool does not become an unmonitored copy of PHI.
Signal it produces: Access log for trace data
Clinical quality monitoring
Faithfulness and hallucination scores on live traffic, so a model that misstates medications or results is caught early.
Signal it produces: Faithfulness score, flagged answers for review
Requirement to evidence
Which records each HIPAA requirement expects
References are to 45 CFR Part 164. The right-hand column is the record you can show an auditor or OCR investigator.
| HIPAA requirement | Telemetry that supports it | Evidence you can produce |
|---|---|---|
| 164.312(b) Audit controls | Request-level traces of every AI call that touches ePHI | Activity log you can search by patient workflow, user or period |
| 164.312(a)(1) Access control | User and service identity recorded on each span; access to traces limited by role | Who used the AI system and who viewed its logs |
| 164.312(c)(1) Integrity | Integrity controls on stored traces, such as immutable storage, agreed in your security review | Proof that logs were not altered after the fact |
| 164.502(b) Minimum necessary | Redaction rules applied before export; fields captured are configurable | Documented redaction policy and measured redaction coverage |
| 164.308(a)(1) Security management process | Risk signals such as unexpected PHI in outputs or unusual access patterns | Inputs to your periodic risk analysis |
| 164.502(e) and 164.504(e) Business associates | Deployment model where PHI stays in your environment, or a vendor under BAA | Data-flow record for your BAA inventory |
Sample evidence
What a HIPAA audit record looks like
One record per AI request, with identifiers already redacted and the access trail attached.
Perimattic AI Suite supports your HIPAA compliance programme. It is not legal advice. Hosting, redaction defaults and any business associate agreement are agreed in your security review.
{
"record_type": "hipaa.164_312_b.activity",
"service": "discharge-summary-assistant",
"request_id": "req_8d21…",
"user": "<clinician-role>",
"phi_redaction": { "applied": true, "fields_redacted": 6 },
"eval": { "faithfulness": 0.94 },
"trace_viewed_by": ["<privacy-officer>"]
}Regulation details last verified on 1 October 2026.
FAQ
Common questions
Short answers to common questions. They are general information, not legal advice.
Does a clinical LLM have to comply with HIPAA?
If it is operated by or for a covered entity and processes PHI, yes. A model that summarises discharge notes, answers patients about their care or reads lab results is part of an information system containing ePHI, so the Security Rule safeguards apply to it and to its logs.
Is sending PHI to an observability vendor a HIPAA problem?
It is allowed only if the vendor is under a business associate agreement and the disclosure meets the minimum-necessary standard. Many teams avoid the question by redacting identifiers before telemetry leaves their environment, or by hosting the observability backend where their PHI already lives.
What counts as de-identified data under HIPAA?
HIPAA offers two methods: Safe Harbor, which removes 18 listed identifiers (names, most dates, contact details, record numbers and others) with no actual knowledge that the rest could identify someone, and Expert Determination by a qualified statistician. Redaction for logging often follows the Safe Harbor list.
Do AI audit logs satisfy 164.312(b)?
They can be a large part of it. The standard asks for mechanisms that record and examine activity in systems that contain or use ePHI. Request-level traces cover the recording; you still need a process for reviewing them, which the alerts and reports in an observability tool support.
Can hallucinations create HIPAA exposure?
A hallucination is mainly a patient-safety problem. It becomes a privacy problem when a model surfaces another patient’s details or sends PHI somewhere it shouldn’t. Monitoring output quality and scanning outputs for unexpected identifiers covers both risks.
Which other frameworks apply to healthcare AI?
Often SOC 2, which healthcare customers ask their vendors for, and the EU AI Act if the system is used in the EU. The compliance hub lists how Perimattic AI Suite supports each one.
Go further
Related tools, guides and services
- Free toolAI compliance readiness assessmentScore your HIPAA, EU AI Act, GDPR, SOC 2 and DORA readiness for AI systems.Open
- Article10 generative AI use cases in healthcareWhere generative AI is being used in clinical and administrative work today.Open
- White papersAI in healthcare white papersResearch on clinical AI adoption, safety and governance.Open
- IndustryAI for healthcareHow Perimattic builds AI for providers, payers and health-tech teams.Open
- Case studyAI lung disease detection for a healthcare providerA regulated clinical AI system Perimattic built and put into production.Open
See what your AI systems are doing, with evidence to back it up
Perimattic AI Suite is in early access. Tell us what you are building and a Perimattic engineer will follow up to scope your first instrumented system.
Prefer email? sales@perimattic.com