Running an open model locally tends to make AI privacy issues easier because sensitive data is kept entirely inside infrastructure you control.

When any sort of compliance is involved, I tend to work out exactly what data an application will handle, where that data can travel, who can access it, and what gets logged. I try to minimize sensitive data wherever possible. Then I treat the AI component like any other potentially sensitive service: restrict network access, authenticate it, encrypt traffic, isolate it from things it doesn’t need, and give it the minimum permissions necessary.

To deal with uptime requirements, I would have backup systems, tested restore procedures, monitoring, audit logs, patch/update procedures, access controls, and some plan for failure. If uptime really matters, that may mean redundant servers or the ability to fail over to another machine. That's true whether the application uses an open model, a proprietary model, or no AI at all.

For regulated work, I normally think in terms of being able to demonstrate controls rather than planning to use some specific tech solution. For example:

• Where is the data stored?
• Is it encrypted in transit and at rest?
• Who has access?
• Can access be revoked?
• Are important actions logged?
• Are backups encrypted and tested?
• What happens when a server fails?
• What external services receive data?
• What software/dependencies need to be patched?
• What is the incident-response procedure?
• Can we document all of those answers?

A self-hosted open model provides tighter control over where the information goes. The harder part is usually all the infrastructure around it: the web application, database, operating system, authentication, remote access, backups, logs, employees/admins, third-party integrations, operational procedures, etc.

And different compliance standards (HIPAA, PCI, SOC 2, GDPR, contractual confidentiality requirements, etc.) all impose different obligations. So there's no generic checklist. Design architecture around particular requirements and have the appropriate compliance/legal people validate it.

Minimize accessible data, minimize trust, minimize access, know every place the data can go, make failures recoverable, log what matters, and document the controls. Configure your LLM provider and your agentic software with those same disciplines.