What Is Private AI?

Enterprises confront many challenges in bringing AI to mainstream applications. There’s the fundamental architecture required to run AI workloads efficiently—compute, storage, and networking all differ from legacy centralized client-server designs in AI networks. And power and cooling of the physical environment remain a challenge.

But topping the list of requirements for enterprise AI is data security. How can a firm adapt a popular large language model (LLM) for internal use while ensuring corporate data remains safe from competitors? How can personally identifiable information (PII) stay secret? And how will data governed by regulations suchas the Health Insurance Portability and Accountability Act (HIPAA) or Europe's General Data Protection Regulation (GDPR) remain compliant within an AI app?

These questions point to private AI, which is gaining in popularity as a specific architecture for keeping data safe while ensuring it can be used in generative AI (GenAI). In this tech primer, we take a detailed look at the design of private AI infrastructure.

Basic Tenets of Private AI

At its most basic, the private AI approach centers on the use of corporate data. To create AI applications, companies must fine-tune models with corporate or domain-specific data and deploy those models for inferencing with corporate data—all while ensuring the protection and integrity of all data used.

To meet these goals, the data used for AI must meet the following criteria:

• Data must be under the total control of the organization.
• Data accessibility must be on a zero-trust basis.
• PII and corporate secrets must be protected in AI workloads.
• Data must reside close to the AI infrastructure.

Let’s take a closer look at each of these requirements.

Keeping Data Under Control

Private AI is based on the concept that proprietary data will be deployed to fine-tune a chosen model. This data must be protected both at its source as well as within the AI workload.

Enterprises may prefer to keep the data used for AI on premises, for instance, to ensure ultimate control. Alternatively, datacenter service providers such as Equinix offer dedicated facilities and bare metal “as-a-service” in order to fully isolate company data from that of other customers.

Technology vendors offer a variety of takes on AI data control. VMware, for instance, has worked with NVIDIA to offer VMware Cloud Foundation with NVIDIA, which combines VMware’s environment, including zero trust security and threat hunting, with managing NVIDIA GPU instances in virtual machines.

Keeping data safe doesn’t mean restricting models for use with only private data. Retrieval-augmented generation (RAG) allows models to be enriched with public data from sources such as social media, while ensuring that the private data used for inferencing remains protected.

Zero Trust a Must

Related to the issue of data control is data protection via zero trust, in which access to any data is restricted to all except qualified users—whose identity then permits access only to certain portions of data.

“As organizations transition to decentralized and dynamic network architectures, the traditional perimeter based security approach becomes outdated and ineffective,” wrote Christopher Hong, IT Support, Project Manager, with Ang Mo Kio Town Council in Singapore. “Zero-trust security per se provides a more adaptive and agile framework to secure digital assets, where an organization assumes trust is never granted and security controls are always enforced.”

CDN vendor Cloudflare uses a range of tools to ensure customer data is protected at all levels via a zero trust approach. The company deploys browser isolation (the technique of keeping Internet data off a corporate browser) to keep applications that use models such as ChatGPT from leaking data to the public Internet.

At least one customer has found this approach useful. “We wanted to let employees take advantage of AI while keeping it safe,” said Tanner Randolph, CISO at Applied Systems, which provides software-as-a-service (SaaS) to the insurance industry. “We prevent users from copying and pasting sensitive data from other apps into our isolated instance of ChatGPT, which prevents a lot of company info from being exposed to the tool.”

AI Serves Zero Trust

Notably, zero trust techniques can benefit from the application of AI, according to both Christopher Hong and Chris Hogan, VP, Enterprise Security Architecture and Innovation at Mastercard. In a blog post, Hogan cited AI’s potential use in behavior analytics that identify normal user behavior, as well as in automating responses to threats and in adaptive access, or denying or admitting access to users based on real-time observation. Hogan wrote:

“The start of many Zero Trust journeys may have been constrained by the limitations of available technologies in the past. However, the stage has dramatically shifted with the sudden proliferation of AI, unveiling a host of capabilities far exceeding expectations. Within this new but not-yet-defined landscape, it's plausible to envision many organizations attaining an optimal posture for Zero Trust.”

Protecting PII and Secrets

Vendors offer a mix of solutions to protect PII and corporate data in AI workloads, including end-to-end data encryption; multiparty computation (MPC, which allows multiple users to share data without revealing all their own data); and regular monitoring and auditing.

A startup called Private AI, based in Toronto, specializes in redacting or replacing PII in over 50 languages. It’s being used to remove PII and confidential company data from unstructured data sets, including those used with ChatGPT.

Another startup, Dasera, based in Mountain View, Calif., provides a platform for data security posture management (DSPM) that encompasses PII. By automatically finding, tagging, and tracking PII in corporate data used for AI inferencing, Dasera claims to reduce the security risks associated with PII.

Keeping AI Data Close

Another tactic for managing private AI is maintaining AI data close to the AI infrastructure on which is it used for inferencing. This approach reduces latency and can improve security, since it restricts the processing of data to isolated locations that can be more easily tracked than multiple distributed sites. AI inferencing at the edge also allows for more efficient use of corporate bandwidth.

In conclusion, private AI is a design approach geared to ensuring the safety and integrity of enterprise data used in GenAI inferencing. The need to provide secure and limited access to data, to monitor its use, and to provide threat remediation will become crucial as companies use their data to inform AI applications.