AI Data Privacy: Requirements for Internal AI Tools
Most privacy plans for an internal AI tool focus on the model provider and stop there. The provider is one of at least five places your data rests. The others are your own logs, your vector store, your masking map, and the files and threads you attach to the API. Each one can keep a copy after the one you planned for is gone.
This guide lists those places, shows what masking can and cannot do, and describes logging and access choices that do not create a second leak.
Map every place the data rests
Start with a data-flow sketch for one real prompt, from the employee's keyboard to the last log line. Mark each store that holds some form of it.
Provider logs. OpenAI keeps API abuse-monitoring logs for up to 30 days unless the law requires longer, per its data controls documentation. Zero data retention removes that window for eligible endpoints.
Provider-side application state. Zero data retention is per endpoint. In the same document, chat completions, responses, and embeddings are eligible, while conversations, threads, assistants, files, and vector stores are not. Files and vector stores are kept until deleted, and objects in the Assistants API that nobody deletes are retained indefinitely. A team can sign a zero-retention agreement and still leave customer documents in a hosted vector store. Check each endpoint you call against the vendor's eligibility table before you tell anyone the data is not retained.
Your own logs and caches. Zero retention at the provider says nothing about your application logs, your tracing vendor, or your error tracker. Prompts leak into all three by default.
Your vector store. Embeddings of sensitive chunks are sensitive. They need the same access rules and deletion path as the source documents.
Your masking map. The table that maps tokens back to real values holds exactly what you tried to keep from the model. Treat it as the most sensitive store in the system.
Choose hosting by who can see the logs
Private endpoints keep traffic off the public internet, which helps compliance reporting. They do not change who can read the prompt after it arrives. The better question for a hosting choice is which humans at which company have any access to prompts and logs.
AWS documents one useful answer for Bedrock. Models run in deployment accounts that AWS operates in each region, and because the model providers have no access to those accounts, they have none to Bedrock logs or customer prompts, per the Bedrock data protection guide. That removes the model vendor as a reader. It does not remove the cloud provider, so the contract question moves rather than disappears.
Self-hosting an open-weight model removes the third-party reader entirely, at the cost of running GPUs, patching serving software, and owning model quality yourself. Choose it when a regulator or contract demands that no outside party sees the text. For most teams, a managed private endpoint with a signed data processing agreement is the cheaper route to a defensible position. Settle the vendor questions on paper before launch: no training on your data, encryption in transit and at rest, a named retention window, and the latest SOC 2 Type II report. A vendor that will not put these in the agreement should not handle sensitive workloads.
What masking can and cannot do
Masking replaces entities such as names, emails, and account numbers with tokens before the prompt leaves your boundary, then restores them in the answer. It cuts exposure and is worth building. It does not take the data out of privacy law.
GDPR Recital 26 says personal data that has undergone pseudonymisation, and could be attributed to a person with additional information, "should be considered to be information on an identifiable natural person." Your token map is that additional information. A masked prompt remains personal data for as long as you hold the key, so retention limits, access requests, and deletion duties still apply to it. The recital is in the GDPR text on EUR-Lex.
Masking also fails in two practical ways.
- Context re-identifies. "[PERSON] the CFO of [COMPANY] resigned on [DATE]" is masked and still points at one human in a small market. Mask quasi-identifiers (title, rare event, small-group attributes) only if the task allows it, and accept that some cannot be masked.
- Some tasks need the entity. A tool that drafts a reply to a named customer cannot work from "[PERSON]". Mask per field and per task, not per prompt. Blanket masking pushes users to paste raw text into an unmasked tool.
Mask names for a document summary, keep the name for the greeting, and say which in the tool catalog so a reviewer can see the choice.
Custom entities need custom rules. A standard entity model does not know your product's code names, so give the masking layer a dictionary of internal terms and review misses monthly.
Log shape, not content
Common advice says to log the original prompt, the masked prompt, and the response so you can audit later. That creates an unmasked copy of everything sensitive in a log system with a wider audience and a longer life than the tool itself. You have rebuilt the leak you were guarding against.
Log the shape of a call instead, and assume anything you keep can be demanded. In May 2025 a federal magistrate in the New York Times case against OpenAI ordered it to "preserve and segregate all output log data" that would otherwise be deleted. As Loeb & Loeb summarized, the order applied regardless of whether OpenAI had committed to delete the data, and the firm advised enterprise and API customers with contractual deletion terms to review their contracts. A deletion promise from a vendor, or from your own retention policy, holds only until someone with legal authority says otherwise. So record which tool ran, which arguments it took and how large each one was, replacing every string argument with its length (something like [withheld: 212 chars]) and redacting values under sensitive keys and anything that passes a card-number check. The content then never reaches the log store, and there is nothing in it to preserve or produce.
Log the user, tool, resource, decision, token counts, and duration. When an incident needs content, re-read it from the source system under the investigator's own permissions.
Respect permissions at retrieval time
A retrieval-augmented tool that ignores document permissions turns every employee into someone who can read everything. Authenticate the user with your identity provider, resolve their groups, and filter the vector search by those groups before ranking.
# Pseudocode for permission-filtered vector search
user_groups = get_user_groups(current_user)
relevant_chunks = vector_db.search(
query=user_prompt,
filter={"allowed_groups": {"$in": user_groups}},
top_k=5,
)Two failure modes sit behind that snippet. Filter before ranking, not after: if you take the top five chunks and then drop the ones the user cannot see, you return fewer than five and bias answers toward whatever few were visible. And group tags copied into chunks at ingestion go stale when someone changes teams. Re-sync access lists on a schedule shorter than your revocation promise, or check the source system at read time for high-risk collections.
For very sensitive data sets, add just-in-time access: a person requests a data set, a policy or an approver grants it for a fixed window such as two hours, and it expires. Watch for access patterns that break a baseline, such as a marketing account pulling thousands of architecture documents through the tool.
Fail closed
If the masking service is down or the permissions database is unreachable, the tool should refuse the request with a plain message. A temporary outage costs a few minutes. A fail-open path sends unmasked text to a provider and cannot be recalled.
The same rule limits fallbacks. A secondary model for availability must meet the same data requirements as the primary. A cheap fallback on a different vendor with weaker terms can fall outside the agreement you negotiated, and it will be used the first time the primary is slow.
Add a person for actions with consequences
For actions that change data, call external systems, or message customers, have the tool show the proposed action with its context and wait for explicit approval. Present the sensitive fields clearly in the approval card, so the reviewer sees what is about to leave the building.
A first-week checklist
- Trace one real prompt end to end and list every store that holds a copy.
- Check each provider endpoint you use against the zero-retention eligibility table.
- Decide hosting by who may read prompts and logs, then put the answer in the data processing agreement.
- Mask per field and per task, and keep the token map under the strictest access controls you have.
- Change logging to record shape and size, with content stored only where a business need and an expiry exist.
- Filter retrieval by the user's groups before ranking, and set a re-sync interval for access lists.
- Make every privacy control fail closed, including fallbacks.
References
- OpenAI API data controls documents the 30-day abuse-monitoring retention and which endpoints are eligible for zero data retention.
- Amazon Bedrock data protection explains why model providers cannot read Bedrock logs or prompts.
- General Data Protection Regulation (EUR-Lex), Recital 26, treats pseudonymised data as personal data when it can be attributed with additional information.
- Loeb & Loeb, Court orders OpenAI to retain all output log data for the May 2025 preservation order and its effect on deletion commitments.