Data Privacy in an AI-First Product: GDPR, CCPA, and the New AI Act

A founder asked me last month whether sending customer data to the OpenAI API counted as "processing" under GDPR. The answer is yes, and the implications are not trivial. When your product passes customer text through an LLM for summarization, classification, or generation, that text is being processed by a third-party sub-processor. Your privacy policy needs to disclose it. Your data processing agreements need to cover it. Your data subject access requests need to account for it. And depending on the data, your risk assessment needs to evaluate it.

Most startups building AI features are not thinking about any of this. They are focused on making the feature work and will deal with compliance later. The problem is that "later" sometimes means after a customer with a legal team asks the question, and at that point the answer is expensive.

This is not legal advice — I am a CTO, not a lawyer — but I have worked through the privacy implications of AI features with enough clients and their counsel to know where the common gaps are and what the practical mitigations look like. This article covers the three regulatory frameworks that matter most and the specific obligations they create for AI-powered products.

GDPR and AI features

GDPR applies to any product that processes personal data of EU residents. If your product has EU users and sends their data through an AI model, GDPR applies.

Lawful basis for processing. You need a lawful basis for sending personal data to an LLM. For most SaaS products, the basis is either contract performance (the AI feature is part of the service the user signed up for) or legitimate interest (the AI feature improves the service). Consent is another option but is harder to manage because it can be withdrawn.

Sub-processor obligations. When you send data to OpenAI, Anthropic, or Google for inference, that provider is a sub-processor under GDPR. You need a Data Processing Agreement (DPA) with the provider. All major model providers offer DPAs, but you need to actually sign them — which many startups have not done.

Data minimization. GDPR requires that you process only the data necessary for the purpose. If your AI feature summarizes customer support tickets, you should send only the ticket text, not the full customer profile. Strip PII that is not needed for the task before sending it to the model.

Data subject rights. Under GDPR, individuals have the right to access, correct, and delete their personal data. If customer data is cached in your RAG pipeline, stored in fine-tuning datasets, or retained in prompt logs, you need to be able to find and delete it on request. This is the practical challenge most teams miss — your data deletion process needs to reach into every layer of the AI pipeline.

Data transfer requirements. If your model provider processes data outside the EU, you need adequate transfer mechanisms (Standard Contractual Clauses or equivalent). Most US-based model providers handle this through their DPA, but verify.

Data Protection Impact Assessment (DPIA). For AI processing that involves profiling, automated decision-making, or large-scale processing of sensitive data, GDPR requires a DPIA. Many AI features cross this threshold and the DPIA has not been done.

CCPA and AI features

CCPA applies to businesses that collect personal information of California residents and meet certain revenue or data volume thresholds. The obligations overlap with GDPR but differ in the details.

Disclosure. CCPA requires that your privacy policy disclose the categories of personal information collected and the purposes for which it is used. If customer data flows through an AI feature, that use case must be disclosed.

Opt-out rights. CCPA gives consumers the right to opt out of the sale or sharing of their personal information. If your AI feature involves sharing data with a third-party model provider, this may constitute "sharing" under CCPA. The practical implication: you may need a mechanism for users to use your product without the AI feature if they exercise their opt-out right.

Service provider agreements. Similar to GDPR's sub-processor requirement, CCPA requires service provider agreements with any third party that processes personal information on your behalf. The model provider's DPA typically satisfies this.

The EU AI Act

The EU AI Act came into force with a phased implementation schedule. Unlike GDPR and CCPA, which focus on data privacy, the AI Act focuses on the AI system itself — its risks, its transparency, and its governance.

Risk classification. The AI Act classifies AI systems into risk categories: unacceptable (banned), high-risk (heavily regulated), limited risk (transparency obligations), and minimal risk (largely unregulated). Most SaaS AI features — summarization, classification, generation, recommendations — fall into the limited or minimal risk category.

Transparency obligations. For AI features in the limited risk category, the main obligation is transparency: users must be informed that they are interacting with an AI system. If your chatbot is AI-powered, the user should know it. If your summary was AI-generated, the user should know it.

High-risk systems. If your AI feature is used for decisions that significantly affect individuals — employment decisions, credit scoring, access to essential services — it may be classified as high-risk. High-risk systems require conformity assessments, risk management systems, human oversight, and technical documentation. This is a substantial compliance burden.

General-purpose AI models. The AI Act also imposes obligations on providers of general-purpose AI models (OpenAI, Anthropic, Google). As a downstream user of these models, you benefit from the provider's compliance but may need to perform your own risk assessment for your specific use case.

The practical checklist

For startups building AI features, the checklist I walk through with clients and their counsel:

Sign the model provider's DPA. This takes 15 minutes and covers the sub-processor and service provider requirements for GDPR and CCPA. Do it today if you have not.

Update your privacy policy. Disclose that customer data is processed by AI services, name the categories of data involved, and describe the purpose. Your users should not be surprised to learn that their data touches an LLM.

Implement data minimization. Before sending data to the model, strip fields that are not needed for the task. If the AI feature does not need the user's email address, do not send it.

Map your AI data flow. Document where customer data goes in the AI pipeline: prompt construction, model inference, response caching, logging, fine-tuning datasets, RAG indices. Every stop in the pipeline is a point where data subject access and deletion requests must reach.

Build a deletion mechanism. When a customer requests deletion, your process must cover the AI pipeline — delete their data from prompt logs, RAG indices, cached responses, and any fine-tuning datasets.

Add AI transparency labels. In the product UI, indicate where content is AI-generated. "This summary was generated by AI" is sufficient. This satisfies the EU AI Act's transparency obligation and is good practice regardless.

Assess your risk category. If your AI feature makes decisions that affect individuals, consult your lawyer about high-risk classification under the EU AI Act.

Counterpoint: do not let compliance paralyze shipping

A warning. The regulatory landscape is complex, but it should not prevent you from building AI features. The obligations are manageable for most SaaS use cases. Sign the DPA, update the privacy policy, implement data minimization, and build the deletion mechanism. That covers the vast majority of the compliance surface. The advanced obligations (DPIAs, high-risk conformity assessments) apply to a minority of use cases. Ship the feature, do the basics right, and escalate to legal counsel for the edge cases.

Your next step

This week, check whether you have signed your model provider's DPA. If you are using OpenAI, check your account settings. If you are using Anthropic or Google, check the same. If you have not signed it, sign it. That single action closes the largest compliance gap most startups have.

Where I come in

I am not a lawyer, but I regularly work with clients and their counsel to map the technical architecture of AI features against privacy and compliance requirements. The CTO's job is to understand the data flow and implement the technical controls. Book a call if your AI features are live and you have not yet done the privacy assessment.