Risks & safety
Data Privacy (in AI)
Also known as: AI and privacy
Protecting personal and confidential information when it is used with AI, both what goes into models and what they might reveal.
Data privacy with AI is about keeping personal and confidential information safe when you use these tools. Two worries: what you type into an AI might be stored or used to train it, and a model trained on lots of data might accidentally reveal private details it learned. For anyone handling customer data, contracts or health information, being careful about what you share with AI, and which service you use, is essential.
AI raises data-privacy concerns at several points: - Input: what you send to an AI service may be logged or used for training, depending on the provider's terms, a real issue for confidential business or personal data. - Training data: models trained on large datasets can memorise and, in some cases, regurgitate private information. - Inference: RAG systems that connect AI to your documents must control who can access what.
Good practice
- Know the provider's data policy (retention, training use, enterprise options).
- Avoid pasting sensitive data into consumer tools; use options that keep data private or on-device (small models).
- Apply access controls and, where needed, anonymisation.
It sits within AI ethics and legal frameworks such as GDPR.
AI data privacy spans the full lifecycle: training-data provenance and consent, memorisation/leakage (models can emit verbatim training data, including PII, under extraction attacks; Carlini et al. 2021), membership-inference risks, inference-time data handling (logging, training-on-inputs), and access control in retrieval systems. Legal regimes (GDPR and equivalents) impose lawful-basis, purpose-limitation, minimisation, and data-subject rights (including erasure, which is technically hard once data is baked into weights).
Mitigations include data minimisation and PII scrubbing, differential privacy in training, deduplication (which reduces memorisation), access-controlled and tenant-isolated RAG, on-prem/on-device deployment, encryption, and contractual/enterprise "no-training" guarantees. Fine-tuning on sensitive data concentrates leakage risk. Emerging techniques (machine unlearning, privacy-preserving training) remain immature, so governance, controlling what data enters models and systems in the first place, is the primary safeguard, closely tied to AI ethics and security.