A New Benchmark for Evaluating AI Assistants' Privacy Protection
Google researchers created a tool to test how well AI assistants protect personal data. They use synthetic data to check how AI understands context and privacy rules in real-world situations.
As AI systems become increasingly adept at handling personal data to perform tasks such as scheduling, composing emails, or making purchases, privacy has emerged as a significant concern. the potential for misuse or unintentional leaks of sensitive information also grows.
To tackle this, researchers from Google have developed CI-Bench, a benchmarking tool designed to evaluate how well AI systems respect privacy during information exchanges.
What is CI-Bench?
It stands for Contextual Integrity Benchmark, inspired by the theory of contextual integrity, which defines privacy as appropriate information flow depending on the situation.
This benchmark helps measure an AI assistant’s ability to navigate complex privacy norms by testing its understanding of contexts such as who is sharing information, what type of information is shared, and under what conditions. For instance, when an AI assistant helps book a medical appointment, it must decide if sharing sensitive health information is appropriate.
CI-Bench systematically evaluates whether the AI assistant can make these decisions correctly by simulating realistic communication scenarios, like dialogues and emails. These simulated interactions provide a rigorous test for assessing how AI handles personal data across various domains, including healthcare, finance, and e-commerce.
To create test cases, the researchers built a synthetic data pipeline, which produces natural dialogues and emails that mimic real-world interactions. Unlike earlier benchmarks that often focused on narrow contexts or small datasets, CI-Bench's diversity and scale offer a more comprehensive understanding of AI's privacy performance.
When tested on existing AI models, including those built on the Gemini language model, CI-Bench revealed critical gaps. Smaller models struggled with understanding context and often failed to distinguish between sensitive and non-sensitive information, leading to inappropriate data sharing. However, when AI models were given clear rules and guidelines, their performance in protecting privacy improved significantly.