Where AI assistants process and keep your questions

01

What happens to your prompt when you use an AI assistant?

When you use an AI assistant, your prompt is processed by a language model either on the provider’s servers or on your own device. Where it runs decides who receives your words; the provider’s policies and your settings then decide how long they are kept, whether they train future models and whether people may read them.

Five questions decide what that means in practice: where the model that reads your prompt runs, what is kept afterwards and for how long, whether it is used to train models, who can read it, and what your account connects it to. A sixth applies whenever an assistant uses the web: what the search engines and sites it contacts receive.

This explainer takes each question in turn and ends with a checklist for judging any assistant’s privacy claims. It describes how assistants work in general rather than any one provider; what each provider states about its own assistant, quoted and dated, is on our comparison pages. It does not rank assistants; the right trade-offs depend on what you ask.

02

Where is your prompt processed?

Most AI assistants process prompts on the provider’s servers, because the largest language models need data-centre hardware to run. A smaller number run a model on the phone itself, so the prompt is processed on the device it was typed on.

In the cloud case, the app sends your prompt to the provider, usually with recent turns of the conversation and any attached files or photos. The connection is encrypted with TLS, which stops anyone on the network path from reading or altering the message 1. TLS protects the message in transit, not at its destination: the provider’s systems decrypt the prompt so the model can read it, and from that point the provider’s policies and your settings govern what happens to it (sections 03 to 05).

On a device, the model’s weights are downloaded once and the computation, called inference, runs on the phone’s own processors. The prompt goes only to the local runtime. Android offers this as a system service: Google states that Gemini Nano “lets you deliver rich generative AI experiences without needing a network connection or sending data to the cloud” 2. Apps can also download and run their own models, as our guide to running a language model on an Android phone explains. The trade-off is size: a phone-sized model knows less and reasons less reliably than a frontier cloud model (see small language models, explained).

Some systems mix the two, answering simple requests on the device and sending harder ones to a server. Others aim to make server-side processing checkable by running it in hardware-isolated environments whose software can be verified remotely, the subject of our explainer on confidential computing. Either way, the useful question is which computer reads the prompt for each feature. On-device AI, explained covers the architecture in more depth, and Figure 1 follows one question through both designs.

An on-device model changes who receives a prompt, not whether the app ever uses the network. It may still download models, check for updates, send analytics or search the web, which is why the checklist in section 08 asks what leaves the device besides the prompt.

03

What is stored, and for how long?

A cloud assistant usually keeps your conversations with your account until you delete them or an auto-delete period ends, and may keep some copies for longer, for safety review, legal reasons or model training.

Defaults differ by provider, by product and by setting. Some assistants keep your history until you delete it, others delete it automatically after a set period that you can often change, and some offer a temporary or incognito chat that does not appear in your history. Our comparison pages quote each provider’s own figures, with the date we checked them.

Deleting a chat is not always the end of it. It helps to separate three kinds of copy: the history you can see, the provider’s back-end copy, which can take some time to be removed after you delete a chat, and copies made for other purposes such as safety review, feedback or training, which follow their own schedules (sections 04 and 05).

On a phone, the question becomes whether the app copies its files anywhere else. Android backs up app data to the user’s Google Drive by default, and each app can opt out 3. Opting out is not quite the whole answer: Android’s documentation notes that for apps targeting Android 12 or later, on some manufacturers’ devices, turning backup off this way still allows device-to-device transfers 3. An app that wants chats to stay on one phone has to exclude them from transfers as well.

04

Are your conversations used to train models?

It depends on the provider, the product and your settings. Some consumer assistants use conversations to train models unless you opt out, others only if you opt in, and business and API products are usually covered by separate terms.

Providers describe these choices in their privacy documentation, and the details matter: which conversations a setting covers, whether feedback you submit can be used when training is off, and whether conversations flagged for safety review are treated differently. Our comparison pages set out, in each provider’s own words and with the date we checked, what OpenAI, Google, Anthropic, Microsoft and Perplexity say about training on consumer chats.

Training matters because it is hard to undo. A trained model does not store conversations as records that can be deleted; what it learns is spread across its parameters, so turning training off applies to future training, not to models that have already been trained. Models can also memorise rare text. Carlini and colleagues extracted hundreds of verbatim sequences from GPT-2’s training data, including names, phone numbers and email addresses, and found larger models more vulnerable than smaller ones 4; later work extended such attacks to production language models 5.

Some providers de-identify data before training. De-identification removes the link to your account. It does not necessarily remove personal details you typed into the conversation itself, which is why temporary or incognito modes, where they exist, are worth knowing about. For an on-device assistant the question changes: conversations that never reach the developer cannot train its models, so what matters is whether anything else, such as analytics, crash reports or feedback, carries them off the device.

05

Who can read your conversations?

With a cloud assistant, people working for the provider or its contractors may read a sample of conversations, to improve quality and enforce safety rules, and stored chats remain subject to the provider’s legal obligations. Anyone with access to your account or your unlocked device can read your history too.

Providers that use human review describe it in their privacy documentation, and the details differ: whether reviewers include contractors, whether a chat is disconnected from your account before review, how long reviewed chats are kept, and whether they are deleted when you delete your history. Some providers ask you not to enter anything confidential that you would not want a reviewer to read. Our comparison pages quote each provider’s own description.

Automated systems scan conversations too, and a flag can change how long one is kept: a chat flagged as breaking a provider’s usage rules can be kept for longer than ordinary history, and providers keep chats where the law requires it.

Shared links and connected apps also pass conversations to other people and services. An on-device assistant with no server removes the provider as a reader, but not the people who can pick up your phone.

06

What does an account link to your conversations?

An account ties each conversation to an identity, typically an email address and sometimes a phone number or payment details. That is what lets your history follow you across devices, and it is also what makes stored chats personal data about you.

Accounts make possible the features that depend on remembering you. Some assistants offer a memory feature that carries details from earlier chats into new ones, and those details can include sensitive information you mentioned in passing. Personalisation can be useful, but it means one conversation can shape later ones, so it is worth checking whether such a feature is on and how to clear it.

Because account-linked chats are personal data, data protection law gives you rights over them in many places. In the European Union, the General Data Protection Regulation includes a right of access (Article 15) and a right to erasure (Article 17) 6, and providers explain how to use them in their privacy policies.

Using an assistant without an account narrows what can be linked, but every service you connect to still sees the network address a request comes from. An assistant that needs no account and sends nothing to its developer leaves the developer nothing to link.

07

What do third parties see when an assistant searches the web?

When an assistant searches the web, the search engine receives the search text and each site the assistant reads receives a page request. Whose network address they see depends on which computer sends the request: the provider’s servers or your own phone.

A cloud assistant typically searches from the provider’s servers, so search engines and sites see the provider’s request, and what the provider shares with its search partners is set by its own agreements and policy. An on-device assistant that searches from the phone sends the requests directly, so the search engine and every site it reads see the phone’s IP address, as for any web request. In both cases the search text is usually a rewritten query rather than your whole prompt, but it still describes what you asked.

Two request headers let an app state a preference. Do Not Track, sent as DNT: 1, asked sites not to track the user; its W3C specification ended as a Working Group Note in 2019 because there had “not been sufficient deployment” 7. Global Privacy Control, sent as Sec-GPC: 1, conveys “a person’s request to websites and services to not sell or share their personal information with third parties” 8. Both are requests rather than technical protections: a site can ignore them, though GPC is designed to work with laws that make such requests enforceable.

Because a search runs on someone else’s computer, it is, along with downloading the model, the part of using an on-device assistant that depends on another organisation’s privacy policy. That is the reason for the design in our research note on asking before going online: Brello 1.0 asks before it searches.

08

How can you judge an AI assistant’s privacy claims?

Judge a privacy claim by whether it names a mechanism you can check: where the model runs, what is stored and for how long, who can read it and what leaves the device, each with a scope and a date. A claim that names no mechanism, such as “private by design” on its own, tells you little.

1. Where does the model run? Look for a plain statement for each feature: on the device, on the provider’s servers, or both.
2. What is kept, and for how long? Look for numbers, in days or months, for your history, for back-end copies after you delete it and for copies kept for review or training.
3. Is training on by default? Check whether conversations train models unless you opt out, only if you opt in, or not at all, and what a temporary or incognito mode still keeps.
4. Who can read conversations? Check whether people review them, whether those people work for contractors, and how long reviewed chats are kept.
5. What does the account link? Check what identity it requires, whether memory or personalisation uses past chats, and whether chats sync or back up.
6. What else leaves the device? Analytics, crash reports, advertising, web search, link previews and site icons all send requests that can reveal what you were doing.
7. Can you check it, and is it dated? Prefer published architecture, open-source code or independent verification to assurances, and look for a “last updated” date and a record of changes.

The same questions apply to Brello. The box below answers them for Brello 1.0, with the limitation that goes with each answer.

09

How do cloud assistants compare with Brello 1.0?

On these questions, cloud assistants differ mainly in their defaults and settings, while an on-device assistant answers most of them by where it runs. Table 1 sets out the general pattern for cloud assistants next to Brello 1.0.

Each provider also offers controls a table can’t capture, and terms change, so check a provider’s own page before relying on a general pattern. Side-by-side comparisons with Brello 1.0, each sourced to the provider’s own documentation and dated, are on the comparison pages for Brello 1.0 and ChatGPT, Brello 1.0 and Gemini, Brello 1.0 and Claude, Brello 1.0 and Copilot and Brello 1.0 and Perplexity, all listed on the comparisons page.

Where Brello 1.0 is weaker matters too. Its models are far smaller than the models behind cloud assistants, so its answers can be wrong more often and it is weaker at long or complex reasoning; and with no sync, a lost or reset phone takes its chats with it. How Brello handles your questions, photos and answers sets out every data flow, and the glossary defines the terms used here.