Conversational artificial intelligence (AI) has changed that. Big AI companies may already know much more about you than you’re comfortable with, even if you’ve never used their services.
The problem goes beyond well-known products like ChatGPT, Gemini or Claude. Many chatbots and companion AI apps can now remember past exchanges, personalise responses, use human-like voices and personas, and even initiate contact. The same features that make them useful also create privacy consent problems.
Information shared without consent
A recent YouGov survey shows 15% of adults in Australia have shared personal thoughts or feelings with a chatbot, and 11% disclosed something they never told anyone else.
The trend is even more pronounced for adolescent users. According to the Australian eSafety Commissioner, 54% of children aged 10–17 have used a chatbot for personal or social advice and one-third for life advice.
Because AI chat windows feel like private spaces for confiding, people also often share information about others – sometimes intimate details – without their knowledge or consent. A colleague might paste an email from you into ChatGPT to solve a problem. A friend might upload screenshots of your messages to make weekend plans. Your partner might discuss your latest quarrel. Your parents might share concerns about your health or finances.
You may never know. Yet information about non-users – people who haven’t even interacted with the chatbot – can still enter these AI systems through billions of everyday interactions. And we have no way of knowing what’s there.
Two sets of legal reforms
The issue is not just that AI companies know stuff about you, but what that knowledge might let them do behind closed doors. Information can be reused for training AI models, personalisation, profiling, or advertising.
Intimate knowledge can be used to influence people, from pushing potentially harmful advertising to political persuasion or reinforcing extremist beliefs.
Australia is considering two sets of law reforms that could help regulate the privacy risks associated with chatbots, including for non-users.
Proposed privacy reforms would require personal information to be handled fairly and reasonably, and introduce a “right to erasure”. This would require large digital platforms to destroy personal information at an individual’s request.
Meanwhile, the proposed digital duty of care – an amendment to the Online Safety Act – would require online services to assess and manage foreseeable risks arising from their design and operation.
Together, these proposals get a lot right. Privacy reform can protect information about non-users, while the digital duty of care can address harms arising from how chatbots are designed and operated.
But how do you exercise a right to erasure if you don’t know who knows what about you? We need specific measures to ensure the legal requirements actually stick.
