Often Confiding in AI? Here Are the Risks Rarely Realised

Jakarta (ANTARA) - Chatbots based on artificial intelligence (AI) are increasingly being used as a place to share problems and seek emotional support. However, sharing personal matters with a chatbot still carries risks, particularly relating to data privacy and response reliability.

A chatbot’s ability to respond conversationally in natural language can make users feel more comfortable when discussing personal issues. The topics covered can be highly varied, ranging from work problems, relationships and family to mental health conditions.

Although it can serve as a means of obtaining information or helping someone organise their thoughts, a chatbot is not a replacement for friends, family, psychologists, psychiatrists or professional health workers.

One thing that needs attention is that the information given to a chatbot can be highly sensitive.

Users sometimes disclose full names, addresses, family problems, health conditions, personal relationships and work issues without realising that such information constitutes personal data.

A study published in Nature Computational Science in 2025 highlights that the use of AI for mental health brings privacy concerns, including how sensitive data is collected, stored and processed. Researchers also stressed the importance of privacy-preserving approaches in developing AI models for mental health.

Therefore, users should not treat a chatbot’s conversation box as equivalent to a professional consultation room bound by special confidentiality rules.

AI can give answers that sound convincing

Another risk is that chatbots can produce answers that sound plausible but are not necessarily correct.

This becomes more important when users discuss mental health problems and ask AI to provide a diagnosis or determine what action to take.

A recent review in npj Digital Medicine, which examined 119 articles on the potential adverse effects of large language model (LLM)-based chatbots, identified a number of issues, including misinformation, overly agreeable responses, bias, data security problems and risks when chatbots are used in severe mental health cases.

This means that a response that feels empathetic does not necessarily indicate that the chatbot clinically understands the user’s condition.

AI can help someone find initial information, organise their thoughts or identify questions to raise when consulting a professional.

However, users should not make a chatbot their sole source of help when facing serious problems.

Research published in Scientific Reports in 2025 tested 29 chatbots claiming to help with mental health issues in scenarios involving suicide risk. Not one met the initial criteria for providing what was considered an adequate response, whilst nearly half were judged inadequate based on more lenient criteria.

These findings show that a chatbot’s capability in everyday conversation does not automatically make it safe for handling mental health conditions requiring professional intervention.

There is a risk of becoming overly dependent on AI

AI’s ability to respond quickly and use language that feels personal can also make some users increasingly comfortable interacting with it.

A review in Nature Machine Intelligence highlighted concerns about emotional attachment to and dependence on AI companions, particularly among vulnerable users.

Research published in Nature Human Behaviour in August 2026 also examined the relationship between AI companion use and psychological wellbeing based on data from 1,131 adult users in the United States as well as thousands of conversation sessions. The study underlined the importance of understanding usage patterns and the emotional relationships formed between users and AI.

Therefore, using AI to talk occasionally is different from making it one’s sole source of emotional support.

What data should not be shared?

To reduce risks, users should limit the personal information entered into a chatbot.

Avoid entering data such as identity numbers, full addresses, telephone numbers, passwords, bank account details, payment card data, personal documents and highly sensitive health information unless absolutely necessary.

When discussing a problem, identities can be anonymised. For instance, names of people, companies, schools or specific locations can be replaced with generic terms.

In this way, users can still ask for help in understanding an issue without providing too much information that could identify themselves or others.

Every AI service may have different policies on how conversations and user data are processed.

Therefore, before using a chatbot for sensitive matters, users should read the privacy policy and data settings of the service being used.

This is increasingly important because recent research on mental health chatbots places data security and privacy among the main risk categories requiring attention.

Users also need to understand the features available on the service, including options for managing conversation history or data, where available.

AI can still be used wisely

This does not mean chatbots should never be used for sharing problems.

AI can be used to help organise thoughts, keep a reflection journal, search for general information, prepare questions before consulting a professional or simply gain an initial perspective.

However, users need to maintain clear boundaries. Do not provide unnecessary personal information and do not treat a chatbot’s answers as a diagnosis or professional decision.

A number of AI companies have also begun developing safety features for sensitive conversations. OpenAI, for example, introduced a Trusted Contact feature in 2026 as an additional layer that can help connect adult users with someone they trust when the system detects conversation about self-harm at a level of serious safety concern.

Even so, such safety features are still no substitute for human help in emergency situations.

Thus, confiding in a chatbot can be done for mild matters, but it should not be done by sharing all personal information without consideration.

The simple principle is to treat every piece of information typed as something worth thinking twice about before sending. Anonymise identities, do not enter unnecessary sensitive data and always verify important information from trusted sources.

For serious mental health problems, especially when thoughts of harming oneself or others arise, human help remains the priority. Contact a trusted person, a health worker or local emergency services.

Ultimately, AI can be a useful tool for talking and seeking information, but the comfort of speaking with a chatbot does not always mean the conversation is entirely safe, confidential or accurate. Users still need to protect their privacy and understand the limits of the technology’s capabilities.