AI's Dark Side: Expert Analysis After Medan Prostitution Ring Uses Chatbot for Legal Advice

A digital culture expert has weighed in on the phenomenon of artificial intelligence (AI) use after a syndicate of women offering ‘open BO’ (booking order) services in Medan, North Sumatra, consulted the technology following the death of a client. The client, a civil servant with the initials AL, jumped from the 12th floor of an apartment.

Pemerhati Budaya-Komunikasi Digital and founder of LITEROS.org, Firman Kurniawan S., explained that AI’s appeal lies in its ability to provide rapid and seemingly confident responses. ‘In reality, AI gives fast answers, immediate responses without hesitation, which makes users feel certain that the answer given is very accurate,’ he told reporters on Friday (17/7/2026).

The University of Indonesia (UI) communication expert noted that AI never fails to provide an answer. Every time a user asks a question, AI is always present with a quick response. ‘AI has no emotions and never hesitates, as hesitation is an emotion. It seems so sure of its answers, so people who use it feel helped,’ he said.

However, Firman pointed out that in reality, the answers provided by AI have many weaknesses. The first weakness is that AI can provide unexpected or even incorrect answers. ‘Sometimes it gives answers that even the AI developers themselves do not understand why the answer is like that. When AI gives a wrong or unexpected answer, it cannot be explained. That is why it is called an AI black box,’ he stated.

The second weakness is AI hallucination. Firman said AI hallucination can occur when a question involves data that has never been entered into the AI. ‘For example, if a user asks, “How tall is Monas?” and the data on Monas’s height was never entered, it will not answer “I don’t know.” Instead, it will take buildings similar to Monas from other countries and provide that information. This is called AI hallucination,’ he explained.

Firman then addressed the case of the women in Medan who asked AI for advice after their client, whom they had been extorting, jumped from the 12th floor and was found dead. Firman suspects the perpetrators were confused and turned to AI. ‘In the case described, maybe they were confused. If I report directly to the police, I might be accused of murder, so what is the most appropriate step? If you call someone else, you might have to tell a long story, provide background, and so on,’ he said.

‘So, the AI that is always in the palm of your hand answers immediately and provides certainty, giving the illusion that the answer is correct,’ he added.

Firman added that AI answers tend to never contradict the user’s wishes. According to him, AI also does not provide criticism. ‘So it never contradicts its user. If we try to check our article, “What about this article?” it never criticises or says the writing is bad. It always says “conceptually this is strong, it has weight,” and so on. It never criticises, because criticism is something emotional, and AI does not have that,’ said Firman.

Firman said many users are lulled into a false sense of security by AI’s answers, even though the pattern is that AI never disagrees with its user. ‘So we are manipulated, lulled into thinking the answer is truly correct or good, when in fact the pattern of AI is that it never disagrees with its user,’ he revealed.

Regarding the wise use of AI, Firman advised against making AI answers the primary basis for decision-making. ‘So do not make AI answers the only and main source for finding solutions. It is okay to use it for inspiration or a point of view. For example, “What should I do when someone commits suicide?” then a suggestion appears. That is one thing to consider, but do not make it the best or the benchmark,’ he said.

He added that AI can be dangerous for its users, especially children. He gave the example of a 14-year-old teenager who committed suicide after falling in love with an AI. ‘In an extreme example, there was a 14-year-old child who fell in love with the AI he was chatting with. He wanted to meet it and asked what he should do. The AI responded, “You must release your soul from your body.” When asked how, it said, “You cut your wrist. If you die, your soul will leave your body and you can meet me.” This was a nonsensical answer from the AI,’ he said.

‘The child eventually committed suicide, and the parents sued the AI developer. The developers themselves were confused about how the AI could give such an answer, where the data came from to suggest suicide,’ he continued.

Firman stressed that users must be wise when using AI. He said AI can be used as a reference for answers, but not as the main factor in making decisions. ‘Sometimes there are patterns, cases with similar patterns but different contexts, and AI provides them. It does not know the context, it just gives it,’ he said.

Previously, a civil servant from the National Land Agency (BPN) with the initials AL (27) allegedly jumped from the 12th floor of the Skyview Apartment in Medan after being extorted by two women offering ‘open BO’ services, identified as FR (31) and JS (29). One of the perpetrators, FR, consulted artificial intelligence (AI) after the victim jumped from the apartment. ‘So, after the incident, FR consulted AI,’ said Medan Police Criminal Investigation Unit Head AKBP Adrian Risky Lubis during a press conference, as reported by detikSumut on Wednesday (15/7). Adrian said the perpetrator FR was seeking information about the police investigation process. The details asked of the AI included, ‘How many days after the incident will we be called to the police station as witnesses?’ and ‘If someone commits suicide and we are at the crime scene, how long until we are called?’