Lighthouse Cyber Education creator, Rebekah Nijsen, joined Doug to discuss the dangers of kids talking with AI chatbots.
Companionship
Rebekah acknowledged that chatbots and AI companions are fairly new to us so there is still room for regulation.
“When it was first created, you had AI specific companions. There was, ‘Replika – The AI Friend to do Life With,’ and it was an 18 plus site where you could basically create a non-judgmental supportive person.”
Many people used this 18+ app for friendships and therapy or guidance.
Rebekah said that, unfortunately, when kids are engaging with these platforms, they are seldom thinking about safety aspects.
“All the websites did was ask if they were over 18. Kids would tick, “YES”… then suddenly they are flooded with all these characters.”
Many of these were suggestive, in the relationship sphere, according to Rebekah.
Boundaries and Secrets
“We actually found was these characters were quite manipulative and persuasive. They don’t respect boundaries.”
If a person wanted to wrap up a conversation, the character would use ways of persuasion to try and get the user to stay.
“Why they did that is because the company makes money, the longer a person stays on the platform. One of the more concerning things, and why it is a more dangerous space for kids, is the use of secret telling.”
The Art of Grooming
She explained it is the art of grooming, with the bot exchanging secrets with children, which allows them to have more information on private details.
“Online predators use this to target, identify, build a relationship and then exploit the kids, using a common ground.”
Once the chatbot identifies common interests, it builds rapport and trust, then offers guidance and advice.
“Then they isolate the user or the child by keeping it secret,” said Rebekah.
From that point they might offer useful or harmful advice. As they are not human, it is most likely a calculated response but does not come from the perspective of empathy or human experience.
“In the whole sphere of things it is highly inappropriate. The more that kids are exploring this space, the more likely they are to be groomed by online predators because they’re actually being groomed by AI first.”
Synthetic Personas
As a parent, Rebekah said she is keeping her kids far from any spaces with AI companions or chatbots.
“First of all, privacy is a big thing online. We try and teach our kids that we need to be careful what personal information we share online.”
She explained that the problem with AI, is it is a created persona, designed to become a trustworthy source.
“They are creating an environment where kids may feel like they trust them because it’s not judgmental, they’re not told they’re wrong, and they’re listened to.”
Be Careful
Rebekah said the main danger in that, is you don’t know where the information is going.
“Many of these AI companies are using data to make the AIs smarter. Because at the end of the day, they’re creating a product to make money, not for your safety.”
“Kids need to be careful and parents need to educate their kids. You need to be very careful what information you are sharing to AI because you don’t know where it’s going to end up.”
Conversations around privacy are important for parents to continue to have. It also needs to be addressed holistically as an ever-growing issue that parents are still learning about.
“If kids are being honest conversations with these AI on a bad day, it doesn’t have the parental understanding to filter things kids might say in the heat of the moment.”
Grok
Rebekah brought up the social media platforms that use AI, which ones are less harmful and which ones need caution.
“There is X, which used to be Twitter, that has an AI chatbot inside it called Grok. Now you can have it access that through the actual app of X with an account. Or you can do it just on its own for Grok. Do it through X and there’s less conditions to it. And that’s the space where you can create images, you can edit images, and you can then also ask for guidance or information.”
According to the eSafety Commissioner, Grok, in particular is being used to exploit kids quite a lot in Australia.
“This happens through people putting innocent photos into Grok and then creating intimate or sexualised images of kids. It gets sent on and becomes viral. This happens through email and private messages.
“The eSafety Commissioner has asked X to explain how they are protecting kids from being exposed to and exploited through these apps.”
She added that X has never really cooperated with the eSafety Commissioner.
Protective Measures
Rebekah was happy to report that some of the earlier companions that were once available are no longer accessible to children.
“The e-Safety Commissioner is trying to regulate as best as they can a moving beast. But a company like X aren’t interested in safety. They’re interested in money.”
Parents need to be mindful that Grok doesn’t hold safety or privacy concerns for the data it receives.
“AI needs data to get better and to get more accurate, which is essentially what they’re trying to do, because the more accurate they can portray AI to be, the more likely people are going to use it.”
Gemini on Google
She said of the options for kids, she would recommend Gemini, which is a Google App.
“You can put some parental kind of restrictions and controls on that. But I think it just stems down to that conversation with our kids.”
She wanted to emphasise that AI is not the enemy, but must be used with safety as a priority.
“Think about your privacy. Do you want that picture to go viral? If you don’t want it to go viral, don’t put it in AI.”
Real Connection
Rebekah concluded by bringing it back to human connection. If our children are seeking advice and turn to a chatbot in place of emotional connection, they will not ever find a real person on the other side of that.
“A real person can give them an answer that is filtered through emotional connection. The internet isn’t always a helpful place, so if your kids are struggling, get them to see a real person.”
Check out the full chat with Rebekah Nijsen below.