Artificial intelligence recently infiltrated the world of Rockstar’s Grand Theft Auto V thanks to a new mod.
The mod is called Sentient Streets, and it lets players interact with over 30 AI-powered NPCs across the mission. Each character has a unique voice from AI text-to-speech and voice cloning website ElevenLabs. The mod is designed for players to have open-ended, voiced conversations with cult members, police officers, and civilians.
Sentient Streets was built by veteran modder Bloc using the Inworld Character Engine, which includes features like assigning goals, actions, relationships, and voices to AI-powered NPCs.
On YouTube, Bloc demonstrated how NPC conversations work in Sentient Streets. You approach a character, press a key to get their attention, and then you can start speaking to them through your microphone. Bloc asked a police officer his name and how he got his nickname, and after a short pause, the NPC responded with a small anecdote. It’s not entirely natural — the NPC repeated some information in back-to-back answers — but every character is programmed with different personalities and backstories.
The NPCs did respond to whatever question Bloc asked through the microphone, but you can definitely tell the answers are AI generated. For example, here’s a line from a new police recruit Bloc spoke to in the gameplay video: “Well, well, well, looks like someone’s got a keen eye for fresh faces. Guilty as charged, Officer Newbie reporting for duty! Ready to bring some sparkle to this city?”
With Inworld’s intelligent #AI NPCs and @elevenlabsio voice technology, experience #GTAV as never before in the Sentient Streets mod by Bloctheworker.
Read more about the ElevenLabs x Inworld integration: https://t.co/WuNz53ZRbq#gamesdev #gamemod #chatgpt #aicharacters pic.twitter.com/HD94rw2Iql
— Inworld AI (@inworld_ai) August 8, 2023
Bloc continued to demonstrate the AI conversations as the demo progressed, including a lengthy back-and-forth with his partner when they were driving around Los Santos in their police car. In an interview with IGN, Bloc said players can push conversations with NPCs pretty far.
“Especially with Inworld’s current AI system in place, characters can really act according to their backgrounds,” Bloc said. “For example, if it’s a crazy person, you can hear that craziness in their speech. Or if a person is a righteous character, you get that vibe from their speech as well. All these variations result in hilarious and sometimes spooky conversations.”
In addition to the AI-generated conversations, the entire story of Sentient Streets revolves around artificial intelligence as well. A deadly cult called the NihiAIists is aiming to take over GTA V’s Los Santos. The group worships an unseen AI as their god, and players take control of an officer for the Los Santos Police Department who works to stop the cult’s plan.
Ethical Questions Surrounding the Use of AI Voices in Mods
Some ethical questions are raised when digging into the origins of these AI-generated voices. Bloc used the standard voice library available on ElevenLabs, and ElevenLabs shed some light on where its voices come from in a statement to IGN.
“Standard voices available by default on the platform are either generated by AI algorithms that sample voice characteristics at random (i.e. they don’t imitate or replicate any specific individual’s voice) or are developed through legally contracted, time-limited partnerships with voice actors, with new custom AI voices created as a result,” ElevenLabs’ statement reads. “ElevenLabs does not offer any AI-voices on the platform that are based on a real person’s voice without explicit permission of that individual.”
Recently, ElevenLabs was used to generate AI voices used in NSFW Skyrim Mods hosted on Nexus Mods. The recent rise of AI has been a concern for both writers and actors, causing one voice actor to tell IGN AI is the “invisible enemy we’re fighting right now.”
Users are able to submit voice clips from real people to convert them into a cloned, AI-generated voice, but ElevenLabs clearly states that submitting real voices that users don’t have the rights to is not allowed on the platform. But, that’s a very difficult thing to track, given how many mods are out there using AI-generated vocies from real voice actors.
“ElevenLabs also allows users to create new, randomly generated AI voices and share them as part of the community-led Voice Library,” ElevenLabs said. “Separately, users have the ability to create cloned voices for their own work, if they have the rights and permissions to those voices. These voices cannot be shared to the Voice Library. Users who contravene the Terms of Service are banned from the — everyone is encouraged to report content they believe has violated these terms.”
On ElevenLabs’ official website, the terms of service state that, “Content which is not protected by law can be reused freely, including for voice cloning. This type of content is also known as content ‘in the public domain’ or ‘public domain’ content.”
ElevenLabs lists a few examples of what qualifies as public domain content, including materials which were not protected by law in the first place, materials where the protection has expired, or materials dedicated to the public domain by their creators. Elsewhere, ElevenLabs cites “a vast amount of audiobooks, and to a lesser extent, podcasts” as the source where the AI was trained.
The use of AI voices in mods is sure to make a lot of players feel uncomfortable about the future of AI, but mod creator Bloc doesn’t see it that way.
“I think AI-generated content will never replace human-created content, and it shouldn’t anyway. AI is a tool, a very powerful and useful tool, but still a tool. Instead of replacing, AI will transform human-created content and possibly improve it.”
Sentient Streets is available to download on Nexus Mods.
Logan Plant is a freelance writer for IGN covering video game and entertainment news. He has over seven years of experience in the gaming industry with bylines at IGN, Nintendo Wire, Switch Player Magazine, and Lifewire. Find him on Twitter @LoganJPlant.