AI is a buzzword presently, with the likes of Google’s Gemini scraping the internet for answers or Morgan Freeman-aping voice generators. If you are after a slightly strange-looking image of a clown riding a unicycle in 2056, there’s probably an AI-powered tool to assist with that task.
But that all seems rather reductive, like putting Ancient Rome expert Professor Mary Beard in an episode of Selling Sunset.
Artificial intelligence could be used for so much more than formulating banal wedding speeches – we perhaps haven’t seen the best of it yet.
In our racing video game and motorsport bubble, players have long been familiar with ‘AI’ – to us, the basic concept is nothing new. We have been racing ‘against the computer’ for decades, as the game and the hardware running it make rivals drive around the same track as you.
Some, undoubtedly, do it better than others. A relatively straightforward critique of a game or simulation is ‘the AI is bad’, with a lack of awareness, bunching on the first lap or repeatedly crashing at the same point.
That’s because, on a basic level, they never really learn. They cannot understand when they make a mistake and struggle to change their behaviour based on what is happening around them.
When Gran Turismo Sophy launched within Gran Turismo 7 for the public in late 2023, the artificial intelligence-powered agent was a revelation. Your opponents could attack, defend and avoid possible incidents in a more naturalistic way.
A key differentiation between the pre-programmed ‘AI’ that many are used to, and an agent such as GT Sophy, is its ability to adapt to context.
It has been the culmination of nearly five years of work, so far, with the July 2024 Gran Turismo 7 update expanding the track roster it can race on.
A new Sony AI racing agent that has eyes
But, for the global team at Sony AI, that was not enough. It has been diligently working away on an exploratory AI agent, in parallel to GT Sophy.
Once again using Gran Turismo as a testbed, and very much still in a prototype stage, the way it receives and interprets information is different – it can ‘see’.
“When you or I play, we, of course, look at the user interface and we parse that information in a way that sort of seems reasonable for us, and use that input to drive,” explains Kaushik Subramanian, Senior Research Scientist at Sony AI.
He works alongside Takuma Seno, a fellow Research Scientist and one of the Lead Authors of a recently published research paper covering the latest development.
“However, when we train GT Sophy, we provide it with very precise information about where it is on the track; the track shape, its exact velocity, its acceleration and where other cars are.
“So there’s a difference there in terms of what Sophy takes in and what human players take in.
“What we wanted to explore is if we could train an agent that works on image inputs.”
“If I were to take an image of what the game looks like – a car driving on a track with track lines, barriers, trees, and clouds – and try to train the racing agent with that as the input, would that be possible?”
So that’s exactly what the exploratory agent uses. Imagine a camera sitting inside a car or a pair of eyes (an ego-centric camera view in Sony AI parlance). The new AI ‘looks’ out of the vehicle and interprets what it can ‘see’, experiencing the velocity in combination with data in the form of the exact speed.
Consequently, the aim is to match what humans have access to when playing the game, with the image helping to explain the shape of the track. Unlike GT Sophy, it is not fed information such as the precise track shape, tyre slip angle (and slip ratio) or the wheel load.
“One of our internal motivations was to try to see how much we can relax some of the data inputs going into the agent,” continues the PhD graduate from the Georgia Institute of Technology, who’s been at Sony AI since 2019 and worked on Gran Turismo-adjacent projects from the off.
“The fascinating part about this is that we don’t have to control what it focuses on. We provide the image, and the AI approach learns what to focus on to get the result we are interested in.
“We tell the AI that its goal is to drive as fast as possible, stay on track and avoid colliding with the barriers.
“Given some training time, the algorithm learns to focus on relevant parts of the image to satisfy that goal. On a straight section, the AI focuses on what is coming up in the distance.
“Whereas if it’s looking at a corner, the pink marking dots [used to determine its focus] might line up on the edges of the track to help it stay within the course limits.”
A world-first learning agent for simulated car racing
For now, the new model only works in a time trial situation, although “It seems like a reasonable next step to now take this work and improve other situations, including races with opponent cars,” theorises Subramanian.
An AI lapping a track quickly is one feat, helping it understand racing etiquette is quite another.
As it stands, this is one car, one track, and in this instance, it has learnt to be significantly quicker than the majority of human players. It can the take optimum line, millimetre perfect, every single time.
This follows an exercise that lasts 10-14 days on a cluster of PlayStations with access to Gran Turismo 7 – this new model is by no means guaranteed to make it into retail versions and is still very much in an exploratory phase.
During this time, the AI uses reinforcement learning to get up to speed. This, in essence, is a form of trial and error, starting out haphazardly but eventually becoming a master.
“At the end, the result is what we refer to as a policy, a policy which acts as a controller,” explains Subramanian. Based in Germany, he has co-authored two research papers on racing and one on autonomous driving.
“After two weeks of training, the controller is ready and has learned how to drive the best lap. I press start. It just gives me that lap time in the first attempt.”
The net result, so far, is an agent that can deliver roughly similar times to the existing GT Sophy but without the requirement of external instrumentation. This is the creation of the first super-human simulated car racing agent with local sensor input.
You can’t help but get the feeling, however, that the Sony AI team would like to push the boundaries even further.
“On our side, there are several goals, including several research questions internally, that we’re trying to push the limits for,” explains the experienced reinforcement learning expert.
“Expanding the vision to other situations is definitely of interest to us. We’re going to try to do the most we can in that space.
“There is, of course, GT Sophy deployed in the game as well and that’s an exciting thought in our mind to make sure that players are happy with the version that’s in the game.
“When we started the project, we weren’t quite sure if it was actually learnable, because we know that humans do it, but the question was, can the AI do it too? It took a while, several years to get to where we are right now, and the results so far have been very promising.”
Sony AI’s latest autonomous racing agent at a glance
- The Sony AI team used PlayStations and Gran Turismo 7 to evaluate the exploratory agent, but it is not confirmed for a public release
- A different AI agent to Gran Turismo Sophy, currently in the research phase
- The first vision-based super-human simulated car racing agent
- It can out-pace the best Gran Turismo drivers on the planet on a single-lap
- Unlike GT Sophy – which is reliant on external instrumentation such as precise track shape information, slip angle, slip ratio and wheel load (among others) – the new agent can ‘see’ pixels (the ‘image’) through an ‘ego-centric camera view’ alongside receiving velocity and acceleration information.
