🤖 This Week’s AI Insight 🤖

For all the impressive features that ChatGPT, Stable Diffusion, or DINO have exposed to the world, they all have one common “limitation”:

They are bounded to the digital world, with no possibility to impact the physical realm… at least not explicitly.

But now we’re one step closer to achieving one of the grand dreams of AI scientists:

That is, the creation of agents that can act in the physical world with the agility, dexterity, and understanding of animals and humans.

In the not-so-distant future, this could not only fully automate all our production lines, but also take up the form of robot assistants, robot soldiers… or even robot lovers.

And the latest demonstration of potential comes with soccer (or football as we call it in Europe) robots that will make you smile, laugh, and also reflect on how society is not prepared, in the slightest, for what’s coming.

The Fascinating Concept of ‘Rewards’

The main characters of our story are sports robots trained using what we describe as Deep Reinforcement Learning, or Deep RL.

In a nutshell, this is the idea of using neural networks, that same concept behind ChatGPT, Dolly, or even Tesla’s Autopilot system, but applied to RL.

But what is RL?

While NLP models like ChatGPT work by processing natural language and generating a textual or image response, and CV models like SAM learn to process images and identify elements and infer patterns from them, RL teaches AI agents to interact with their environment.

To further understand, you can think of training RL agents as training your dog puppy.

By rewarding him/her with treats for good behavior and “punishing” without them otherwise, the dog learns the pattern of actions (the policy) that will yield him/her the greatest amount of treats (rewards) possible.

But how do we teach robots to play sports?

It’s all about winning

As you know, the aforementioned sport is not only about kicking a ball; the robot needs to learn to get up, move, shoot, and, in 1v1 situations, defend its goal.

Hence, the DeepMind team assembled a set of rewards that would allow the robots to learn to play it by:

• Rewarding it for scoring, getting up, and defending

• Penalizing it when receiving goals, or staying on the ground

To put this into practice, Google DeepMind leveraged a very powerful concept that’s gaining a lot of traction recently in the world of AI, distillation.

Like Father, Like Son

The researchers first trained two robots:

• A scoring teacher, a robot that learns to score goals effectively

• A get-up teacher, a robot that learns to get up from different poses effectively

Next, they should this process of distillation, which is getting a student robot to learn from them.

To do so, they defined a training objective that first forced the robot to “clone” actions from their teachers.

But wait, why not just learn both things from the beginning?

By applying distillation, a smaller model can learn to act like larger models very efficiently, considerably reducing costs while achieving similar results.

Then, once the student equaled its teachers’ abilities to score and get up, they limited their influence and made it play against copies of itself in previous versions, something known as self-play.

By making the student play worse versions of itself in a 1v1 environment, it developed new emergent behaviors apart from getting up and scoring, like:

• Defending its goal (by penalizing goal-receiving)

• Running (it increases the chances of winning)

• Understanding the opponent’s movements and reacting to them

• Getting up quickly, as being on the ground is a surefire way to lose

And the results?

Well, the following videos blew my mind and will blow yours.

Resilient and Committed to Winning

I could bore you with numbers as to how superior are these robots to previous research on this topic (156% faster, 63% quicker to get up, or 24% better at scoring), but videos are better than anything I can explain to you:

• 1v1 soccer match

• Analyzing all emergent skills

• Recovering from ruthless pushes from a researcher 

But this isn’t about sports, and you know that.

From Sports to Love

The field of robotics is poised to impact the labor market as much as anything we can think of.

• It has the power to fully automate factories, killing millions of jobs.

• It has the power of creating machines that interact with the real world as well as you do, with all the amazing use cases and risks that entails.

Soon, Black Mirror could be seen as a predictor of our future more than a series portraying dystopian realities.

Needless to say, AI is converging quickly into making these futuristic science fiction stories lose the fiction part.

Even scarier, humans are naturally drawn to become romantically entrenched with machines if these anthropomorphize.

We’ve already seen cases of this.

👾Top AI news for the week👾

🤨 According to a Google employee, Microsoft won’t win the AI race… and neither will Google

🤑 Microsoft rumored to plan private ChatGPT offerings

☣️ The biggest risk with GenAI is not jobs, it’s hyper-personalized advertising

🐶 Snoop Dogg shares his alarming views on AI in a hilarious way

💸 Now we know how much it cost to build ChatGPT… and it was huge

💣 Google goes into wartime mode

🤩 The biggest breakthrough in programming in years is Mojo

🧐 While ChatGPT has surface-level conversations, Pi goes deep and gets to know you

🎤 You can now hear Kayne West singing Hello by Adele, and many more

😔 The great AI job wipeout is finally starting