🏝 TheTechOasis 🏝

Breaking down the most advanced AI systems in the world to prepare you for your future.

5-minute weekly reads.

When two of the brightest minds in the history of Deep Learning share the same opinion about something, you simply listen.

Recently, Andrew Ng, Google Brain co-founder, ex-Chief Scientist at Baidu, founder of Coursera and DeepLearning.ai and Stanford University professor, and Andrej Karpathy, co-founder of OpenAI, ex-Director of AI at Tesla, and also Stanford University professor, have shared the view of what we might call the ‘Everything will be an LLM’ paradigm.

In this world, LLMs like ChatGPT become completely omnipresent in our lives, dictating how we work, how we organize our lives, or even what we see.

In this world, these LLMs become an extension of our body, both physically and digitally, and elevate our lives to a level of comfort, security, and wisdom that no other human in history has had the opportunity to behold.

And they aren’t alone.

Meta, Microsoft, Mistral, or Adept, some of the most prominent AI companies in the world, seem to be pushing this in their own way, and we will dive in to understand how these companies want to make AI to humans almost what oxygen is.

And companies like Apple, a mere observer at this point when it comes to Generative AI, are rumored to be tracing their complete strategy toward this vision, too.

But this path isn’t free from challenges.

For that vision to become a reality, we need to improve current LLMs in terms of efficiency and performance, so we will immerse ourselves into the forefront of flagship AI research, even highlighting papers just days old, to understand how the industry is turning miles into centimeters when it comes to making LLMs improve.

Finally, we will envision how this view fits inside the greatest obsessions of companies like OpenAI, Artificial General Intelligence (AGI), to make a bold bet:

You get an LLM, you’re friend gets an LLM, everyone gets an LLM

In a small tweet thread last week, Andrej Karpathy talked about a future where your ‘LLMs would talk to mine’.

The Exocortex and the edge

He predicted that in a not-so-distant future, everyone would have an ‘army’ of LLMs that would dictate much of what we do or see on the Internet.

For instance, he proposed ideas like using computer-level LLMs to block any unwanted ads or ads that the LLM knows are of no interest to me.

Moreover, what Andrej truly envisions is what we define as the ‘exocortex’, the hypothetical artificial external information processing system that would augment a brain's biological high-level cognitive processes.

In other words, Andrej thinks that in the near future, all of us will have LLMs acting as this ‘exocortex’ to amplify human cognition to a new level.

This could mean enhancing memory, processing speed, multitasking abilities, or even introducing entirely new cognitive functions that the human brain doesn't naturally possess.

On the other hand, Andrew Ng shares a less ‘CyberPunky’ view: The Cambrian explosion of ‘AI on the edge’.

In a recent newsletter issue of DeepLearning.ai’s The Batch, Andrew foresees a future where researchers get better and better at building hyper-performant LLMs that can be run on small devices such as your laptop or even your smartphone (this is of utmost criticality for Apple’s AI strategy, as we will discuss later).

Andrew argues that a perfect combination of LLM training improvements, high commercial interest from firms like Apple or Microsoft, and growing privacy requirements from the younger generations will cause the AI industry to transcend from huge, know-it-all models like ChatGPT into hyper-specialized smaller models that can be efficiently run in the confined security of our personal devices.

But are the industry and investors really pushing this vision forward?

The Great Question

Go big or go small? This question is becoming common among AI researchers.

On one side, we have AI giants like OpenAI or Anthropic, both actively pursuing the goal of building AGI while looking for ways to train larger and larger AI models.

On the flip side, companies like Apple or Meta seem to be pushing for Generative AI products that are small enough to be used on your phones or your laptop.

And then… there’s Microsoft. As you probably know, Microsoft is the main shareholder of OpenAI, owning almost half the company.

Thus, you may assume that their vision is clearly aligned with OpenAI. But the answer is… not really?

Yes… but no

As shared four days ago when Microsoft posted its FY24 Q1 earnings, everything seemed amazing.

Revenues were up 13%, in almost every category (including, inexplicably, an 8% increase by LinkedIn)… until we reached ‘devices’.

A 22% decrease.

As a great deal of Microsoft’s business model depends on consumer-end hardware and software, this is not great news.

Consequently, any technology that enhances the need for laptops is going to be welcomed by them, especially if we consider they are about to launch their newest super product, Copilot365.

Copilot365 is an AI agent, an LLM like ChatGPT embedded into Office 365 products like Word, Powerpoint, Teams, or Outlook, to boost productivity.

Under the hood, it’s an LLM that is connected to several APIs and has the capacity to execute actions based on user prompts.

A tooled-up ChatGPT, to be clear. Naturally, everyone just assumes that the LLM powering Copilot is ChatGPT.

But is it?

In an article a few weeks ago, it was rumored that Microsoft was scared of the huge costs coming from running huge models like GPT-4 when using them in software like Copilot that’s going to be used by millions in just a few weeks.

Consequently, they were venturing into the idea of using smaller models. A few months ago they released Orca, a 13-billion-parameter model that showed impressive performance that in many instances even surpassed GPT-4.

To train Orca, they used a process called distillation. Distillation involves teaching a student model to imitate a teacher model.

In this case, instead of training a model from scratch to predict the next word, they train a model to imitate the probability distribution of the larger model.

In layman’s terms, this smaller model learns to assign similar probabilities to the next word to what the larger model, often GPT-4, would.

This way, GPT-4’s prediction capabilities are transferred to a much smaller model that, despite being very small, is capable of making predictions only a much larger model would be capable of.

However, this process isn’t perfect.

In most instances, the student model will only imitate eloquent answers, but not reasoning.

This makes sense, as if you learn by heart how to do a math problem for an exam, but if they modify certain aspects, you are incapable of solving it because you’re missing the reasoning part.

However, Microsoft took a different approach with Orca. Instead of making it learn to imitate a response, they also taught the model to reason its way into the answer.

Consequently, many assume the LLMs running behind the different Copilots being launched by Microsoft are these models like Orca or Phi-1 (the latter for code), and not ChatGPT as initially thought.

But besides distillation, another option can simply be training a model from scratch more efficiently.

Mistral and Adept lead the way

Recently, a new wave of AI models are being launched that include a handful of surprises that make them not only great models, but also quite cheap to train.

A group of Chinese researchers trained a 100-billion-parameter LLM called FLM-101B with just $100k investment, using a growth strategy where the model was progressively expanded architecture-wise, instead of training models from scratch with the desired size.

Mistral recently launched their Mistral-7B model that incorporates features like Slide-window attention, an evolution of the self-attention mechanism where words only directly attend to a specific number of previous words instead of all of them, severely decreasing the huge computational requirements of vanilla transformers.

Also quite recently, Adept, a top-3 Generative AI company in terms of investment only overshadowed by OpenAI and Anthropic, launched Fuyu, a small model tailored to suit white-collar activities and that is multimodal despite not having an image encoder, as shown below:

And a paper released just a few days ago signals that models are about to get even smaller. A team of Chinese researchers have managed to quantize a model to just 4 bits per parameter.

In layman’s terms, this means that if you have a model with 100 billion parameters at full precision (32 bits, or 4 bytes) that would mean that the model requires, just to be hosted in memory (a requirement for Transformers) 400 GB.

With 4-bit quantization, that number decreases by 8-fold to 50GB. From needing at least 6 state-of-the-art GPUs to just one.

But besides technological advances, other aspects help support the idea that this is the way to go.

And the most exciting thing is that this is probably the best way to achieve AGI.

But why?

Subscribe to Leaders to read the rest.