• TheTechOasis
  • Posts
  • Stunning Discovery, Living Computers, brain-controlled Games, & More

Stunning Discovery, Living Computers, brain-controlled Games, & More

In partnership with

THEWHITEBOX
TLDR;

  • 👏 New Premium Content on embedded software development, beating GPT-4o with ten lines of data, improving LLM performance by 20% with ten lines of code, and AgentQ’s deep dive 

  • 📰 Hot news from video generation models, Neuralink, living computers, & OpenAI

  • 🏝️ Insights on the most used GenAI products, the project that saved billions with AI, & more

  • 🆕 Trend of the Week: Neuroscientific breakthrough with huge impact on AI. Dreams are Simulations.

Power your competitive advantage with intelligent automation from ELEKS

ELEKS' intelligent automation service transforms your business operations through data-driven solutions. We automate complex tasks, streamlining processes to increase productivity and reduce operational costs. Our tailored solutions adapt to your changing needs and help you unlock new growth opportunities by freeing your team to focus on high-value tasks.

The result? Enhanced customer satisfaction, improved client retention, and a stronger market position.

OTHER PREMIUM CONTENT
Other New Premium Content

PREMIUM CONTENT
Things You Have Missed For Not Being Premium…

Newsreel📰

HARDWARE
Living Computers are here… for rent

To replicate one human’s brain computing-power-to-energy-use ratio, with 86 billion neurons and just 20W of power required with a digital processor data center, you would need a 10 MW power plant, enough to provide electricity for 1,600 homes.

But FinalSpark is proposing the use of Bioprocessors, human brain organoids that provide the required compute in a much cheaper way.

And now, you can rent them over the Internet for just $500. In the meantime, you can watch a live stream of the system.

TheWhiteBox’s take:

I mean, the technology is fascinating. By stimulating the organoids in a specific way, the researchers induce certain neurological pathways in the organoids that can imitate logic gates, memory, and other key elements of compute processing to enable them to perform calculations.

However, I have many concerns about this method. Is it scalable? The cells have a 100-day service period, and I don’t think mass production of human brain organoids is easy to find.

But the incentives to find cheaper compute alternatives are huge, considering that we don’t have a definitive answer to how we are going to handle a potential increase of 40 GW of power requirements for AI in just two/three years, so large that the total data center demand for US and China in 2027 will outpace total France and Italy combined energy consumption in 2022… and that’s without considering Europe, too.

VIDEO GENERATION
The Text-to-Video Wars Intensify

Both the Chinese Kling and Luma’s Dream Machine, along with Runway two of the three biggest rivals to OpenAI’s Sora, have seen quite the quality upgrades this week.

TheWhiteBox’s take:

While the field of video generation is advancing rapidly, we have yet to see a single model that can confidently understand the world. They simply don’t understand causality.

Using Kling as an example, in the fourth video, you can see a kid biting into a gyoza dumpling that… survives the bite unscathed. On a final note, Ideogram has released its second version, which has improved text generation, too; image and video synthesis is clearly a hot topic right now, but not more than a fun toy to play with.

LARGE LANGUAGE MODELS
OpenAI’s GPT-4o Fine-tuning

OpenAI has announced fine-tuning for its best model, GPT-4o, with 1 million free tokens per organization until September 23rd.

TheWhiteBox’s take:

It’s a trap. Nobody needs it unless they are deep into the OpenAI flywheel or your use case is highly complex (refer back to Killer Applications rounds one & two for such examples).

Fine-tuned open-source models yield better performance for a fraction of the cost. By using adapters, you don't have to pay 20 or 30 times more just because it’s ChatGPT.

You are, quite literally, paying more for less.

And even if you can build a fine-tuned GPT that is better than any open-source model, you still have to trust OpenAI won’t steal your data… coming from a company that blatantly stole YouTube data from Google (and creators).

BRAIN-COMPUTER INTERFACES
OpenAI’s GPT-4o Fine-tuning

Neuralink has released footage of a patient playing video games with his mind, a video which you can see here.

TheWhiteBox’s take:

LLMs are exciting yes, but not “eliminating-disabilities-forever” exciting like Brain-Computer Interfaces such as Neuralink. But how do they work? 

If you are surprised by the fact that someone can interact with a computer using the mind, think of Artificial Intelligence models as a data mapping.

They map a series of inputs to a set of outputs by studying the input data (brain signals captured by the Neuralink brain decoder implanted in the patient’s brain), finding key patterns such as ‘whenever the brain activates this way, the patient is thinking of doing this movement’, and extrapolating these patterns to cursor and keyboard actions to interact with the video game.

In fact, you can view the similarity with ChatGPT directly; while LLMs map a set of input words to the next one (the output) using a text decoder, here we are mapping brain signals to cursor/keyboard actions using a brain decoder. But at the end of the day, they are all neural networks (and most presumably, we can assume Neuralink models are Transformers, too).

At this pace, we could see functional brain interfaces by the decade's end; how cool is that?!

LEARN
The Insights Corner

😱 DreamMachine 1.5 Announcement, by Luma AI

👩‍🔬 The 20% project that saved Google billions by turning data center management into an AI game, by Sequoia’s Training Data pod

TREND OF THE WEEK
Our Dreams Are Training Simulations

A remarkable breakthrough has been achieved in neuroscience; Humans learn through dreams by actively simulating the events (and the consequences) occurring in them, implicitly proving the existence of world models.

But what does that have to do with AI?

The discovery suggests that, in our effort to build human-level intelligence in machines, we might be doing it all wrong.

Today, we are embarking on a fascinating read that will teach us a lot about how your brain works and what inspiration the AI industry is taking—or lack thereof—in its arrogant path to creating artificial human-level intelligence.

A Controlled Hallucination

In 1995, a construction worker fell from a scaffolding in New York City. To his horror, he had fallen into a 15-cm nail that had traversed its shoe completely.

The Perception of Pain

Naturally, he was in indescribable pain, to the point that he was quickly put on fentanyl and taken to the hospital. But when doctors analyzed the injury, to their surprise, the nail had completely missed the worker’s foot.

But then, how was he in so much pain? Well, because his brain said he should be feeling pain. And so pain he felt.

This is why philosopher Andy Clark refers to reality as a “controlled hallucination.” Or, as famous cognitive psychologist Donald Hoffman would put it, ‘Reality isn’t what you think it is, but who you are.’

In layman’s terms, reality is a mix between what our brain predicts will happen and what it eventually perceives (through the senses) to be happening, leading to situations where the brain may misinterpret reality, as we just saw.

But then, how does our brain learn?

The trial and error feedback loop

As you may imagine, the brain uses perception as feedback on its beliefs, adapting them over time. For instance, a baby learns that gravity exists by picking and dropping stuff, shaping his/her brain’s prediction that picking and ‘letting go’ a new object will result in the object falling.

Consequently, how well your brain leverages this feedback loop largely influences your learning, which can lead to exceptional circumstances.

For instance, some neuroscientists speculate that a poor (or unbalanced) ‘feedback loop’ may play a crucial role in brain pathologies like depression, chronic pain, or autism.

As for the latter, some scientists suggest that autism results from the brain giving too much weight to incoming sensory input. By receiving too much outside information, the brain becomes incapable of discerning what is noise and what is valuable information to make successful predictions.

This may be why some people on the spectrum are highly awkward in societal interactions: Their brains cannot detect low-frequency, faint societal cues that others give them as, to their brain, ‘all outside-in information is valuable.’

This could also explain why autistic people are so innately superior to non-autistic ones in tasks that require extreme perception, proving that inserting autistic people isn’t about forcing them to be like the rest, but finding where their unique particularities shine through.

But besides this particular case, your brain is actually pretty darn arrogant and thinks it knows best.

Nonetheless, our outgoing neurological pathways (what the brain predicts) outnumber the incoming pathways (what our senses perceive) by two to one. Simply put, although counterintuitive (and to the horror of behaviorists), reality is experienced more from the inside out than from the outside in.

In other words, this Greek-time idea (picked up by John Locke) that our brains are a ‘Tabula Rasa,’ a blank slate completely shaped by experiences, may not be accurate, and that a majority of our reality is shaped by the expectations of our brain, which can lead to situations in which our brain can completely alter our reality.

And I’m not referring to drugs.

Regaining sight back

In a 2019 study, a woman with almost complete blindness regained her sight completely. But how was that possible?

When they evaluated her at first, they quickly realized that she responded to different visual stimuli, suggesting that her sight was fine. However, she was still totally blind.

But how?

Importantly, she had suffered severe migraines that, over the years, had incentivized her to look for dark places to avoid the pain. Thus, they hypothesized that her brain had convinced itself that she was blind when she wasn’t.

Fascinatingly, after a series of therapeutic approaches, like reinforcing every positive signal that her sight system was just fine to her and her family, and even using hypnotherapy, they ‘tricked the brain back to normal.’

There have also been cases in which a woman with multiple-personality disorder was blind or normal depending on the personality. At times she could see, but would go literally blind whenever she had one of the blind personalities.

All in all, the way we see the world is largely dependent on what our brain expects it to be, which leads us to today’s story.

Do AIs Learn What They Should?

This piece of research has proven that, while dreaming, rats modify their internal representation of direction.

Despite not moving due to muscle atonia (the muscles go into a state of temporary paralysis to avoid you reenacting your dreams), the rat’s brain actively modified the rat’s representation of heading (moving its head).

But what does that even mean?

In simple terms, without the rat moving, the brain simulated the movement, causing an internal reaction identical to the one that would have occurred if the rat actually moved.

Ok, so what? Well, this proves two things:

  • The brain’s influence on our reality is so significant that it can even simulate reality to the point our body physically reacts to brain simulations (think of the unharmed man on fentanyl).

  • Dreams may be simulation environments. Based on the results, the brain could be actively learning from dreams by modifying its own beliefs (representations) based on a simulation of reality.

And what does this all have to do with AI? Well, two things:

  • We are evaluating our progress in AI in an utterly wrong fashion

  • It hints at how humans developed intelligence and how machines should follow.

Measuring The Wrong Thing

Today, an AI model's intelligence is evaluated by the quality of its generations, not by the quality of its thoughts. In other words, we make a model say something, and we assess if that ‘something’ is ‘intelligent’ instead of evaluating where that intelligence action ‘came from.’

Nevertheless, based on this research, having high-quality internal representations of the world (a world model) is key. Thus, we should evaluate the intelligence of the thoughts that lead to those generations.

But what’s the difference? While a model outputting intelligent text may signal that it’s intelligent, it may also signal that the model has memorized that output.

Let’s see this with an example.

The way we evaluate AI’s intelligence today is like giving a kid a multiple-choice test and letting it take it in a room without your surveillance.

The only thing you know is that all answers to the test are in the answer sheet the kid has access to inside the room, but the kid won’t tell you whether he/she used it or not (because that would be cheating).

However, we can’t know whether the kid is cheating if he/she aced a multiple-choice test by copying the answer sheet or reasoning out every question.

However, as billions of dollars are invested into the idea that the kid—the LLM—is very intelligent, society chooses to trust the kid and move on. The kid is proclaimed ‘super intelligent’, and more billions come in.

If we were living in an honest world, we would test our LLMs in situations where cheating is not an option (we have done it already, and, surprise, they suck).

But why do they suck? Well, according to people like Yann LeCun, the representations built by these LLMs, the proper understanding they have developed of the world, are tenuous.

But how do we create models that learn strong representations?

One way is through reconstruction, which the researcher mentioned above proposed with JEPAs, a non-generative architecture that assesses how well a model can reconstruct images and videos.

Source: Meta

By forcing it to reconstruct partially observable images, the machine can only do so if it understands what the partially observable side represents. Using the image above, only if the model truly understands what a dog is can it reconstruct the rest of the image.

Importantly, it does all this in representation space, akin to how the rats in the research simulated actions inside their own brains without actually having to do them.

This is totally different from how LLMs learn. LLMs learn by generation, making them particularly sensitive to learning unnecessary stuff required as part of the generation. For example, for Sora to learn what a tree is, it needs to generate every single aspect of the tree, including the leaves.

On the other hand, a JEPA learns to reconstruct what a tree is like, just like a rat (or us) might simulate turning its head in his/her dreams. The point is, while a JEPA might need a generative model in case it wants to draw a tree, it does not require drawing the tree to learn what a tree is, just like you didn’t learn to draw trees to learn what they were. Therefore, it can stick to learning the key components of ‘what a tree is’ and delegate the drawing to a generative model.

In other words, in that scenario, the AI’s world model is a JEPA, and the generative model is merely a tool. In essence, LLMs learn nothing like humans do. However, sadly, absolutely no R&D capital is flowing into anything but Generative AI.

Thus, are we putting money in the wrong place? To make matters worse, we are missing an even bigger issue.

Dream Machines and Active Inference

Another component that frontier AI blatantly misses is life-long learning. They only learn during training, not afterward.

This goes against everything we’ve spoken about today. Our brain is in a never-ending learning loop, constantly predicting what will happen and adapting its predictions based on feedback from our senses (perception is another great missing piece in our AIs).

Unsurprisingly, AIs do none of that. They have read the entire Internet and hope for the best.

❝

In a way, the AI industry today is nothing but a group of people projecting their grandiose and economically incentivized beliefs onto naive investors with loads of money and few options other than AI to invest in.

However, LLMs being in the wrong direction in our path to building human-level intelligence doesn’t mean they don’t have tangible value. In fact, it could undoubtedly be the case that LLMs are the right path, but one that requires new breakthroughs like the ones below:

  1. Dream Machines: While actively learning as they interact with the world is too expensive, just like humans seem to use dreams to learn, we could create LLMs with dream states, where the AIs reenact their experiences of the day while you sleep and actively learn from them (fine-tuned based on the new data). The next day, your AI would be smarter based on the feedback ingested the day earlier.

For instance, in 2020, MIT researchers proposed Dreamcoder, a model with wake and sleep states.

  • During waking, the model performed tasks.

  • Then, during the sleep phase (divided into two distinct stages), the model would dream and reenact new tasks and actively learn from them, similar to how the dreaming rats updated their movement representations during their dreams.

This option would be considerably cheaper, as providers could use cheaper energy hours to perform the training. Notably, the memory requirements to serve these models wouldn’t skyrocket either.

  1. Active Inference: In a more extreme case, the AIs would actively learn like us. In other words, they would interact with the world somehow and use a reward model (similar to what AgentQ uses, which we saw last Sunday) to learn from that feedback.

But this scenario is far, far away due to the insane costs that would entail. All things considered, in my humble view, until we reach this phase we can’t dare say that we are getting any closer to AGI.

And without even considering the fact that we have a body dictates a great part of our intelligence (and what we learn, thanks to senses), we might be getting closer to creating smarter databases but not human-level intelligence, let alone AGI.

TheWhiteBox’s take:

Even though it wasn’t initially AI-related, I felt compelled to share the fascinating progress we are making in neuroscience. Could we be about to prove that the brain dreams on purpose to learn from what it has experienced during the day?

We probably don’t know yet, but the mere idea that it is true is worth sharing. Interestingly, it has also allowed us to realize, once again, how distant our efforts are from imitating the brain's functioning.

Thus, how panicked should we be? Should we trust people like François Chollet, a great proponent of active inference, as we discussed previously in this newsletter, that LLMs are pulling us away, not closer, from AGI?

How would markets react? Probably not great. Most of the money invested in AI is strictly related to LLMs. And adding insult to injury, that bet is, concerningly, extremely undiversified. Long story short, we are betting everything on the following:

  • One single architecture, the Transformer

  • One single hardware, the GPU

  • One single ML type, Generative AI

While the first two are the reasons behind almost all AI investments in the last two years, they’re all upended by the assumption that ‘Generative AI is the way.’

But as we’ve seen today… is it?

We won’t know until we see LLMs running on active inference workflows. But if that scenario doesn’t live up to its promise, there’s quite the case to be made that ‘GenAI is not the way.’

And that, my friend, is a multiple trillion-dollar error.

Don’t want to miss any of my content? Join Premium below!

What’s coming next? On Sunday, we will be looking at Big Tech’s Big Plays, the trillion-dollar moonshots of Big Tech. 

For business inquiries, reach me out at [email protected]