Indonesian Political, Business & Finance News

Colibrì Lifts the Elephant: How a Tiny Program Runs Giant AI on a Laptop

| | Source: REPUBLIKA Translated from Indonesian | Technology
Colibrì Lifts the Elephant: How a Tiny Program Runs Giant AI on a Laptop
Image: REPUBLIKA

A small bird is trying to lift an elephant. That is roughly the first impression one gets when reading about Colibrì, a software project that enables a computer of modest specifications to run a giant AI brain like GLM-5.2. The AI brain named GLM-5.2 is composed of around 744 billion parameters—figures resulting from learning that determine how the AI recognises patterns, understands questions, and formulates answers. Even after compression, the file size is still around 370 GB. Miraculously, Colibrì can run it on a 12-core laptop with only 25 GB of RAM. It does not load the entire giant parameter set into the computer’s RAM at once. It only retrieves the parts currently needed, while the rest remain stored on the SSD or NVMe and are only called upon when their turn to work arrives. For now, Colibrì is only carrying the ‘elephant’ known as GLM-5.2. This AI brain uses a method called Mixture-of-Experts (MoE). It can be likened to a company with 744,000 employees, but each task only requires about 40,000 people with the appropriate expertise. The remaining 704,000 do not need to crowd into the office; they can wait until their skills are needed. That is the gap Colibrì exploits. In the conventional method, running GLM-5.2 requires a computer with extraordinarily large memory or several high-end graphics cards, which are certainly not priced for the budget-conscious. Colibrì chooses a different path: what is currently working enters the RAM, and what is not yet needed waits on the SSD. The technical details are, of course, more complex. What is certain is that the Colibrì experiment proves that the impossible can become possible through creativity and ingenuity in solving a problem. The name Colibrì itself feels like a metaphor too perfect to be a coincidence. In Italian, colibrì means hummingbird, a tiny creature from the Americas with unusual flying abilities. It can hover, spin, dart, and even fly backwards. Its small body requires a constant supply of energy from flower nectar. The hummingbird does not become mighty by enlarging its body like an eagle. It overcomes its limitations with agility and a different way of moving. So too is the software that bears its name. It does not fight a giant AI model with an equally giant computer. It moves between RAM, VRAM, and storage on the computer, taking what is needed, leaving what is not yet required, and then moving on. It does not defeat weight with strength, but with motion. It does not match greatness with greatness, but with cleverness. Herein Colibrì offers a broader lesson: limitations do not always have to be answered by adding power; sometimes they force us to find new ways through creativity. We live in an era that loves to solve deficiencies with additions. A slow computer? Add RAM. An AI model does not fit? Buy a GPU. A server is not powerful enough? Buy a new server. A company lacks capacity? Increase the budget. None of that is wrong, especially if you have the money to buy it. Colibrì was born precisely because its creator, Vincenzo, known as JustVugg, did not have a machine deemed suitable to run such a large model. He could have accepted his fate: a 744-billion-parameter model is simply not in my laptop’s league. End of story. But Vincenzo did not give up. He asked a different question: which part of this giant model is actually needed at any given moment? From that question, a door appeared in what seemed like a solid wall. In the history of technology, such doors are often found by those who do not have enough money to buy a new wall. Limitation indeed has two faces: it can be a reason to surrender, or it can be the mother of creativity. For creativity is often not the ability to make something more complicated, but rather finding a simple path that was invisible because we were too accustomed to the expensive one. For us in Indonesia, this lesson feels close to home. Not every school, Islamic boarding school, university, library, law office, community health centre, or research institution can afford an AI server worth hundreds of millions to billions of rupiah.

View JSON | Print