A small bird is trying to lift an elephant. That is roughly the first\nimpression one gets from reading the story of Colibr\u00ec, a software\nproject that enables a budget-class computer to run a giant AI brain\nsuch as GLM-5.2.<\/p>\n
The AI brain called GLM-5.2 is built from roughly 744 billion\nparameters \u2014 numbers learned during training that determine how the AI\nrecognises patterns, understands questions and composes answers. Even\nafter compression, its file size remains around 370 GB.<\/p>\n
Remarkably, however, Colibr\u00ec can run it on a 12-core laptop with only\n25 GB of RAM. It does not load the entire giant parameter set into the\ncomputer\u2019s RAM at once. It only fetches the parts currently needed,\nwhile the rest stays stored on an SSD or NVMe drive and is called up\nonly when its turn comes to work.<\/p>\n
For now, Colibr\u00ec is indeed only carrying the \u201celephant\u201d known as\nGLM-5.2. This AI brain uses an approach called Mixture-of-Experts (MoE).\nIt can be likened to a company with 744,000 employees, yet each task\nonly requires around 40,000 people whose expertise matches the job.<\/p>\n
The remaining 704,000 need not crowd into the office; they can wait\nuntil their expertise is required. That is the gap Colibr\u00ec exploits. In\nthe conventional way, running GLM-5.2 demands a computer with\nextraordinary memory or several top-class graphics cards that are\ncertainly not priced for the budget market.<\/p>\n
Colibr\u00ec chose a different path: what is currently working goes into\nRAM, while what is not yet needed waits on the SSD. The technical\ndetails are naturally more complicated. What is certain is that the\nColibr\u00ec experiment proves the impossible can become possible through\ncreativity and ingenuity in solving a problem.<\/p>\n
The name Colibr\u00ec itself feels like a metaphor almost too perfect to\nbe a coincidence. In Italian, colibr\u00ec means hummingbird, a tiny creature\nfrom the Americas with unusual flying abilities. It can hover, spin,\ndart, and even fly backwards.<\/p>\n
Its small body requires a constant supply of energy from flower\nnectar. The hummingbird does not become mighty by growing its body like\nan eagle. It overcomes its limitations through agility and a different\nway of moving.<\/p>\n
So it is with the software bearing its name. It does not confront a\ngiant AI model with an equally giant computer. It moves between RAM,\nVRAM and the computer\u2019s storage, taking what is needed, leaving what is\nnot yet required, then shifting again.<\/p>\n
It does not defeat weight with strength, but with movement. It does\nnot match enormity with enormity, but with ingenuity.<\/p>\n
Here lies Colibr\u00ec\u2019s broader lesson: limitations do not always have to\nbe answered by adding power; sometimes they instead force us to find new\nways through creativity.<\/p>\n
We live in an era fond of solving shortages through additions. Slow\ncomputer? Add more RAM. AI model doesn\u2019t fit? Buy a GPU. Server not\npowerful enough? Buy a new server. Company short on capacity? Increase\nthe budget. None of that is wrong, especially if there is money to buy\nit.<\/p>\n
Colibr\u00ec was born precisely because its creator, Vincenzo, known as\nJustVugg, did not possess a machine considered worthy of running a model\nthat large. He could simply have accepted his fate: a\n744-billion-parameter model is indeed not for a laptop like mine. End of\nstory.<\/p>\n
But Vincenzo did not stay still. He posed a different question: which\nparts of such a huge model are truly needed at any given moment? From\nthat question, a seemingly solid wall began to have a door.<\/p>\n
In the history of technology, doors like that are often found by\nthose who lack the money to buy a new wall. Limitation indeed has two\nfaces: an excuse to give up, or the mother of creativity.<\/p>\n
For creativity is often not the ability to make something more\ncomplicated, but rather to find a simple path unseen because we are too\naccustomed to the expensive one.<\/p>\n
For us in Indonesia, this lesson feels close to home. Not every\nschool, Islamic boarding school, university, library, law office,\ncommunity health centre or research institute can afford an AI server\nworth hundreds of millions to billions of rupiah.<\/p>", "url": "https:\/\/jawawa.id\/newsitem\/colibri-lifts-the-elephant-how-a-tiny-program-runs-giant-ai-on-a-laptop-1785077468", "image": "" }, "sponsor": "Okusi Associates", "sponsor_url": "https:\/\/okusiassociates.com" }