RAM Hoarders in Panic Facing Google's AI Breakthrough...

KOMPAS.com - RAM prices have risen dramatically over the past year, caused by supply shortages due to priorities for the Artificial Intelligence (AI) industry. However, in some regions, RAM prices have been observed to fall recently.

It turns out that this decline is due to RAM hoarders being in a panic and releasing the stocks they have stored. Upon investigation, they are panicking in the face of Google’s technological innovation that can make memory usage in AI programmes far more efficient.

There has been a change in behaviour among memory distributors, particularly in China. Previously, these distributors had hoarded large quantities of RAM when prices were high.

After the announcement of Google’s breakthrough launch, these hoarders are reported to have begun “clearing their warehouses” or releasing their RAM stocks onto the market.

They are panicking over the potential decline in RAM demand from large-scale AI data centres (hyperscalers) if Google’s AI technology is widely adopted, leading to a drastic drop in global memory prices.

The hoarders’ warehouse-clearing action due to Google’s breakthrough has contributed to the RAM price decline in some regions, although it has not yet spread evenly globally.

The Google AI technology causing panic among RAM hoarders is TurboQuant. This technology is essentially an AI-based memory compression algorithm. TurboQuant can significantly save memory usage in AI.

TurboQuant was developed by the company’s research division, Google Research, with a main focus on efficiency in memory usage during the inference process, that is, when the AI model is run, not trained.

This technology targets one of the main bottlenecks in modern AI systems, namely the limitation of “working memory”, particularly in the component called KV cache, which is temporary memory used by the model to process and remember data context.

TurboQuant’s method relies on a technique called vector quantisation, which is a method of simplifying numerical data representation in vector form to make it more compact without losing important information.

With this approach, data that previously required large space can be compressed significantly, while still maintaining the AI model’s accuracy.

Technically, TurboQuant relies on two main methods: PolarQuant and Quantisation-aware Joint Learning (QJL).

Meanwhile, QJL trains the AI model to be “aware” that the processed data will be compressed, so the model can adapt and still produce accurate outputs even when working with compacted data.