Sixty-six percent of companies worldwide said they would invest in genAI over the next 18 months. according to IDC research. Among organizations that indicate they will increase IT spending for genAI in 2024, infrastructure will account for 46% of total spending.
The problem: A key piece of hardware needed to build that AI infrastructure is in short supply. While GPUs are in high demand to run the most massive applications great language models (LLM) behind genAI, the market still needs high-performance memory chips for AI applications. The market is tight for both, for now.
GPUs used for training and inference tasks in LLM can consume large amounts of processor cycles and be expensive to use. Smaller, more industry- or business-focused models can often provide better results tailored to business needs and can use common x86 processors with NPUs.
“While much of the focus is on using high-performance GPUs for new AI workloads, the major hyperscalers (AWS, Google, Meta and Microsoft) are investing in developing their own AI-optimized chips “said Priestley.
While chip development is expensive, the use of custom-designed chips can improve operational efficiency, reduce the costs of providing AI-based services to users, and reduce costs for users to access new AI-based applications. AI, according to Priestley.
“As the market moves from development to deployment, we expect this trend to continue,” Priestley said.
Leave feedback about this