NEC Develops High-speed Generative AI Large Language Models (LLM) with World-class Performance: Press Releases

Tokyo, Japan, April 24, 2024 — NEC Corporation (NEC; TSE: 6701) has expanded its “NEC cotomi” generative AI services with the development of “NEC cotomi Pro” and “NEC cotomi Light,” two new high-speed generative AI Large Language Models (LLM) featuring updated training data and architectures.

With the rapid development of generative AI in recent years, a wide range of organizations have been considering and verifying business transformation using LLMs. As specific application scenarios emerge, there is a need to provide models and formats that meet customer needs in terms of response time, business data coordination, information protection and other security aspects during implementation and operation.

NEC’s newly developed NEC cotomi Pro and NEC cotomi Light are high-speed, high-performance models that deliver the same high performance as global LLMs, but at more than ten times the speed.

Generally, to improve the performance of an LLM, a model needs to be made larger, but this slows down the operating speed. However, NEC has succeeded in improving both speed and performance with the development of an advanced new training method and architecture.

“NEC cotomi Pro” achieves performance comparable to top-level global models such as “GPT-4” and “Claude 2,” with a response time that is approximately 87% faster than GPT-4 using an infrastructure of two graphics processing units (GPU). In addition, the even faster “NEC cotomi Light” has the same level of performance as global models such as “GPT-3.5-Turbo,” but can process a large number of requests at high speed with an infrastructure of about 1 to 2 GPU, providing sufficient performance for many tasks.

Specifically, in an in-house document retrieval system using a technique called RAG, the system achieved a correct response rate higher than GPT-3.5 without fine-tuning and a correct response rate higher than GPT-4 after fine-tuning, with a response time that is approximately 93% faster.