Viettel AI Race: Optimizing AI Inference
Key Facts
- Category: Competitions
- Deadline: N/A
- Format: Online
- Student Level: University Students
Summary
The Viettel AI RACE competition focuses on the problem of optimizing LLM model inference on resource-constrained infrastructure, aimed at students with a background in programming and AI.
Details
Introduction
Viettel AI Race is a competition focused on optimizing AI inference on resource-constrained infrastructure. According to the article, participants must deploy a 1.2B parameter LLM model on a configuration consisting of only 18GB VRAM, 3 vCPUs, and 8GB RAM, with the requirement to handle 420 requests and 70 concurrent conversations.
The problem is scored based on the ERS metric, where both TTFT and TPOT affect the score. The article also indicates that the competition is highly competitive, with a baseline score of around 42 ERS and the threshold for the top 8 being approximately 72 ERS.
Participants
- Students interested in AI, programming, and system optimization
- Participants can join individually or in teams, as described in the article
Requirements
- Solve the problem of serving a 1.2B LLM model on limited resources
- Only use vLLM; do not interfere with weights or tokenizers
- Adhere to a maximum limit of 5 attempts per participant per day
Important Dates
- The article was published on 2026-08-01
- Participants in the article mentioned trying for 7 days and submitting over 35 times
- The top 8 threshold mentioned is a minimum of 72 ERS