Yandex AliceAI-Foundation-80B-A3B-Base
· Source: original
Inference of AliceAI-Foundation-80B-A3B-Base: minimum 4×A100
🛠️
The task is to deploy inference for Yandex AliceAI-Foundation-80B-A3B-Base: a pretrain with 80B parameters in the A3B configuration. The weights are on HuggingFace, Yandex's release publication is on Habr (EN), and the breakdown of the inference budget is on Habr.
The minimum for inference is 4×A100. The model will also run on a couple of GPUs, but only with CPU offload.
Quantization is an unacceptable compromise: the author is not willing to go for it. The GPUs were planned to be rented in the cloud — the name is not specified in the text.
What didn't go according to plan. The first attempt — Datasphere on Jupiter Notebook — failed and cost 6000₽. A VM with 4×A100 was not granted: the quotas could not be raised via tickets, and for the Gen2 configuration the quota isn't even raised. The result — a return to Datasphere Jobs, because A100s were allocated for Jupiter Notebook.
The breakdown includes a screenshot and a budget calculation
🛠️ Want to apply this in practice?
Inspired by the news — a selection of prompts for developers from our library:
- Codebase Ecosystem Atlas
- Comprehensive Go Codebase Review - Forensic-Level Analysis Prompt
- Quality Engineering Agent Role
🔗 The entire prompt library · "Programming" category
A ready-made product on the topic: AI Content Factory: an autonomous content generation and publishing system — grab it and apply it right away.