Ternary neural net in C# — a BitNet 1.58 analog weighing 29 KB
· Source: original
⚖️ A ternary analog of BitNet 1.58 in C#: 29 KB, and 17.1 KB in the extreme variant
On the left is Microsoft BitNet 1.58, on the right is a neural network based on ternary (trit) logic, written in C#. The weight is 29 KB; the extreme build shrinks to 17.1 KB.
The key difference lies in the approach to weights: here the entire training path has been traversed, from FP32 to FP4, all the way to ternary logic.
The main idea: the craze for quantization will end when mass-produced models trained on ternary/trit logic from the very start appear.
Relevance assessment date for the material — September 13, 2026. A breakdown of formats from FP32 to FP4 is in the article on habr.com