亚洲模型实验室 · arXiv 论文
Kimi Sparsity Scaling Law
复现 Kimi K2 Figure 5：固定激活专家数、改变总专家数，比较不同 sparsity 下的训练缩放。
sparsityScalingCurvesInformation grammar
它如何组织信息
多 sparsity 训练轨迹 + log FLOPs 横轴 + 每系列虚线缩放拟合
- Visual system
- VS-61 Kimi sparsity trajectories
- Component ID
kimi-sparsity-scaling
Use when
适用判断
适合 MoE 稀疏度实验；横轴 compute 和激活专家配置必须一致。
Figure-level evidence
Figure 5 · PDF p.7
固定激活专家数，以总专家数变化比较 sparsity scaling law。
- Verified
- 2026-07-12
- Paper source
- Kimi K2 Technical Report
Structured data
可替换的数据模型
示意数据，仅复现信息结构，不代表最新榜单结果。
{
"series": [
{
"sparsity": 8,
"points": [
[
110000000000000000000,
1.78
],
[
220000000000000000000,
1.66
],
[
430000000000000000000,
1.55
],
[
850000000000000000000,
1.43
],
[
1.1e+21,
1.39
]
]
},
{
"sparsity": 16,
"points": [
[
100000000000000000000,
1.73
],
[
200000000000000000000,
1.62
],
[
400000000000000000000,
1.51
],
[
800000000000000000000,
1.4
],
[
1.05e+21,
1.35
]
]
},
{
"sparsity": 32,
"points": [
[
100000000000000000000,
1.69
],
[
190000000000000000000,
1.58
],
[
380000000000000000000,
1.48
],
[
750000000000000000000,
1.37
],
[
1e+21,
1.31
]
]
},
{
"sparsity": 42,
"points": [
[
120000000000000000000,
1.7
],
[
230000000000000000000,
1.59
],
[
460000000000000000000,
1.49
],
[
870000000000000000000,
1.38
],
[
1.08e+21,
1.32
]
]
},
{
"sparsity": 64,
"points": [
[
110000000000000000000,
1.68
],
[
210000000000000000000,
1.57
],
[
420000000000000000000,
1.46
],
[
820000000000000000000,
1.36
],
[
1.02e+21,
1.3
]
]
}
]
}Related components