Benchmark Atlas

亚洲模型实验室 · arXiv 论文

Continuous vs Discrete Emergence

复现 Yi Figure 3:在同一 few-shot 任务上并列连续误差与离散 exact match,避免把测量阈值误当作能力突然出现。

yiemergencein-context-learningmeasurement
In-context learning · error and exact-match views复现 Yi Figure 3:在同一 few-shot 任务上并列连续误差与离散 exact match,避免把测量阈值误当作能力突然出现。 示意数据,仅复现信息结构,不代表最新榜单结果。 In-context learning · error and exact-match viewsYi Technical Report · Figure 3 / PDF p.11DEMO DATA2 COEFFICIENTS · [1, -1]shots →solid: error ↓ · dashed: exact match ↑5 COEFFICIENTS · [1, 1, 1, 1, 1]shots →solid: error ↓ · dashed: exact match ↑continuous error reveals gradual learning before discontinuous exact-match emergence
Illustrative demo data · renderer: continuousDiscreteEmergence

Information grammar

它如何组织信息

任务难度分行 + 连续误差实线 + exact-match 虚线 + shots 共享横轴

Visual system
VS-78 Yi continuous-discrete emergence
Component ID
yi-continuous-discrete-emergence

Use when

适用判断

适合研究 emergent ability 与测量伪影;连续、离散指标必须来自同一预测样本。

Figure-level evidence

Figure 3 · PDF p.11

两个线性系数推断任务,比较 difference-to-target 与 exact match 随 shots 变化。

Verified
2026-07-13
Paper source
Yi Technical Report

Structured data

可替换的数据模型

示意数据,仅复现信息结构,不代表最新榜单结果。

{
  "shots": [
    4,
    8,
    12,
    16,
    20,
    24,
    28,
    32,
    36,
    40
  ],
  "tasks": [
    {
      "label": "2 COEFFICIENTS · [1, -1]",
      "continuousRange": [
        0,
        50
      ],
      "exactMax": 0.6,
      "models": [
        {
          "label": "Yi-34B",
          "continuous": [
            43,
            35,
            28,
            20,
            16,
            18,
            14,
            12,
            8,
            10
          ],
          "exact": [
            0.08,
            0.28,
            0.3,
            0.38,
            0.36,
            0.28,
            0.45,
            0.32,
            0.47,
            0.26
          ]
        },
        {
          "label": "Llama-2-70B",
          "continuous": [
            46,
            39,
            37,
            33,
            30,
            28,
            26,
            24,
            23,
            22
          ],
          "exact": [
            0.02,
            0.1,
            0.2,
            0.35,
            0.42,
            0.45,
            0.4,
            0.38,
            0.46,
            0.55
          ]
        },
        {
          "label": "Yi-6B",
          "continuous": [
            49,
            42,
            39,
            36,
            31,
            22,
            20,
            25,
            18,
            17
          ],
          "exact": [
            0,
            0.02,
            0.04,
            0.08,
            0.1,
            0.16,
            0.06,
            0.14,
            0.08,
            0.12
          ]
        }
      ]
    },
    {
      "label": "5 COEFFICIENTS · [1, 1, 1, 1, 1]",
      "continuousRange": [
        0,
        170
      ],
      "exactMax": 0.8,
      "models": [
        {
          "label": "Yi-34B",
          "continuous": [
            150,
            54,
            45,
            41,
            38,
            47,
            36,
            34,
            37,
            39
          ],
          "exact": [
            0,
            0,
            0.01,
            0.02,
            0.01,
            0.03,
            0.02,
            0.04,
            0.01,
            0.03
          ]
        },
        {
          "label": "Llama-2-70B",
          "continuous": [
            60,
            14,
            8,
            5,
            10,
            7,
            6,
            11,
            8,
            6
          ],
          "exact": [
            0.12,
            0.44,
            0.56,
            0.78,
            0.56,
            0.74,
            0.66,
            0.72,
            0.68,
            0.76
          ]
        },
        {
          "label": "Yi-6B",
          "continuous": [
            165,
            83,
            76,
            57,
            45,
            39,
            34,
            36,
            29,
            31
          ],
          "exact": [
            0,
            0,
            0.02,
            0,
            0.03,
            0,
            0.02,
            0.01,
            0.03,
            0.01
          ]
        }
      ]
    }
  ]
}

Related components

同一图表家族的其他语法