Benchmark Atlas

亚洲模型实验室 · arXiv 论文

Loss-to-Benchmark Transfer Calibration

复现 Seed1.5-VL Figure 3:把 OCR/grounding 子类 loss 的 token 缩放与相关下游评测指标的局部校准关系串成两段证据链。

seedvlmscaling-lawcalibrationtransfer
Training tokens → subcategory loss → downstream metrics复现 Seed1.5-VL Figure 3:把 OCR/grounding 子类 loss 的 token 缩放与相关下游评测指标的局部校准关系串成两段证据链。 示意数据,仅复现信息结构,不代表最新榜单结果。 Training tokens → subcategory loss → downstream metricsSeed1.5-VL Technical Report · Figure 3 / PDF p.14DEMO DATAloss vs training tokenslog tokens →ChartQA vs -log loss-log OCR loss →InfoVQA vs -log loss-log OCR loss →OCRloss vs training tokenslog tokens →RefCOCO vs -log loss-log grounding loss →RefCOCO+ vs -log loss-log grounding loss →GROUNDINGtraining tokens → subcategory loss → multiple related downstream metrics · local calibration only
Illustrative demo data · renderer: lossMetricTransferFacets

Information grammar

它如何组织信息

能力子类分行 + token-loss 幂律首面板 + 同一 loss 映射多个下游指标的局部 log-linear 双面板 + 因果阅读箭头

Visual system
VS-81 Seed linked loss-transfer facets
Component ID
seed-loss-metric-transfer

Use when

适用判断

适合用训练代理指标预测相关 benchmark;只应在论文支持的局部 loss 区间内解释,不外推到饱和区。

Figure-level evidence

Figure 3 · PDF p.14

六分面分别连接 OCR/grounding loss 的 token 幂律与 ChartQA、InfoVQA、RefCOCO、RefCOCO+ 指标。

Verified
2026-07-13
Paper source
Seed1.5-VL Technical Report

Structured data

可替换的数据模型

示意数据,仅复现信息结构,不代表最新榜单结果。

{
  "rows": [
    {
      "label": "OCR",
      "panels": [
        {
          "label": "loss vs training tokens",
          "axis": "log tokens →",
          "points": [
            [
              0.04,
              0.92
            ],
            [
              0.15,
              0.82
            ],
            [
              0.28,
              0.77
            ],
            [
              0.4,
              0.64
            ],
            [
              0.54,
              0.55
            ],
            [
              0.65,
              0.43
            ],
            [
              0.76,
              0.32
            ],
            [
              0.9,
              0.16
            ],
            [
              0.98,
              0.08
            ]
          ]
        },
        {
          "label": "ChartQA vs -log loss",
          "axis": "-log OCR loss →",
          "points": [
            [
              0.08,
              0.12
            ],
            [
              0.2,
              0.24
            ],
            [
              0.32,
              0.3
            ],
            [
              0.46,
              0.48
            ],
            [
              0.62,
              0.55
            ],
            [
              0.76,
              0.71
            ],
            [
              0.91,
              0.84
            ]
          ]
        },
        {
          "label": "InfoVQA vs -log loss",
          "axis": "-log OCR loss →",
          "points": [
            [
              0.06,
              0.1
            ],
            [
              0.22,
              0.2
            ],
            [
              0.38,
              0.36
            ],
            [
              0.54,
              0.47
            ],
            [
              0.68,
              0.64
            ],
            [
              0.82,
              0.78
            ],
            [
              0.95,
              0.86
            ]
          ]
        }
      ]
    },
    {
      "label": "GROUNDING",
      "panels": [
        {
          "label": "loss vs training tokens",
          "axis": "log tokens →",
          "points": [
            [
              0.02,
              0.94
            ],
            [
              0.17,
              0.85
            ],
            [
              0.31,
              0.71
            ],
            [
              0.46,
              0.62
            ],
            [
              0.6,
              0.48
            ],
            [
              0.73,
              0.37
            ],
            [
              0.86,
              0.23
            ],
            [
              0.98,
              0.1
            ]
          ]
        },
        {
          "label": "RefCOCO vs -log loss",
          "axis": "-log grounding loss →",
          "points": [
            [
              0.08,
              0.16
            ],
            [
              0.22,
              0.28
            ],
            [
              0.36,
              0.25
            ],
            [
              0.49,
              0.52
            ],
            [
              0.63,
              0.58
            ],
            [
              0.77,
              0.73
            ],
            [
              0.93,
              0.9
            ]
          ]
        },
        {
          "label": "RefCOCO+ vs -log loss",
          "axis": "-log grounding loss →",
          "points": [
            [
              0.07,
              0.12
            ],
            [
              0.22,
              0.3
            ],
            [
              0.37,
              0.36
            ],
            [
              0.51,
              0.49
            ],
            [
              0.66,
              0.64
            ],
            [
              0.8,
              0.76
            ],
            [
              0.95,
              0.92
            ]
          ]
        }
      ]
    }
  ]
}

Related components

同一图表家族的其他语法