Skip to content

Eco-efficiency methodology

The 1-to-5 star energy rating answers one question: how much energy does this model burn? It says nothing about how capable the model is. A tiny model can earn five stars by being weak and cheap to run; a frontier model can earn one star while being far more useful per unit of energy.

The eco-efficiency score closes that gap. It measures capability per unit of environmental cost: how much useful output a model delivers for the energy, water, and carbon it consumes. It is a distinct, separate measure from the energy-only star rating, and the two are never conflated.

The method is reimplemented from Section 4.5 of How Hungry is AI? (CC BY 4.0; see licenses).

Inputs and output

Eco-efficiency is computed with Data Envelopment Analysis (DEA) using cross-efficiency weighting. DEA frames each model as converting environmental inputs (the cost) into a capability output (the benefit).

Inputs (the environmental cost):

Input Unit
Per-query energy Wh
PUE dimensionless
WUE_site L/kWh
WUE_source L/kWh
Carbon intensity factor (CIF) kgCO2e/kWh

Per-query energy comes from the energy methodology; PUE, WUE, and CIF come from the versioned provider reference data described in the water methodology.

Output (the capability): the AI-Index composite described below.

The AI-Index capability composite

The AI-Index is the single capability output of the DEA. It is a weighted average of seven public benchmark scores with fixed weights, the composite that How Hungry is AI? uses as its DEA output (its “Artificial Intelligence Index”, built from benchmark results published by Artificial Analysis):

Capability area Area weight Benchmarks (weight each)
Reasoning and knowledge 50% MMLU-Pro, Humanity’s Last Exam (HLE), GPQA (1/6 each)
Math 25% MATH-500, AIME (1/8 each)
Coding 25% SciCode, LiveCodeBench (1/8 each)

The weights are fixed (50 / 25 / 25) and stored as governed constants, so the composite is reproducible and comparable across the catalog rather than tuned per model. A model missing any of the seven benchmark scores has a null AI-Index; Venturi never re-normalizes a partial set of scores.

The AI-Index and the benchmark scores behind it are DEA inputs only. Because the benchmark results are Artificial Analysis data, Venturi publishes neither the scores nor the AI-Index; a catalog version publishes only the resulting eco_efficiency_score, with the attribution described in licenses.

Score availability is a property of each catalog version. The active public catalog, 2026-06-13.ap2, publishes no eco-efficiency score: its eco_efficiency_score and ai_index fields are null on all 51 rows. Version history states the score availability of each published version.

Why cross-efficiency

Plain DEA lets each model choose the input and output weights that flatter it most (self-evaluation). Cross-efficiency instead scores each model under its own optimal weights and under the optimal weights of every peer, then averages the results, which reduces self-evaluation bias and produces a fairer ranking. This is the formulation described in Section 4.5 of How Hungry is AI?, in contrast to the standard Charnes-Cooper-Rhodes (CCR) and Banker-Charnes-Cooper (BCC) models. The result is a unitless eco_efficiency_score: higher is more capability per environmental cost.

Because each score is computed against the weights of the other models in the same catalog version, the score is relative to that peer set: it can change when models enter or leave the catalog even if the model’s own inputs did not change. The score carries no standard deviation of its own.

Null, never zero

eco_efficiency_score is null whenever any input is null: if energy is unknown, or a water, carbon-intensity, or benchmark term is missing, the score is not computed rather than fabricated. This is the same null-not-zero discipline applied to energy and carbon and water. In a ranking, a model with no score sits in a separate unscored group; it is never ranked as least efficient.

Distinct from the star rating

To be explicit about the two scores, because they are easy to confuse:

Compared on Energy star rating Eco-efficiency score
Question answered How much energy does it burn? How much capability per environmental cost?
Inputs Per-task energy only Energy + PUE + water + carbon-intensity vs. capability
Scale 1 to 5 stars (relative quintiles, per task) Unitless DEA cross-efficiency score
Method 20% quintile bands at render time Data Envelopment Analysis (cross-efficiency)
Comparability Within one task (and, for text generation, one size class) and one catalog version Among the models scored in the same catalog version

A model can be five-star on energy and middling on eco-efficiency, or the reverse. Read each for what it measures.

Where to go next

Give feedback on this page

Sign in to send feedback. Opens in a new tab.

Copy page details 
Docs feedback: Eco-efficiency methodology
Page: https://docs.venturi.systems/energy/eco-score-methodology/