GEH statistic

The GEH statistic, named after transport planner Geoffrey E. Havers, is an empirical measure developed in the 1970s in the United Kingdom for traffic engineering to evaluate the goodness-of-fit between observed field data and modeled estimates of traffic volumes, particularly in calibration and validation of microsimulation models.^[1] It provides a dimensionless index that balances absolute and relative differences, making it suitable for comparing flows across varying magnitudes without undue bias toward high-volume links.^[2] The formula for the GEH statistic is given by

\text{GEH} = \sqrt{\frac{2(E - V)^2}{E + V}}

where E represents the model-estimated volume and V the observed field count, typically in vehicles per hour.^[2] This expression, mathematically akin to a chi-squared test but empirically derived for traffic data, yields values closer to zero for better agreement; for instance, it penalizes discrepancies more severely at higher flows while allowing broader relative tolerances at low volumes.^[3] In practice, the GEH statistic is applied to individual links, ramps, and aggregate flows in freeway and arterial network models, with acceptability thresholds varying by jurisdiction—for example, GEH < 5 for at least 85% of links and GEH < 4 for total flows in U.S. Federal Highway Administration guidelines, or GEH < 3 for state facilities in Texas Department of Transportation criteria.^[2]^[1] While widely adopted in software like VISSIM and Aimsun for its simplicity and robustness, it has faced criticism for non-linearity and insensitivity to certain error types, prompting alternatives like the SQV statistic.^[3]^[4]

Background

Origin and History

The GEH statistic derives its name from Geoffrey E. Havers, a transport planner who developed it in the 1970s while working for the Greater London Council in England. Havers proposed the measure as a heuristic tool tailored to the challenges of traffic engineering in urban settings, particularly in London, where accurate comparisons between real-world data and predictive models were essential for infrastructure planning.^[5] The initial purpose of the GEH statistic was to evaluate the goodness-of-fit between observed traffic volumes and those generated by simulation models, under the assumption that traffic flows follow a Poisson-like distribution where variance approximates the mean.^[3] This approach addressed limitations in traditional metrics like percentage differences, which could be misleading for low-volume links, by incorporating both relative and absolute discrepancies in a balanced manner suitable for hourly flow data.^[3] Early adoption of the GEH statistic occurred through its integration into official guidelines by the UK Highways Agency, which included it in the Design Manual for Roads and Bridges (DMRB) for validating highway assignment models. This endorsement helped standardize its use in traffic forecasting and engineering practices across the UK, establishing it as a benchmark for model calibration in public sector projects.^[6]

Mathematical Formulation

The GEH statistic is mathematically formulated as

\text{GEH} = \sqrt{\frac{2(M - C)^2}{M + C}},

where M denotes the modeled traffic volume and C the observed (counted) traffic volume, both typically expressed in vehicles per hour.^[3] This expression incorporates the squared difference (M - C)^2 to quantify deviation, normalized by the sum M + C in the denominator, which balances absolute and relative discrepancies between the two volumes.^[3] The square root provides a scale akin to a standard deviation, while the factor of 2 derives from the Poisson index of dispersion, under the assumption that traffic flows follow a quasi-Poisson distribution where variance is proportional to the mean.^[3] When applied to hourly flows in vehicles per hour, the resulting GEH value has units of (\text{vehicles per hour})^{1/2}, rendering it not strictly unitless but effectively scale-invariant for comparative purposes across links.^[3]

Applications and Interpretation

Primary Uses in Traffic Modeling

The GEH statistic serves as a key goodness-of-fit measure in traffic modeling, enabling practitioners to evaluate the alignment between modeled and observed traffic volumes by balancing absolute and relative differences. It is particularly valuable in scenarios requiring precise comparisons of traffic data sets, supporting the refinement and reliability of models used in transportation planning and engineering.^[2] One primary application involves comparing manual traffic counts with automated counts to ensure data consistency across collection methods. For instance, in real-time video-based vehicle detection systems, GEH quantifies the accuracy of algorithm-derived volumes against manual observations, with values below 5 indicating strong agreement and validating the automated approach for practical deployment in urban traffic monitoring.^[7] GEH is routinely employed to validate travel demand forecasting models against base-year observed data, assessing how well simulated traffic patterns replicate historical counts at key locations. This process helps confirm the model's ability to represent real-world travel behavior before applying it to future scenarios, such as infrastructure impact assessments. In traffic simulation models, GEH facilitates parameter adjustment during calibration, as outlined in guidelines for analytical travel forecasting. By iteratively minimizing GEH values between simulated and field-measured flows, modelers refine inputs like origin-destination matrices and route choices to achieve acceptable replication of traffic conditions.^[8] Additionally, GEH assesses flow accuracy in highway assignment models, particularly at individual links or screenlines, to gauge the overall quality of traffic distribution outputs. This use supports the evaluation of assignment procedures in large-scale networks, ensuring modeled link volumes align closely with observed data from automatic traffic counters.^[3]

Thresholds and Quality Criteria

The GEH statistic is interpreted through standardized thresholds that classify the agreement between modeled and observed traffic volumes. A GEH value below 5.0 is generally considered indicative of a good match, while values between 5.0 and 10.0 suggest that further investigation is needed to identify potential discrepancies, and values exceeding 10.0 signal poor performance requiring model adjustments.^[9]^[3] In practice, the GEH is typically applied to hourly traffic flows, such as peak-hour volumes in vehicles per hour, to ensure comparability with observed data. Aggregate quality is often assessed by calculating the percentage of individual links or observation points meeting the thresholds, with criteria such as at least 85% of links achieving GEH < 5.0 commonly used for overall model validation.^[9]^[2]^[3] Adjustments to these thresholds may be applied in context-specific scenarios, particularly for low-volume links where the statistic inherently allows higher tolerance due to greater relative variability in counts, as the formula weights absolute and percentage differences to accommodate Poisson-like distributions in sparse data.^[10]^[3]

Limitations

Key Criticisms

One key practical drawback of the GEH statistic is its dependency on the magnitude of traffic volumes, which imposes stricter relative tolerances on higher absolute flows compared to lower ones. This scale sensitivity arises because the formula balances absolute and relative errors in a way that requires smaller percentage discrepancies for large-volume links to achieve acceptable GEH values, rendering it less suitable for direct comparisons between datasets with differing scales, such as hourly versus daily traffic flows.^[3] The GEH statistic exhibits asymmetry in its treatment of predictions, particularly in contexts influenced by capacity constraints, where over-predictions and under-predictions may not be penalized equivalently due to underlying distributional skews in traffic counts. This can introduce bias in model validation, as the metric may undervalue errors in congested scenarios where traffic volumes are censored by road capacity, leading to uneven assessments of model performance across diverse network conditions.^[3] A notable lack of standardization in GEH application stems from the absence of universal guidelines for aggregating results across spatial networks or temporal periods, such as screenlines or multi-hour totals. While thresholds like GEH < 5 are commonly applied to individual links, extending the metric to aggregated levels often yields inconsistent variance patterns, complicating network-wide evaluations and potentially misrepresenting overall model fit.^[3]^[11] Furthermore, the GEH statistic tends to overemphasize accuracy in low-flow conditions by permitting larger percentage errors at sparse volumes, which aligns with Poisson-distributed counts on lightly loaded roads but can mask significant relative discrepancies in data-scarce areas. This leniency may obscure validation issues in rural or off-peak scenarios, where even modest absolute errors could have disproportionate impacts on planning decisions.^[3]

Statistical Shortcomings

The GEH statistic lacks a formal statistical derivation, positioning it as an empirical formula rather than a rigorous hypothesis test, despite its superficial resemblance to established metrics like the chi-squared test under a Poisson distribution assumption.^[3] Although rooted in the Poisson index of dispersion—which assumes traffic counts follow a Poisson-like process where variance equals the mean—the GEH does not derive from a systematic probabilistic framework, leaving its thresholds (such as 5.0 for acceptable fit) as ad hoc guidelines without underlying derivation from first principles.^[3] This absence of formal grounding means the metric cannot generate p-values or confidence intervals, depriving users of a probabilistic interpretation to assess the significance of discrepancies between observed and modeled volumes.^[4] A key theoretical flaw in the GEH is its failure to account for the variance structure inherent in traffic count data, including variations due to sample size, road type, time of day, or congestion levels. The metric implicitly assumes a fixed proportionality between variance and mean (often six times the mean for links), but empirical analyses of automatic traffic count (ATC) datasets reveal variance-to-mean ratios as high as 17, indicating that this assumption frequently does not hold and leads to unreliable assessments of model fit.^[3] Consequently, GEH treats all observations equally regardless of their statistical reliability, overlooking how smaller sample sizes or higher variability in urban or congested settings inflate errors without adjustment for standard deviation.^[12] The nonlinear scaling of the GEH formula exacerbates these issues by producing inconsistent sensitivity across different volume ranges, where the same absolute or relative difference yields varying GEH values depending on the scale of flows. For instance, low-volume links may show exaggerated discrepancies due to the metric's square root form, while high-volume scenarios understate errors, complicating the distinction between meaningful and negligible deviations without scale-specific normalization.^[3] This nonlinearity lacks a probabilistic basis, as the metric does not provide a consistent threshold for significance akin to standardized tests, resulting in arbitrary interpretations that vary by context.^[4] Furthermore, the GEH's design poses challenges for aggregation and cross-dataset comparisons, as it evaluates individual link or observation pairs without inherent normalization for differing scales or variances, making it difficult to combine results into network-level metrics or benchmark against diverse datasets. Screenline totals, for example, often yield misleadingly low GEH values due to aggregated high means overshadowing individual variances, hindering reliable synthesis across models or regions.^[3] Without adjustments for these structural inconsistencies, the metric's utility in broader statistical validation remains limited, underscoring the need for theoretically sound alternatives in traffic modeling.^[4]

Alternatives

SQV Statistic Overview

The SQV (Scalable Quality Value) statistic was introduced in 2019 by Markus Friedrich and colleagues as a scalable quality measure designed to validate travel demand models by comparing observed and modeled single values, such as traffic volumes or trip distances.^[13] This development addressed the need for a metric adaptable to varying data magnitudes, extending beyond traditional measures limited to specific scales like hourly traffic counts.^[13] The SQV is mathematically defined as:

\text{SQV} = \frac{1}{1 + \sqrt{\frac{(M - C)^2}{f \cdot C}}}

where M represents the modeled value, C the observed (counted) value, and f a scaling factor calibrated to the expected magnitude of the indicator being evaluated.^[13] For example, f = 1,000 is typically used for hourly traffic volumes, while smaller values like f = 1 apply to person trips per day.^[13] The statistic produces unitless outputs ranging from 0, indicating a poor match between modeled and observed values, to 1, signifying a perfect match.^[13] The scaling factor f ensures applicability across diverse indicators, such as traffic volumes, trip durations, or distances, by normalizing the comparison to the data's inherent scale.^[13] As a variant of the chi-squared statistic, SQV balances absolute and relative errors in model validation, providing a more flexible assessment than non-scalable alternatives while maintaining interpretability for single-value comparisons.^[13] This foundation allows it to quantify deviations in a manner sensitive to both the magnitude of differences and their proportionality to observed counts.^[13]

Advantages and Enhancements of SQV

The Scalable Quality Value (SQV) extends its utility across various fields in traffic and travel demand modeling, including the validation of hourly and daily traffic volumes, trip distances, and mode shares. By incorporating a scaling factor f, SQV adapts to different data magnitudes; for instance, f = 1 is suitable for trip distances measured in kilometers, ensuring consistent application without distortion from units or scales.^[4] This flexibility allows SQV to evaluate model performance in diverse contexts, such as comparing simulated versus observed passenger volumes at rail stations or network-wide flow distributions.^[14] SQV establishes clear quality categories to guide model refinement: values ≥ 0.85 indicate a good match between observed and modeled data, while ≥ 0.90 signify a very good fit; lower thresholds suggest areas requiring calibration adjustments. Unlike the GEH statistic, which exhibits magnitude bias by being overly sensitive to differences at higher flows, SQV addresses this through its symmetric treatment of observed and modeled values, providing equitable assessment regardless of absolute scale.^[4] Its probabilistic foundation, rooted in normal distribution assumptions, enhances reliability by implicitly accounting for observational variance and sample size via scaling, thereby weighting errors appropriately without explicit chi-squared derivations.^[4] Further enhancements of SQV include its scalability for data aggregation, enabling seamless transitions from single-link validations to broader network-level analyses in large-scale simulations. This property supports iterative model improvements in tools like MATSim, where SQV facilitates precise calibration by quantifying fit across aggregated metrics such as total daily trips or modal splits. Overall, these advantages position SQV as a robust, adaptable metric that resolves key limitations of traditional measures like GEH, promoting more accurate traffic forecasting.^[4]^[15]