Mean percentage error
The mean percentage error (MPE) is a statistical metric used primarily in forecasting to quantify the average percentage deviation between predicted and actual values, serving as an indicator of systematic bias in predictive models.[1] It differs from absolute error measures by preserving the direction of errors, allowing positive (over-forecasting) and negative (under-forecasting) deviations to offset each other in the average. This relative measure scales errors into percentage units, facilitating comparisons across datasets with varying scales or units.[1] The MPE is calculated using the formula:where n is the number of observations, a_t is the actual value at time t, and f_t is the forecasted value at time t.[1] This computation essentially averages the individual percentage errors, defined asMPE = (100 / n) × Σ [(a_t - f_t) / a_t]MPE = (100 / n) × Σ [(a_t - f_t) / a_t]
((actual - forecast) / actual) × 100 for each period. A positive MPE suggests a tendency to over-forecast on average, while a negative value indicates under-forecasting, making it a valuable proxy for assessing bias in demand planning, economic projections, and other time-series predictions.[2]
Despite its utility for bias detection, MPE has notable limitations. The cancellation of opposing errors can result in a deceptively low value even when individual errors are large, masking poor overall accuracy. Furthermore, the metric is undefined or infinite when actual values equal zero, as division by zero occurs, which poses challenges in datasets with intermittent or zero-demand scenarios.[3] For these reasons, MPE is often used alongside scale-independent accuracy metrics like the mean absolute percentage error (MAPE) to provide a more complete evaluation of forecast performance.[2]
Definition and Calculation
Definition
The mean percentage error (MPE) is a statistical measure that assesses forecast accuracy by computing the average of signed percentage errors between predicted and observed values across a dataset.[2] This approach quantifies the relative deviation of forecasts from actual outcomes, expressed as percentages, enabling evaluation of predictive performance in a normalized manner.[4] MPE has been used in forecasting and statistical analysis since at least the late 20th century.[5] In statistics and forecasting, MPE serves as an indicator of directional deviation, preserving the sign of individual errors to reveal systematic tendencies toward over- or under-prediction without taking absolute values.[6] A positive MPE suggests overall overestimation, while a negative value indicates underestimation, thus facilitating bias detection in models.[2] Unlike the related mean absolute percentage error (MAPE), which focuses on absolute deviations, MPE highlights bias direction.[2]Formula
The mean percentage error (MPE) is defined as the arithmetic mean of the individual percentage errors over n observations.[7] The percentage error for the t-th observation, denoted p_t, is computed as p_t = 100 \times \frac{a_t - f_t}{a_t}, where a_t represents the actual value at time or index t, and f_t represents the corresponding forecasted value.[7] The MPE is then obtained by averaging these percentage errors: \text{MPE} = \frac{1}{n} \sum_{t=1}^n p_t = \frac{100}{n} \sum_{t=1}^n \frac{a_t - f_t}{a_t}. This formulation assumes a_t > 0 for all t to avoid division by zero.[7] The scaling factor of 100 converts the relative errors into percentage units, facilitating intuitive interpretation of the metric as an average bias in percent terms.[7] This standard formula derives from first calculating the signed relative error (a_t - f_t)/a_t for each observation to capture directionality, scaling it by 100 to form p_t, and finally taking the unweighted arithmetic mean across all n periods.[7] In cases of non-uniform datasets where observations vary significantly in importance or scale, a weighted variant (WMPE) may be applied, such as \text{WMPE} = \left( \frac{\sum_{i=1}^n e_i}{\sum_{i=1}^n a_i} \right) \times 100, where e_i = a_i - f_i is the error for the i-th observation; this effectively weights each relative error by the actual value a_i, emphasizing larger observations in the average.[8]Properties and Interpretation
Indication of Bias
The mean percentage error (MPE) serves as a signed metric that detects systematic bias in forecasting models by preserving the direction of individual errors. A positive MPE value indicates under-forecasting, where predictions are systematically lower than actual values on average, while a negative MPE signals over-forecasting, with predictions exceeding actuals. This directional sensitivity allows practitioners to identify and adjust for consistent tendencies in model outputs, unlike absolute error measures such as the mean absolute percentage error (MAPE), which disregard error signs and thus cannot distinguish between over- and under-prediction.[9][7] In interpreting MPE, values close to zero suggest unbiased forecasts, implying that the model neither systematically over- nor under-predicts. The magnitude of the MPE provides insight into the strength of the bias; for instance, an MPE of +5% reflects an average underestimation of actual values by 5%, while -3% indicates a 3% overestimation. These thresholds help quantify the practical impact of bias, guiding decisions on model refinement or recalibration to minimize directional inaccuracies.[9][10] To assess whether an observed MPE represents statistically significant bias, MPE can be analyzed using confidence intervals or hypothesis tests, such as a t-test on the underlying percentage errors to determine if the mean differs significantly from zero. If the 95% confidence interval for MPE excludes zero, this provides evidence of non-random bias in the forecasts, enabling rigorous evaluation of model reliability beyond point estimates.[11]Mathematical Properties
The mean percentage error (MPE) is defined as the average of the individual percentage errors, given by the formula \text{MPE} = \frac{100}{n} \sum_{i=1}^{n} \frac{y_i - \hat{y}_i}{y_i}, where y_i is the actual value, \hat{y}_i is the predicted value, and n is the number of observations. This formulation reveals that MPE is linear in the errors e_i = y_i - \hat{y}_i, as it can be rewritten as \text{MPE} = \frac{100}{n} \sum_{i=1}^{n} \frac{e_i}{y_i}, a linear combination of the e_i for fixed actual values y_i. Consequently, scaling all errors by a constant factor k results in the MPE scaling by the same factor, facilitating additive assessments of bias when aggregating across disjoint subsets of data with similar characteristics, such as equal sample sizes and comparable y_i scales. A key advantage of MPE is its scale invariance, stemming from the percentage-based normalization. Since both the numerator (error) and denominator (actual value) are in the same units, MPE remains unchanged if all actual and predicted values are multiplied by a positive constant, such as converting from dollars to euros. This property allows MPE to be compared across datasets with different measurement scales, unlike unit-dependent metrics like mean error. Unlike absolute error measures, MPE exhibits symmetry in its treatment of over- and under-predictions through the signed nature of the percentage errors, permitting positive and negative deviations to cancel each other out in the summation. For instance, a +10% error paired with a -10% error yields a net zero contribution to MPE, which can lead to underestimation of overall inaccuracy if errors are balanced but individually large. This cancellation effect highlights MPE's utility for detecting systematic bias but underscores its distinction from metrics like mean absolute percentage error, which prevent such offsetting. In large samples, under assumptions of independent and identically distributed percentage errors with finite mean (e.g., in stationary forecasting processes), MPE converges in probability to the expected percentage error by the law of large numbers, providing a consistent estimator of population bias. This asymptotic behavior assumes no division by zero and finite variance, ensuring reliable inference as sample size increases.Limitations and Challenges
Undefined Cases
The mean percentage error (MPE) becomes undefined in cases where any actual value a_t = 0, as the formula involves division by the actual value in the denominator, resulting in division by zero.[12] This issue arises because the percentage error for each observation is computed as \frac{a_t - f_t}{a_t} \times 100, where f_t is the forecast, and the mean is taken across all observations; when a_t = 0, the term is mathematically indeterminate.[13] Such undefined cases are particularly prevalent in forecasting applications involving intermittent or sporadic data, where actual values frequently include zeros, such as in zero sales periods, event counts with no occurrences, or non-positive quantities in inventory management.[14] For instance, in demand forecasting for products with irregular sales patterns, zero actuals can represent periods of no demand, making standard percentage-based metrics like MPE inapplicable without adjustment.[14] To mitigate this problem, common strategies include excluding observations where a_t = 0 from the MPE calculation, which preserves computability but may lead to biased evaluations if zeros are frequent.[15] Another approach is to add a small positive constant \epsilon (e.g., $10^{-9}) to the denominator, modifying it to a_t + \epsilon, though this can introduce minor distortions in the error estimate, especially for small non-zero values. Alternatively, practitioners may switch to logarithmic or symmetric percentage error variants, such as the symmetric mean absolute percentage error (sMAPE), which avoids division by zero by using the average of actual and forecast in the denominator.[16][14] The presence of undefined cases can result in incomplete or unreliable model evaluations, as excluding zeros reduces the sample size and potentially overlooks critical low-demand scenarios. To address this, it is recommended to pre-process datasets by identifying and flagging zero actuals upfront, allowing for consistent application of appropriate metrics across analyses.Sensitivity to Scale
The relative nature of the mean percentage error (MPE) renders it highly sensitive to the scale of actual values, particularly when those values are small but non-zero. In the percentage error calculation, (a_t - f_t)/a_t, even modest absolute deviations can produce disproportionately large relative errors when a_t is close to zero, amplifying the perceived inaccuracy of forecasts. For instance, if the actual value a_t = 1 and the forecast f_t = 2, the resulting percentage error is -100%, illustrating how small denominators exaggerate errors far beyond their absolute magnitude. This amplification effect is a well-documented limitation in forecasting literature, where percentage-based metrics penalize inaccuracies in low-scale observations more severely than in high-scale ones.[13] In datasets with heterogeneous scales, such as sales records ranging from $1 to $1000 across items, MPE calculations can become dominated by low-value observations, skewing the overall metric and reducing its reliability for evaluating forecast performance across the entire dataset. Low-volume items contribute outsized percentage errors that overshadow errors in higher-volume items, leading to a biased representation of model accuracy. This issue is particularly pronounced in diverse or multi-series forecasting scenarios, where varying magnitudes distort comparative assessments.[9] To mitigate these scale-related sensitivities, MPE should be used in conjunction with scale-robust metrics, such as the mean absolute scaled error (MASE), which normalizes errors relative to in-sample naive forecasts and avoids division by small values. Data normalization techniques, like logarithmic transformations, can also help stabilize scales before applying MPE, especially in contexts like intermittent demand forecasting where low values are common. In such scenarios, combining MPE with absolute error measures provides a more balanced evaluation.[13] Empirical studies on intermittent and low-volume demand series have demonstrated MPE's volatility, with post-2000s research showing that it often yields unstable rankings of forecasting methods due to sensitivity in sparse data environments. For example, analyses of intermittent demand patterns reveal that MPE can produce counter-intuitive results, overemphasizing errors in periods of low activity and undermining its utility for model selection in real-world applications like inventory management. This volatility is evident in comparisons across stochastic demand models, where MPE fails to consistently identify superior forecasters compared to mean-based alternatives.[17]Comparisons with Related Metrics
Versus Mean Absolute Percentage Error
The mean percentage error (MPE) and mean absolute percentage error (MAPE) are both relative measures of forecast accuracy, but they differ fundamentally in how they treat errors. MPE calculates the average of signed percentage errors, given by \frac{100}{n} \sum_{t=1}^{n} \frac{a_t - f_t}{a_t}, where a_t is the actual value and f_t is the forecast at time t, allowing it to indicate the direction of bias (positive for under-forecasting, negative for over-forecasting). In contrast, MAPE uses absolute values to focus on the magnitude of errors, defined as \frac{100}{n} \sum_{t=1}^{n} \left| \frac{a_t - f_t}{a_t} \right|, ignoring the sign to provide a measure of overall inaccuracy. This signed versus absolute distinction means MPE can detect systematic over- or under-prediction, while MAPE quantifies the typical error size without regard to direction.[18] MPE is particularly useful for bias detection in forecasting applications, such as inventory management where consistent underestimation could lead to stockouts, whereas MAPE is preferred for assessing general accuracy across models or datasets since it avoids error cancellation and yields a non-negative value interpretable as a percentage deviation. For instance, MAPE will always be positive and reflects the average proportional error, making it suitable for comparing performance on series with different scales, but MPE can equal zero even if substantial errors exist, as positive and negative deviations may balance out, potentially understating the true inaccuracy. In practice, analysts often compute both to gain complementary insights: MPE for directional trends and MAPE for magnitude.[18] Both metrics share vulnerabilities that limit their robustness. They become undefined when any actual value a_t = 0, as division by zero occurs, a common issue in datasets with intermittent demand or zeros, such as sales records. Additionally, they exhibit high sensitivity to small actual values, where even minor forecast errors can produce disproportionately large percentages, amplifying outliers and distorting overall assessment. These properties necessitate preprocessing, like excluding zero values or using scaled alternatives, to ensure reliable interpretation.[18] Historically, MAPE gained prominence in the 1980s through its adoption as the primary accuracy metric in the M-Competition, a seminal forecasting evaluation organized by Spyros Makridakis and colleagues, which compared extrapolation methods on real time series data and established MAPE's role in benchmarking simplicity versus complexity. MPE, as a bias-oriented complement to absolute measures like MAPE, appeared more frequently in forecasting literature during the 2000s, often alongside discussions of error properties in textbooks and empirical studies seeking to address MAPE's directional limitations.[19][18]Versus Mean Error
The mean error (ME), also known as average bias, measures the average signed deviation between actual values and forecasts in the original units of the data, calculated as ME = \frac{1}{n} \sum_{t=1}^n (a_t - f_t), where a_t is the actual value and f_t is the forecast at time t.[20] In contrast, the mean percentage error (MPE) normalizes these signed errors by the actual values to express bias on a percentage scale, given by MPE = \frac{100}{n} \sum_{t=1}^n \frac{a_t - f_t}{a_t}, enabling direct comparability across datasets with varying scales or units.[20] This scaling transforms ME into a relative metric, highlighting proportional bias rather than absolute differences.[21] MPE offers advantages over ME by being unit-free, which facilitates comparisons in heterogeneous environments, such as evaluating forecasts across multiple time series with different magnitudes.[20] For instance, ME remains tied to the data's scale, making it less intuitive for cross-dataset analysis, whereas MPE provides a standardized percentage interpretation of bias.[22] However, MPE is more sensitive to small actual values, potentially amplifying errors when a_t is near zero and risking undefined results if any a_t = 0, while ME maintains stability in such cases but fails to convey relative impact.[21] In practice, ME is suitable for assessing absolute bias in datasets with consistent units, such as single-series production forecasts where raw deviations matter for operational decisions.[22] Conversely, MPE excels in multi-product or cross-series forecasting, like supply chain management involving diverse item volumes, where relative bias informs scalability and prioritization.[20] Both metrics share a signed nature to detect systematic over- or under-prediction, but MPE's relativity better supports benchmarking across varied applications.[21]Applications and Examples
Use in Forecasting
Mean percentage error (MPE) finds primary application in time series forecasting across domains such as demand and sales prediction, economic indicators, supply chain management, and financial modeling, where it serves as a key indicator of systematic bias in predictive models.[2] In supply chain contexts, MPE helps evaluate forecast accuracy for inventory planning by quantifying average deviations in percentage terms, enabling adjustments to avoid overstocking or shortages.[23] Similarly, in finance and economics, it assesses biases in projections like GDP growth or commodity prices, as seen in evaluations of U.S. Department of Agriculture long-term agricultural forecasts, where MPE revealed tendencies toward over- or under-forecasting harvested areas.[24] In forecasting workflows, MPE is integrated to tune models toward unbiased predictions by identifying directional errors that might otherwise cancel out in absolute measures, often combined with metrics like mean absolute percentage error (MAPE) or root mean squared error (RMSE) for a holistic assessment of both bias and accuracy. This approach supports decision-making in risk management, such as refining demand plans in manufacturing to minimize operational costs from biased estimates.[2] Software tools facilitate MPE computation in forecasting pipelines. In R, the forecast package's accuracy() function directly calculates MPE alongside other error metrics for time series models.[25] Python implementations are available through libraries like NumPy for custom calculations or extensions in forecasting packages such as statsmodels, where MPE is derived from mean error scaled by actual values.[26] In Excel, MPE can be computed using array formulas like AVERAGE((actual - forecast)/actual * 100) across datasets for quick model evaluation.[27] Post-2000s advancements in computational forecasting have highlighted MPE's role in detecting biases in specialized applications. For instance, in weather prediction, services like meteoblue employ MPE to measure errors in precipitation sum forecasts, aiding improvements in model reliability for hydrological planning.[28] In stock price forecasting, a 2024 study on hybrid CNN-LSTM models for share prices reported an MPE of 2.438%, demonstrating low bias in predictions for volatile markets and underscoring MPE's utility in financial algorithm tuning.[29]Numerical Example
To illustrate the calculation of the mean percentage error (MPE), consider a hypothetical dataset from five time periods in a sales forecasting scenario, with the following actual values A_i and forecasted values F_i:| Period | Actual (A_i) | Forecast (F_i) |
|---|---|---|
| 1 | 100 | 95 |
| 2 | 150 | 139 |
| 3 | 200 | 180 |
| 4 | 50 | 60 |
| 5 | 100 | 90 |
- Period 1: \text{PE}_1 = \frac{100 - 95}{100} \times 100\% = 5\%
- Period 2: \text{PE}_2 = \frac{150 - 139}{150} \times 100\% \approx 7.33\%
- Period 3: \text{PE}_3 = \frac{200 - 180}{200} \times 100\% = 10\%
- Period 4: \text{PE}_4 = \frac{50 - 60}{50} \times 100\% = -20\%
- Period 5: \text{PE}_5 = \frac{100 - 90}{100} \times 100\% = 10\%
- Period 4 (varied): \text{PE}_4 = \frac{5 - 15}{5} \times 100\% = -200\%