Applying Es Metrics to Analyze Schedule Performance
Applying EVM and Es Metrics to Analyze and Forecast Schedule Performance in the Spanish Context of the Building Sector
Abstract Most building projects finish their execution exceeding the contractually established completion date. Although Earned Value Management (EVM) is a technique widely developed in many countries to control the project schedule, in Spain it is not used in the construction industry. Furthermore, EVM has an extension called Earned Schedule that allow for more reliable control of the deviations occurring over the planned duration. This paper examines an application model of the EVM metrics recently proposed to be used in the Spanish context of the construction industry, but focusing only on time control. In order to verify the metrics obtained with this model, three prediction methods well established in the literature are applied on real data of six building projects, establishing critical thresholds to assess the practical validity of the predictions. Although the data set is relatively small, the developed analysis produces encouraging results that may serve as a starting point for further studies.
Keywords Earned value • Earned schedule • Forecasting • Building projects
Earned Value Management (EVM) is a tool of project management that enables performance assessment of project execution, integrating the joint management of scope, time and cost. It allows identifying deviations in the execution and makes predictions about project final results considering the data collected until the control date and several performance assumptions. It is used in the construction industry from other countries and it has been the subject of several studies in this sector. However, the EVM methodology is not used in Spain in the construction sector because, first, large Spanish construction companies have their own management systems clearly focused on cost control, leaving time control in the background, and secondly, more importantly, many professionals of this sector do not know its existence .
The study developed aims to assess the practical validity of a simplified model recently introduced to apply the EVM methodology in the Spanish context of the construction industry. The research focuses only on time control, leaving aside the cost control. For this purpose, several EVM prediction methods, that enable estimating the final duration of a project, are applied using the revised model on a set data from building real projects. Both basic principles and practical use of the EVM methodology have been widely described in many sources of the literature (see e. g. ) and in this article it is assumed its understanding by the reader.
2 Model to Apply the EVM Metrics 2.1 Definition
In construction sites performed in Spain, the project budget (Budget at Completion, BAC) is decomposed by a Work Breakdown Structure (WBS) based on Chapters, Subchapters and Work Units (WU), according to the diagrams shown in Fig. 1. With this premise, Valderrama and Guadalupe  have presented two models to apply the EVM methodology, oriented towards the Spanish construction industry: Simplified Model for the Project Manager, representing the owner’s perspective; and Complete Model, oriented towards the construction company. These models use two essential variables for each WU on which the budget of a project is divided: Quantity (Q) and Price (P).
This paper focuses on the simplified model, which is based on the traditional procurement method used in the Spanish construction industry, where partial payments to the constructor are carried out monthly by work certifications. The model uses as Earned Value (EV) the certificate amount, considering only the portion of work included in the planned budget of the project. Table 1 shows the equations involved in this model at project level and its correspondence with the key metrics of the EVM methodology.
Although Valderrama and Guadalupe  indicate that in the simplified model it is not required to have planning, this study considers essential the existence of a schedule made by the contractor before starting work. However, in most projects, planning undertaken by contractors do not reflect the temporal distribution of cost for each of the scheduled activities, and these do not follow a clear pattern of correspondence with the WUs of the project. This makes impossible a scheduling analysis at levels below the project level. For this reason it was chosen project level control for the experimentation proposed in the research (see Fig. 1).
It must also be considered that, in a procurement system with open measurement, as projects developed by public administration in Spain (the law allows variations up to 10 % of contract value) when the EV metric is matched with the certified amount it is assumed the possibility of committing errors of earned value measurement, even discounting the overrun certificates until the control date. In this context, an EV measurement error is made when a WU finishes with less quantity than it is expected, since it should be accredited 100 % of planned value at the time this happens. Similarly, when a WU exceeds its planned quantity does not mean that it is complete and 100 % of planned value can be accredited, but it should be known the new quantity and calculate the percent completed with regard to it. The above errors may be admissible provided that they do not involve a change of project scope and the magnitude of the deviations on the total budget will not be significant. Otherwise, it proceeds to make a modification of budget and re-baseline. Moreover, with a fixed price procurement system, more typical of the private sector, such errors could be avoided whenever it is used the percent completed for each WU and there will not be deviations in quantities.
After completing all planned works, by definition, the EV must match the BAC, regardless of the last value certificated.
Table 2 Real project data
M. A. Guerrero et al.
|Project||Budget (In thousands||Planned duration||Actual duration||Deviation||Deviation|
|3||2,305.65 k€||18.0||17.0||− 1.0||− 5.6|
|5||779.52 k€||12.0||10.0||− 2.0||− 16.7|
4 The Data Set
There have been 5 health building projects collected for the development proposal, promoted by the public administration (open measurement), and a private residential building project (fixed price), all performed by different contractors. All projects are located in southeastern Spain and the works were completed between the years 2006 and 2012. None of the projects underwent changes in its scope during the construction phase. The general project data are shown in Table 2. According to monthly character of certifications, once work starts there is a control point with data at the end of each month.
It is necessary to indicate that in the final certification of the 5 public projects (prior to settlement certification) the sum of deviations in absolute value to chapter level was about 4 % of the total budget. However, in all projects the amount of this final certification matches the planned budget (EV = BAC), indicating a clear tendency for compensation of errors during the execution of works. Bearing in mind all this, the excesses of measurement were not deducted from the EV.
Table 3 Methods and proposed scenarios
|Expected performance to||Forecast methods|
|remaining work||Planned value||Earned duration||Earned schedule|
|According to plan||EAC(t)PV1||EAC(t)ED1||EAC(t)ES1|
|Following the trend of performance indices (SPI/SPI(t))||EAC(t)PV2||EAC(t)ED2||EAC(t)ES2|
5 Forecasting Final Duration and Evaluating Its Quality
In order to predict the project duration defined by EAC(t) (Estimate at Completion (time)), three forecasting methods have been selected from EVM methodology: the planned value method (PV, ), the earned duration method (ED, ) and the schedule earned method (ES, ). Both PV and ED methods are based on traditional metrics while the ES method is based on a new metric developed to improve controlling of the execution time by two new indices (SV(t) and SPI(t)). With each of the proposed forecast methods we use two possible scenarios that define the performance of work remaining. These methods have been evaluated several times using the simulation and for greater depth in knowledge of proposals EAC(t) predictive equations it is recommended to consult . Table 3 summarizes the methods and proposed scenarios.
The quality of the forecasts has been assessed according to the criteria of accuracy and timeliness. For measuring the accuracy criterion, the MAPE (Mean Absolute Percentage Error) has been selected. Regarding the timeliness of forecast, the study has followed the approach that divides the horizon of the project in three phases: Early (0–30 %), Middle (30–70 %) and Late (70–100 %) . However, the practical validity of the forecasting methods should be evaluated in the context of a larger decision making system, setting critical thresholds for the selected criteria . For accuracy, in some studies related to construction projects, maximum acceptable errors around 10 % have been set (see e. g. ). Table 4 summarizes criteria, error metrics and thresholds used in the experimentation performed.
Table 4 Criteria for assessing forecasts
Criterion Error metric Threshold Equation
2 Timeliness MAPE Early (0–30 %) <10% (5)
Middle (30–70 %) <10% Late (70–100 %) <10%
%C EV( )
n is the total number of control periods,
%C is the percentage completed of the project and RD is the real duration.
6 Results and Discussion
In this section the MAPE average values are analyzed. Figure 2 shows these values in each project stage as well as the overcoming frequency of the critical thresholds.
Overall accuracy Using the scenario #1 it is achieved, on average, a more accurate forecast, regardless of methods used. All three methods obtain similar accuracy for scenario #1 but with scenario #2, considering the performance indices SPI and SPI (t), there is a noticeable difference of accuracy in favor of the ES method. Furthermore, according to the thresholds defined, only the ES method under scenario #1 obtain, on average, a MAPE lower than 10 % although all three methods reach values very close.
Early stage The accuracy results obtained during this phase of the project follow the same pattern as those achieved for the entire project, only that with higher error values. If we consider the thresholds defined above, any method yields a MAPE lower than 10 %, but the results are much more accurate when the methods are used under the scenario #1.
Middle stage In this stage it is repeated the same trend in accuracy results but with significantly lower errors. Regard to determined thresholds, the ES method yields a MAPE less than 10 % with both scenarios, while the PV and the ED methods reach this result only under scenario #1.
Late stage The ES method clearly outperforms both ED and PV methods, using any of the two scenarios, and in the same way the ED method outperforms the PV method. Only the ES method yields a MAPE less than 5 % in both scenarios.
While being aware of the small sample size, we also performed non-parametric tests of significance using SPSS. In the early, middle and general stages, using individually the Mann-Whitney test with both PV and ED methods, we found that differences in accuracy between scenarios #1 and #2 were statistically significant on the 5 % level. Also, applying the Kruskal-Wallis test with the three methods in each scenario of the final stage, we found values of p = 0.069 with scenario #1 and p = 0.134 with #2. These values do not become significant on the 5 % level, but are a good indicator of the results observed in the final stage graphs (Fig. 2).
The developed research revises a simplified model of application for EVM methodology, focused on the Spanish construction industry. In order to verify the practical validity of the control time with this model, we apply three prediction methods on real data of six building projects and compare the results obtained with established critical thresholds.
The results of experimentation show that the scenario assuming the remaining work will be developed according to the planned schedule produces, on average, more accurate forecasts during the early and middle stages of the project, regardless of the prediction method used. Under this scenario, practical results are obtained during the middle stage with any method and in the final stage only with the ES method. On the other hand, considering the scenario based on performance indices SPI and SPI (t), only the ES method does not outperform the thresholds fixed in the middle and final stages. Consequently, in the final stage, regardless of the scenario considered, the method ES outperform, on average, both PV and ED methods, that accuse the known anomalous behavior of the SPI index .
The summary of the study is that using the simplified model to apply the EVM methodology at project level, in the Spanish context of the building sector and once overcomed the initial stage, practical results can be obtained in the final duration forecast, even accepting with this model certain errors in the measurement of the earned value. However, it should be noted that the study developed is based on a relatively small sample of projects and although the results appear consistent, confirming partially other studies , in order to generalize them, it would be necessary to apply the same research methodology to a considerably larger sample with different types of real building projects. It is also noted that the critical thresholds established in the investigation, although reasonably based on other studies, may vary according to the needs of each organization and other parameters of the project (e. g. the kind of contractual penalty).