Project Delivery Done Right. What Do You Need To Track And Analyze?

Anna Shalomova

PPM Consultant

Most likely, we will not be too original in stating that management implies measurement. This topic is particularly relevant for large companies. This article does not claim to be the absolute truth. We are ready to offer a list of metrics that must be measured in project delivery and given to senior management to assess the situation.

The role of project portfolio management in supporting strategic business priorities has become more important than ever. When project portfolio management becomes more successful, it must be more responsible and prove its value. In this article, we will highlight the metrics that, in our opinion, will help track project delivery. Although the project might not be able to start tracking all these metrics immediately, it should follow most of them in its quarterly reports when it reaches a particular maturity.

IT Standards: Metrics in Software Development Projects

When talking about metrics and analytics, we ask ourselves, “Why do we need them?”. Obviously, to assess the possible risks in advance. Therefore, the first metrics you need to track and analyze are the risk factors for your project and project delivery. Risk factors are not the risks themselves. These are the reasons behind the risk.

Risk Factors

For your project (development of an information system for an external customer), you can determine the following main risk factors:

• Risks associated with staff turnover (in industry, in a company).
• Limitations imposed by quality requirements.
• Internal political factors (internal policy, which may hinder the implementation of the project).
• External political factors (maybe important when working with large government clients).
• Lack of technological expertise in the team.
• The human factor: people avoid responsibility, and having a negative experience in previous projects.
• Low level of organizational maturity in the company.

Not all these factors can always be used as a basis for numerical metrics. However, to put a kind of “state of affairs indicators” for those risks that you can evaluate qualitatively (“everything is still good/bad/terrible”) does not hurt.

Main Metrics

In general, for a project you should consider it quite reasonable to begin to determine the following types of metrics:

• Employee turnover rate.
• Resource utilization rate.
• Indicators related to the timing and budget of the project.
• Indicators to assess the quality of the product being developed.
• Integral indicators of project delivery.

In general, you can use the following approach for selecting metrics for a project:

• Metrics of life cycle stages and schedule. Monitor the work schedule for the life cycle stages and compare actual and planned values.
• Project cost/value-added metrics: monitor the cumulative costs in comparison with the budget and the total cost of the project, constantly updating the data as the project progresses.
• Requirement change tracking metrics: monitor the number of changes in project-wide requirements.
• Development process metrics: monitor the number of requirements implemented in the model versus the total number of conditions in the project.
• Failure type metrics: track software failure reasons.
• The remaining metrics for defects. The graphical representation of the number of failures monthly throughout the project.
• Performance metric overview: Track phase error density and use charts to determine “peaks” and “dips” on the curve and exceed the maximum permissible values.

Proactive Project Status Analysis

Three types of metrics can be used to analyze the state of a project: proactive analysis metrics, diagnostic metrics, and retrospective metrics. First, we need to eliminate the trouble long before it happens. Second, we need to see how things are going on the project. And last but not least, we need to learn from the history of their victories and defeats.

To understand what lies ahead and what can happen, we can prepare several data sets and numbers for analysis: planned scope of work, budget, and resource plan.

The complexity factor of a project

It is a quantity that characterizes the total volume of all external artifacts of a project, multiplied by the complexity coefficient determined for each of the external artifacts.

The total risk associated with the schedule

It is a total value reflecting changes in the schedule of the project, considering the probability of their occurrence. For example, you can estimate how much time your project team will spend on sick leaves and how much time on vacations. You can estimate how long the supplier will delay the project delivery of equipment, which is always late with deliveries by at least three weeks.

The overall risk associated with a budget

It is the total value of all unforeseen expenses for project delivery, based on a budget plan for unforeseen expenses, considering the probability of their occurrence. For example, you can include compensation for unforeseen changes in labor costs, overhead costs, etc., arising during the work on the project.

The complexity of the project plan

It is the number of interrelations and interdependencies between the various works in the schedule, referred to as the total number of all works in the schedule. It affects the time reserve that needs to be laid in the project to account for the shift in the timing of unexpected tasks “pushed” by the connections. Also, it is challenging to add new people to a project with many interconnections, for example, by reorganizing the business.

The density of the project

The ratio of the total duration of all work in the schedule when executed in series to the sum of its own and the total reserve (reserve) of time for all planned works. The higher the density, the more complex the project will be. Do not just perform on time—it will be harder to complete the project.

Project Independence

The relationship between the number of dependencies for internal project work and the total number of dependencies, including dependencies on external work and suppliers.

Human resources

The maximum number of project participants (time to think: Do we even have the necessary number of people? Are there any “spare” ones for an emergency?)

Total time margin

Atotal reserve of time for all planned works.

Of course, all these numbers:

• a) are drawn based on facts you know (for example, a vacation schedule, average annual number of days spent at the hospital for each project team member, and other statistics on his/her work).
• b) regularly updated. In general, they can be quite correct only after your analysis phase has come to an end.

Do you think all this is so trivial that it is not worth mentioning? Then those project managers who can answer at any time whether he/she has gone beyond the budget for unforeseen expenses on his/her project, let them tell you how they do it. How are unforeseen budget expenditures calculated for the portfolio of projects under their management? Who can make resource plans with accuracy?

Diagnostic Metrics

Key metrics that allow you to assess what is happening in your project right now. Each of the listed quantities is measured regularly.

• The planned and actual budget of the work performed. A constantly changing value (with a cumulative total) of the total cost of all the planned work on the project.
• The ratio of the actual budget to the planned budget. If this ratio does not exceed 1, then the project fits into the budget in the medium term. If not, then, most likely, the budget will be overspent.
• Dispersion of costs: the difference between the planned budget of work that should have been completed at the current time according to the schedule and the actual budget, allowing an estimate of the amount of overspending or unspent budget in the project.
• Schedule execution: the ratio of the actual labor costs for completing completed works to the planned labor costs for implementing these works (planned labor intensity). This parameter is used to assess the risk of non-compliance with the schedule.
• Schedule variance: the difference between the actual labor costs for the work performed and the planned labor costs. This is an absolute expression of the same parameter, represented as a coefficient.
• Schedule lag: the total length of the delay time for tasks not met in the schedule.
• The ratio of finished work. The ratio of completed (closed) tasks to the total number of works scheduled for completion now.
• Labor productivity is calculated as the ratio of the volume of work to elapsed time (deviation from planned values).

Retrospective analysis

In project and portfolio management, everyone should understand why it is essential to collect Retrospective data. The distinction between diagnostic and retrospective metrics is that the first is used to analyze the situation “here and now” for emergency response. At the same time, retrospective analysis assesses past events and compares them to an undoubtedly bright future. The most important ones are:

• Reliability ratio. The ratio of the product of the project cost to its duration to the cumulative value of the increasing planning error.
• The ratio of the actual labor productivity to the planned one is calculated according to the actual and planned schedule. The difference is calculated as a percentage of the planned schedule.
• Percentage of labor involved in the project (phase, a separate group of tasks).
• The ratio of the amount of work in testing to the total amount of work.

Most project management systems in today’s market are either too focused on specific metrics and analytical capabilities or expensive and intricate. That’s why small and medium companies are missing out on the prospect of proper control over their operation. That is why it makes so much more sense to use BI to track and control many of these metrics. Mainly because the analysis becomes much more complex and time-consuming for the managers as the operations and the pace of conducting business grow.

Simplified Microsoft PPM

To provide PMOs and executives with a way to track, manage, and make decisions on current projects and portfolios, provide higher portfolio visibility and project delivery cost assessment, and give a more flexible tool, Microsoft created both affordable and feature-rich solutions—Power BI. You get the whole list of advantages with the “Project Online + Power BI + FluentPro Packs” nuclear combo. Organizations can track, manage, and analyze the progress of portfolios, projects, quality, teams, and people—sort projects by type, status, issues, milestones, and risks. You can sort costs by project type and get summaries and a prognosis.

It’s possible to go “deeper” and get a detailed view of every project, timeline, overlaps, and dependencies if needed. You can also track resource allocation and utilization, deliverables, team workloads, and all kinds of “history”. It is a gift because keeping track of previous successes and failures can make the prognosis much easier and more accurate.