Project Delivery Done Right. What Do You Need To Track And Analyze?
Most likely, we will not be too original in once again stating that management implies measurement. This topic is particularly relevant for large companies. This article does not claim to be the absolute truth. We are ready to offer a list of metrics that need to be measured in projects and given to senior management to assess the situation.
The role of project portfolio management in supporting strategic business priorities has become more important than ever. When project portfolio management becomes more successful, it also needs to be more responsible and prove its value. In this article, we will highlight the metrics that, in our opinion, will help track project performance. And although the projects might not be able to start tracking all these metrics right away, when it reaches a certain maturity, it should track most of them in its quarterly reports.
IT Standards: Metrics in Software Development Projects
The first thing we need to ask ourselves when talking metrics and analytics is “Why do we need them?” Obviously, to assess the possible risks in advance. Therefore, the first metrics you need to track and analyze are the risk factors for your project. Risk factors are not the risks themselves. These are the reasons behind the risk.
For your project (development of an information system for an external customer), you can determine the following main risk factors:
- Risks associated with staff turnover (in industry, in a company)
- Limitations imposed by quality requirements
- Internal political factors (internal policy, which may hinder the implementation of the project)
- External political factors (may be important when working with large government clients)
- Lack of technological expertise in the team
- Human factor: people avoiding responsibility, having a negative experience in previous projects
- Low level of organizational maturity in the company
Not all these factors can always be used as a basis for numerical metrics. However, to put a kind of “state of affairs indicators” for those risks that you can evaluate qualitatively (“everything is still good/bad/terrible”), it does not hurt.
In general, for a project you should consider it quite reasonable to begin to determine the following types of metrics:
- Employee turnover rate
- Resource utilization rate
- Indicators related to the timing and budget of the project
- Indicators to assess the quality of the product being developed
- Integral indicators of project progress
In general, you can use the following approach for selecting metrics for a project:
- Metrics of life cycle stages and schedule: monitor the work schedule for the life cycle stages and compare actual and planned values.
- Project cost/value-added metrics: monitor the cumulative costs in comparison with the budget, as well as the total cost of the project, constantly updating the data as the project progresses.
- Requirement change tracking metrics: monitor the number of changes in project-wide requirements.
- Development process metrics: monitor the number of requirements implemented in the model versus the total number of requirements in the project.
- Failure type metrics: track software failure reasons.
- The remaining metrics for defects. The graphical representation of the number of failures month by month throughout the project.
- Performance metric overview: Track phase error density and use charts to determine “peaks” and “dips” on the curve, as well as exceeding the maximum permissible values.
Proactive Project Status Analysis
Three types of metrics can be used to analyze the state of a project: proactive analysis metrics, diagnostic metrics, and retrospective metrics. First, we need to try to eliminate the trouble long before it happens. Second, we need to see how things are going on the project. And last but not least, we need to learn from the history of their victories and defeats.
To understand what lies ahead and what can happen in the end, we can prepare several data sets and numbers for analysis: planned scope of work, budget, resource plan.
- The complexity factor of a project is a quantity that characterizes the total volume of all external artifacts of a project, multiplied by the complexity coefficient determined for each of the external artifacts.
- The total risk associated with the schedule is a total value reflecting changes in the schedule in the project, considering the probability of their occurrence. For example, you can estimate how much time your project team will spend on sick leaves and how much time on vacations. You can try to estimate how long the supplier will delay the delivery of equipment, which is always late with deliveries by at least three weeks.
- The overall risk associated with a budget is the total value of all unforeseen expenses for a project, based on a budget plan for unforeseen expenses, considering the probability of their occurrence. For example, you can include compensation for unforeseen changes in labor costs, overhead costs, etc., arising during the work on the project.
- The complexity of the project plan is the number of interrelations and interdependencies between the various works in the schedule, referred to the total number of all works in the schedule. It affects the time reserve that needs to be laid in the project to account for the shift in the timing of tasks that are unexpectedly “pushed” by the connections. Also, it is very difficult to add new people to a project with a large number of interconnections, for example, by reorganizing the business.
- The density of the project – the ratio of the total duration of all work in the schedule when they are executed in series to the sum of its own and the total reserve (reserve) of time for all planned works. The higher the density, the more difficult the project will be. Do not just perform on time—it will be harder to complete the project.
- Project independence – the relationship between the number of dependencies for internal project work and the total number of dependencies, including dependencies on external work and suppliers.
- Human resources: the maximum number of project participants (time to think: Do we even have the necessary number of people? Are there any “spare” ones for an emergency?)
- Total time margin: a total reserve of time for all planned works.
Of course, all these numbers:
- a) are drawn based on facts known to you (for example, a vacation schedule, average annual number of days spent at the hospital for each of your project team members, and other statistics on his/her work).
- b) regularly updated. In general, they can be quite correct only after your analysis phase has come to an end.
Do you think all this is so trivial that it is not worth mentioning at all? Then those project managers who can answer at any time whether he/she has gone beyond the budget for unforeseen expenses on his/her project, let them tell you how they do it. How are unforeseen budget expenditures calculated for the portfolio of projects under their management? Who can make resource plans with an accuracy of?
Key metrics that allow you to assess what is happening in your project right now. Each of the listed quantities is measured on a regular basis.
- The planned and actual budget of the work performed: a constantly changing value (with a cumulative total) of the total cost of all the planned work on the project completed to date.
- The ratio of the actual budget to the planned budget: if this ratio does not exceed 1, then the project fits into the budget in the medium-term; if not, then, most likely, the budget will be overspent.
- Dispersion of costs: the difference between the planned budget of work that should have been completed at the current time according to the schedule, and the actual budget, allowing to estimate the amount of overspending or unspent budget in the project.
- Schedule execution: the ratio of the actual labor costs for the completion of completed works to the planned labor costs for the implementation of these works (planned labor intensity). This parameter is used to assess the risk of non-compliance with the schedule.
- Schedule variance: the difference between the actual labor costs for the work performed and the planned labor costs. This is an absolute expression of the same parameter, which is represented as a coefficient.
- Schedule lag: the total length of the delay time for tasks that are not met in the schedule.
- The ratio of finished work. The ratio of the number of completed (closed) tasks to the total number of works scheduled for completion now.
- Labor productivity calculated as the ratio of the volume of work to elapsed time (deviation from planned values).
In project and portfolio management everyone should understand why it is important to collect Retrospective data. The distinction between diagnostic and retrospective metrics is that the first is used to analyze the situation “here and now” for emergency response. While retrospective analysis is used to assess past events and compare them to an undoubtedly bright future. The most important ones are:
- Reliability ratio: the ratio of the product of the project cost to its duration to the cumulative value of the increasing planning error.
- The ratio of the actual labor productivity to the planned one, calculated according to the actual and planned schedule: the difference is calculated as a percentage of the planned schedule.
- Percentage of labor involved in the project (phase, a separate group of tasks)
- The ratio of the amount of work in testing to the total amount of work
The majority of project management systems on today’s market are either too focused on certain metrics and analytical capabilities or are expensive and intricate. That’s the main reasons why small and medium companies are missing out the prospect of proper control over the operation. That is why it makes so much more sense to use BI to track and control many of these metrics, especially because the analysis becomes much harder and time-consuming for the managers as the operations and the pace of conducting business grows.
To provide PMOs and executives with a way to track, manage, and make decisions on current projects and portfolios, provide higher portfolio visibility, project cost assessment, and give a more flexible tool, Microsoft created both affordable and feature-rich solution—Power BI. With “Project Online + Power BI + FluentPro Packs” nuclear combo, you get the whole list of advantages. You can, track, manage, and analyze the progress on portfolios, projects, quality, teams, and people—sort projects by type, status, issues, milestones, and risks. You can sort costs by project type and get summaries and a prognosis. If need be, you can go “deeper” and get a detailed view of every project, every timeline, project overlaps, and dependencies. You can also track resource allocation and utilization, as well as deliverables, team workloads, and all kinds of “history,” which is a gift on its own, because by keeping track of previous successes and failures, you can make the prognosis much easier and more accurate.