One of the main reasons why projects fail is not defining clear project success criteria. A project can only be successful if the success is defined. And this ideally upfront. Unfortunately, I have seen many projects that skipped this part completely.
When starting a project, it’s essential to work actively with the organization that owns the project to define success criteria across three levels:
1) Project delivery
2) Product or service
3) Business
Project Delivery Success
Project delivery success is about defining the criteria by which the process of delivering the project is successful.
Essentially this addresses the classic triangle «scope, time, budget». It is limited to the duration of the project and success can be measured as soon as the project is officially completed (with intermediary measures being taken of course as part of project control processes).
Product or Service Success
Product or service success is about defining the criteria by which the product or service delivered is deemed successful.
For example, the system is used by all users in scope, uptime is 99.99%, customer satisfaction has increased by 25%, operational costs have decreased by 15%, and so on.
These criteria need to be measured once the product/service is implemented and over a defined period of time. This means it cannot be measured immediately at the end of the project itself.
Business Success
Business success is about defining the criteria by which the product or service delivered brings value to the overall organization, and how it contributes financially and/or strategically to the business.
For example, financial value contribution (increased turnover, profit, etc.) or competitive advantage (market share won, technology advantage).
Overall Project Success
These levels combined will determine your overall project success. You can be successful on one level but not others. This sounds all good in theory, but in practice, it is not so easy to define the criteria for levels 2 and 3. This is one of the main reasons why so many organizations only look at level 1: scope, time and budget. They are easy to define and measure.
Personally, I think level 1 matters very little if levels 2 and 3 are not met. So in spite of the “project management triangle”, the fact is that delivering a project on time, in scope and in budget is not enough. The project must be delivered successfully – meaning that the objective(s) that motivated the project in the first place have to be reached.
This is where so-called OKRs come into the game.
What are OKRs?
OKR is an abbreviation for Objective and Key Result. The concept was invented at the Intel Corporation and is widely used amongst the biggest technology companies in the world including Google and Zynga.
OKRs are originally meant to set strategy and goals over a specified amount of time for an organization and teams. At the end of a work period, your OKRs provide a reference to evaluate how well you did in executing your objectives.
You can use the same concept for defining project success. Spending a concerted effort in identifying your project strategy and goals, and laying it out in a digestible way with OKRs can truly help your project team and stakeholders see how they are contributing to the big picture and align with other teams.
Objectives
Any project has one or more objectives. The goal of setting an objective is to write out what you hope to accomplish such that at a later time you can easily tell if you have reached, or have a clear path to reaching, that objective. Choosing the right objectives is one of the hardest things to do and requires a great deal of thinking and courage to do well.
Key Results
Assuming your Objectives are well thought through, Key Results are the secret sauce to using OKRs. Key Results are numerically-based expressions of success or progress towards an Objective. All Key Results have to be quantitative and measurable. As Marissa Mayer, a former Google’s Vice President, said:
“If it does not have a number, it is not a Key Result.”
The important element here is measuring success. It’s not good enough to make broad statements about improvement (that are subjectively evaluated). We need to know how well we are succeeding. Qualitative goals tend to under-represent our capabilities because the solution tends to be the lowest common denominator.
For example, if I create a goal to “launch new training for the sales team” I might do that for one sales member. If I alternatively make a Key Result of “train 50 sales team members” and only train 10, I’ve still 10x-ed my original goal.
Don’t Turn OKRs Into a Project Task List
Do you measure effort or results? Are your OKRs focused on your objective or on the means to get there? As we mentioned before, when used correctly, OKRs define success criteria for a project. OKRs should determine whether a project achieved success. But to do that, OKRs cannot be based on activities for three main reasons:
1) We want a results-focused culture, and not one focused on tasks.
2) If you did all your tasks and nothing improved, that is not a success. Success is improving something: customers are more satisfied, sales are higher, costs have been reduced.
3) Your project is just a series of hypotheses. An idea is just a non-validated hypothesis. In the same way, we don’t know if our project will improve our results or add value to the organization. The project is just a hypothesis so you cannot attach your OKRs to a non-validated bet. See No validation? No Project! for more on this topic.
Nobody works on projects as a hobby. Behind every project is a desire to improve one or more metrics. So, instead of tracking the delivery of a project, we should measure the indicators that motivated it in the first place.
Use Value-based Key Results
There are two basic types of Key Results:
1) Activity-based Key Results: These measures the completion of tasks and activities or the delivery of project milestones or deliverables. Some examples of Activity-based Key Results are:
– Release a beta version of the product.
– Launch a new feature
– Create a new training program.
– Develop a new lead generation campaign.
– Write a solution design document
Activity-based Key Results usually start with verbs such as launch, create, develop, deliver, build, make, implement, define, release, test, prepare and plan.
2) Value-based Key Results: These measures the delivery of value to the organization or its customers. Value-based Key Results measure the outcomes of successful activities. Some examples of value-based Key Results:
– Increase Net Promoter Score from X to Y.
– Increase Repurchase Rate from X to Y.
– Maintain Customer Acquisition cost under Y.
– Reduce revenue churn (cancellation) from X% to Y%.
– Improve average weekly visits per active user from X to Y.
– Increase non-paid (organic) traffic from X to Y.
– Improve engagement (users that complete a full profile) from X to Y.
The typical structure of a Value-based Key Result is:
Increase/Reduce Metric M from X to Y
Where X is the baseline (where we begin) and Y is the target (what we want to achieve).
Using the “from X to Y” model is better than writing a change in percentages because it conveys more information. Compare the two options below:
1) Increase the number of new users by 20%.
2) Increase the number of new users from 4000 to 4800.
Option 1 can be confusing since it’s hard to tell how ambitious the target is. Are we talking about increasing the number of new users from 500 to 600 or 4000 to 4800?
Examples
When project teams start with Value-based OKRs, it is common for them to get stuck listing activities as Key Results. To convert those activities into value, think about what would be the consequences of being successful with this task. What would be the desired outcomes?
If we are successful with X,
– Key Result #1
– Key Result #2
– Key Result #3
Example 1
If we are successful with the new campaign, we will
– Increase Net Promoter Score from 29 to 31%
– Reduce churn from 3.2 to 2.7%
Example 2
If we successfully migrate the platform, we will
– Reduce infrastructure costs from X to Y.
– Maintain availability during migration in 99,99%.
– Maintain revenue of $ X.
OKRs as a Communication Tool
As you might have guessed by now, effective OKRs are widely shared and meant to be understood by project teams, related teams, and stakeholders. In that regard, they can serve as a communication tool for directing teams to solve complex challenges with constraints.
As a communication tool, OKRs bring two key things to an organization:
1) Easily digestible direction such that every project member/stakeholder in the organization understands how they contribute to the mission; aka focus.
2) Expectations amongst teams and their individual members; aka accountability.
Closing Thoughts
Defining measurable results becomes easier as you learn what you should be measuring and what ultimately matters for your project and business. In my work with large project teams, I find that the quality of OKRs has a good correlation with the understanding of the project. Just going through the exercise of either defining OKRs, or reworking current project plans into OKRs is a highly effective evaluation tool.