Measuring development teams
In each profession there has always been the urge from management to measure the productivity of employees and software development is not different in that. Over the 12 years that I am professionally involved in software development I have seen different attempts to measure teams or individuals on their performance in:
- Lines of code produced
- Bugs solved
- Bugs created
- Amount of features completed
- Hours worked
I totally understand that management wants to measure teams but the first thing to ask yourself is what you really want to know. All too often teams or individuals are measured on metrics because those are the metrics that they can measure and not the metrics that they need to measure.
Going after the metrics that can be measured instead of the metrics that should be measured usually results in measuring things that the team(s) see as a burden instead of something that can help them. The result of this is quite often that the team starts gaming the metrics which results in even less useful information from these metrics.
To measure what actually matters you will have to start with the basics and the basic measurement for teams should be the happiness of the employees that work in the team and the happiness of the customer that works with the products that are delivered by the team.
Step 1: measuring team happiness
In my opinion the most important metric to measure is the happiness of your employees. Does that sound weird? Isn’t measuring the output of a team what you want to know? Well I can guarantee you one thing, if your team is not happy, output will be crappy as well. So how do you measure something abstract such as team happiness? There is a try simple method for that, it is called the niko-niko index. nikoniko is a Japanese phrase that means smiling. Creating a niko-niko index for a team means that you will track the mood of the team members by the user of smile figures.
The best way to do this is to put out in the open, put it on a large whiteboard in the development room, or at the coffee machine. The idea is that you ask all team members to express their happiness about the day before they go home. Make a nice chart and just let the team members draw a smile expressing their mood of the day.
This gives a clear picture about the happiness of the team. This part of introducing a metric like this is easy, you just draw the chart and ask the team to draw a smile when they leave the office. It’s as simple as that. But please don’t step into the biggest pitfall here. When you ask people to express their feelings, even if it is in a a low profile way like writing down smiles, you should make sure not to neglect them and should act on them.
If you look at the whiteboard in the image you can see that several people are unhappy for multiple days in a row, pay attention to those people. Use the niko-niko index as a discussion starter. The most logical place to analyze the data from the niko-niko index is at the retrospective but be pragmatic about it depending on the contents of your niko-niko table. If you start the sprint and after 3 days you have three sad faces for all team members, act on it directly, waiting for the retrospective would be a bad decision in this case.
The second pitfall of this method that people should be aware of is not to over analyze it. If you see a string of happy faces and a single sad face and the person explains that he was just unhappy that day because the tester found a bug in a section he has been working on don’t turn it into a huge psychological analysis. And this is maybe the most difficult part of it, sense the discussion and know when to stop and when to continue with a deeper analysis.
Nevertheless I would say just try it with your team, it’s fun and can be an eye opener from time to time.
Step 2: Measure customer satisfaction
The second most important metric in software development is customer satisfaction. The second? Yes in my opinion employee happiness is more important than the customer satisfaction. Of course customer satisfaction brings the revenue to the company and is therefore very important. But in my opinion employee happiness has such a big impact on everything that happens in your company that without a doubt a negative employee happiness will have a negative impact on the customer happiness as well, maybe not directly, but it will definitely have it’s impact.
So how do you measure customer satisfaction? That’s what makes this metric difficult. Measuring customer satisfaction can be quite difficult depending on the project that you are running. If you are developing software for one specific customer it is quite easy because it is clear who is your customer and you probably have direct communication with that customer so the easiest thing is to just ask them for it.
When you are developing products it is more difficult to get feedback from your customers. Because who exactly are your customers? Quite often the mistake is made to consider the stakeholders (also sometimes referred to as the business side of the company) as the customer. I made that mistake as well and considered people like sales consultants, partner channel ma
nagers and product managers as customers of the development team.
Of course the “business” side of the company are very important stakeholders. They have a stake in the project and are the people that have the most communication with customers and end users. But don’t forget about your actual customers because in the end it is their happiness factor that results in more sales and therefore an increase in revenue for the company.
Measuring actual customer satisfaction is going to be a big challenge in product development companies. Therefore it is often neglected because the sales people are being considered as representatives of the customers and it is difficult to convince a company to put effort in gathering customer satisfaction information directly from the customer because it is often seen as criticism to their sales department. People should however not forget the most important success factors in software development.
Customer involvement is still the most important reason why agile projects have a much higher success rate than traditionally executed projects, which in general lack customer involvement during the development cycle. So if customer involvement is the most important success factor for your projects, shouldn’t you get the feedback directly from that customer?
Besides the fact that you can get the feedback directly from the customer, it will pay of to invest time in getting your actual customers closer to your development team because it will also increase the involvement of the customer in all other areas of the development process and even more important it will increase commitment of the development team to actually build what the customer needs.
So how do you get customer feedback directly? There are numerous of ways you can think of, some things that come to mind:
- Invite them to your review meetings where the product is demonstrated.
- Create key customer and user groups and involve them in discussions.
- Organize Webinars.
But, but, but…. what about velocity?
The velocity of a team is often used as a metric to measure the performance of a team because it seems logical to measure a team on the amount of work that they can complete in a sprint. But what is the use of measuring a team by a metric that has no reference other than the reference of the team itself. Don’t make the mistake to compare velocity of different teams, in the agile world you will look terribly silly when you are saying “Team A has a velocity of 35 and team B has a velocity of 25 so team A has a higher performance.”
Velocity is a nice metric for the team and product owner to use to decide the amount of work that roughly fits in a release cycle, but it is in no way a metric to measure the performance of the team. So can the velocity tell us anything of what is going on in the team? Of course velocity can be used to signal problems in a team. For example when you see the velocity dropping to an unusual low level sprint after sprint then there is probably something wrong within your team. It could be technical complexity, it could be demotivated team members, it could be an increased temperature in the office during the summer, basically it could be anything. All kinds of problems can reflect on a teams velocity. But my guess is that you will spot these issues with the employee happiness from the niko-niko index long before you can see it in the velocity trend.
Use velocity for what it is intended for, as a planning aid to decide what fits in a release cycle when starting a new release cycle and during the release cycle to anticipate what can be achieved before the end date of the release cycle.
But I want more metrics to measure my teams.
Ok, so you want more metrics to measure your teams? It is not that the employee happiness and the customer happiness metrics that I described are the only means to measure a team. There are definitely more ways to measure a team. Don’t look for default metrics that work for all teams, in my opinion the two metrics mentioned in this post are the only two metrics that apply to all software development teams regardless of the methodology that is used and regardless of the organization where the teams reside.
If you want to specify other metrics discuss it together with the teams. Any metric being used with a team is part of the team’s identity so together with the team you should decide on the metrics that can be measured for the team and the usefulness they bring to the team and secondly to the organization.
I will return to the discussion on how to setup metrics as part of a team’s identity in a future blog post so stay tuned!
Trackback from your site.