Roger joined our team as a strong software engineer with valuable experience and lofty expectations from our team. We had every reason to believe Roger would be a great hire. And from the very beginning, Roger struggled to accomplish any work assignment. When I checked his work history I found that it took him forty-seven days to complete his first change in our codebase. Yikes. The next question I pondered was “how long does it take on average for a developer to complete their first pull request?” I needed a larger dataset to tell me more about the one data point about Roger. This question, and others like it, would send me down my very first rabbit hole of collecting, organizing and analyzing the data from our software engineering teams.

I like to say “your data tells a story.” Every team and organization has data buried in the tools they use that shows how the team operates. I use this data to drive decisions to reward people, discover problems, tweak process and operate more efficiently as a team. I have managed or co-managed 200+ software engineers and dozens of teams practicing scrum or kanban. I am routinely amazed at insights we can attain simply by collecting, organizing and analyzing the data from our tools.

The systems, tools and apps we regularly use are filled with information about how we work, work effiency, when we work and general interactions within the team. Data alone is of little value similar to an unorganized bucket of magnetic plastic letters. We must find ways to organize, process and extract insights that drive action and the decision making process. The most powerful metrics are those that motivate us to take action.

Building metrics from our data gives us the power to make data-driven decisions in our teams and organizations. Using metrics, we can rely less on anecdotal experiences with people and teams which are often riddled with bias and difficult or impossible to confirm (or deny!). When we have a team member that we suspect is struggling then we should leverage the data to further our understanding of how this person performs within both absolute and relative metrics.

I’ll be completely honest: sharing some of this information makes me a tad uncomfortable because this information can be abused by bad managers. I mean this sincerely: use these metrics carefully to reward team members that are doing great work, analyze situations with underperforming employees, reward the entire team for jobs well done, project delivery dates based on project scope or build forecasts for staffing a project based on historical workloads.

Goodhart’s Law [1] states “When a measure becomes a target, it ceases to be a good measure”. The team will alter their behavior as they become more aware of how their performance is measured and incentivized. Observe the organic behavior of the team for a period of time while using this information to make specific and targeted decisions. Predicting the associated changes in behavior from incentives is difficult so another reminder to use good judgement.

Do not hide the metrics from the team nor lie about your intentions or what you have learned. The team will pickup on dishonesty and impure motives which ruins trust. Publicly highlight and reward the great work that team members are performing. Quietly and privately work with team members that are struggling to find success.

Back to Roger. We recognized there was a clear problem and initiated more direct oversight and coaching. Through more candid conversations, we learned that Roger didn’t actually like the culture of our organization nor our software engineering practices. It was a bad fit for us and him. Roger left after a few months. As a result, I learned that I should talk more directly about our teams and culture during the recruiting process to be as honest and as clear as possible to candidates.

Here’s another story. Over the course of 18 months we scaled our software engineering head count to over 100 people across North America. In hindsight, I think our oversight of people was too lax. I discovered multiple people that were doing nothing. Software engineers that were simply choosing to not do any work. I measured their contributions and it scored at zero. Dozens of months of people-time with nothing to show for it.

On a positive front, we used data to confirm our anecdotal accounts with the top performing software engineers on the team. In those cases, the data helped us make legitimate cases for promotions and compensation. In other situations, the data drove decision making to make informed choices on personnel as we reduced (contracting) teams near the end of the project.

To leave the reader with actionable information for KPI’s for Software Engineering teams I have this for you. First, ensure the teams are making changes in the codebase through pull requests. Second, review the pull requests by person per week. Pull requests are nearly an industry standard and teams should be familiar. Also, pull requests are easy to work with and track. Pull Requests by person per week is the best KPI I can recommend that is near universal across teams and organizations so start here. To grow your metrics it will depend on the organization so I recommend to start simple and grow them intentionally. Ask yourself questions to see if you can formulate an answer with the data.

Remember, your data tells a story. Use the data to gain a tactical advantage for your team and organization to keep winning.

[1] https://en.wikipedia.org/wiki/Goodhart%27s_law