Whether you think developer productivity can’t be measured (like Martin Fowler) or are obsessed with getting precise insights into what slows down developers or makes teams collaborate better, as an engineering leader, you’ll probably have to and want to apply some kind of developer productivity metrics system.

Instead of debating whether you can measure developer productivity or whether you should, let’s figure out how to do it correctly.

Developer Productivity: What to Track, When, Why, and How?

One thing everyone seems to agree on — McKinsey, Gergely Orosz, Kent Beck, Martin Fowler, Dave Farley, and hundreds of engineering leaders working at organizations of all shapes and sizes — is that some standard metrics systems, like the DORA metrics, are the best we have so far.

At least as a starting point. When Orosz asked 17 engineering leaders at 17 companies what metrics they track, he got 17 different answers.

And that’s OK. When you go just one level beyond the DORA metrics, every company optimizes for what they care about.

The secret to getting developer productivity metrics right is to measure what matters to you, what matters to your engineering teams, and what is applicable to your way of working, your product, and the stage of the company.

The 4 Rules of Measuring Developer Productivity the Right Way

1. Contextualize Developer Productivity Metrics to the Team’s Specific Goals

Not all teams are alike. The metrics that matter for a team working on a highly experimental, rapidly evolving product will be different from those for a team maintaining a large, stable enterprise application. Engineering leaders should tailor productivity metrics to the specific context of the team and the project.

“I’m a big proponent of the DORA metrics, but it’s important to understand the context for which you’re going to apply them. We have Atlas, which is a SaaS product, and productivity can easily be measured with time to deploy, time to failure, and time to recovery. That’s the context — cloud-native development — that the DORA metrics were meant for.

But we also have non-SaaS offerings, and our customers wouldn’t want to download a new distro package every day, not even every week or even every month. We have a major version cycle of one year.
You can also measure time to ‘deploy’ in that context; it’s just going to be slightly different —- not measuring packages made available to download, but in our staging environment.”

Tara Hernandez, MongoDB

Full episode of HangarDX podcast episode on Designing MongoDB developer productivity at Mongo DB

2. Developer Experience => Developer Happiness => Developer Productivity

The goal of developer productivity metrics should be to identify and reduce inefficiencies and friction in developers’ workflows. Although they’re different concepts and shouldn’t be confused, Developer Productivity, Engineering Effectiveness, and Developer Experience are intertwined.

Quantitative metrics like DORA or SPACE should be paired with developer satisfaction surveys to diagnose waste and friction that frustrate developers. Ultimately, good developer experience practices lead to developer satisfaction, which leads to developer productivity.

3. Measure what Matters (and Stop When it Doesn’t)

When you understand the issues in your organization, you can start tracking whether they are getting better or worse. A good way to avoid metrics overload is to measure only what really matters to your team or organization in addition to the basic DORA or SPACE metrics.

“When you have identified a metric/signal that is too high or too low, you measure it until it’s where you wanted it to be. Then you STOP. 

Otherwise, you end up with a dashboard with over 100 metrics that look like the stock market!

Half of the metrics are going up, and half of them are going down, and no one knows whether that’s good or bad.”
Chris Westerhold, HTEC

4. Tie Metrics with Dollars

Developer productivity metrics are a means to an end, and different layers of organization need different sets of metrics. Many leaders see developer experience as just a cost center. It’s key to present what bad developer experience is costing the entire company in dollars. Build metrics that can help make that visible to them!

One of the ways is the classic triangle of cost, speed, and quality. By understanding that one of them has to be compromised unless friction is removed within the company, the C-suite will have an understanding of how the engineering organization is affecting the business results. Sixty-nine percent of developers are losing more than 8 hours a week to inefficiencies, according to Atlassian’s Developer Experience Report). That’s 20 percent of wasted developer hours!

3 Ways How Developer Productivity Metrics Can Go Wrong

1. Measuring the Productivity of Individual Developers

Measuring the productivity of individuals in any way — be it the lines of code or the number of closed tickets or something else is pointless. No one likes to be micromanaged, and software development is a team effort anyway. What you measure should say to developers that you’re measuring that with them, not against them.

2. Metrics for the Sake of Metrics

Don’t measure things that are easy to measure but don’t have a real impact. As Goodhart’s Law says — when a measure becomes a target, it ceases to be a good measure. When you set very specific goals for people, especially if those metrics are tied to financial rewards or career progress, they will tend to optimize for those metrics no matter the consequences.

Avoid metric pitfalls by setting your developer productivity metrics in pairs of positive and negative metrics. So, if the goal is to lower code review times, you don’t want to achieve that by engineers just approving whatever comes their way and the number of bugs in production going up.

3. Not Caring About Culture and Satisfaction

We tend to think of productivity as just an output. But as an engineering manager, you also have to consider people. That’s why the SPACE framework takes into account developer satisfaction and well-being. Because, again, a happy developer equals a productive developer.

“Engineers are not people working in the widget factory.

I don’t want to think about productivity as telling a developer — last week, you produced 50 widgets, and this week, you have to produce 51.

What if he told me that he produced 51 widgets this week, but he was really unhappy because the widget machine was broken, and he did it with brute force? You realize that if you invested in fixing the widget machine, you’d get 100 widgets produced a week!”
Max Kanat-Alexander, LinkedIn

Listen to the full Hangar DX podcast episode on Developer Productivity and Happiness framework at LinkedIn

Interested in measuring productivity and improving developer experience?
Join The Hangar DX, a curated community for developer-experience (DX) enthusiasts!

The Hangar DX is a community of senior DevOps and senior software engineers focused on developer experience. This is a space where vetted, experienced professionals can exchange ideas, share hard-earned wisdom, troubleshoot issues, and ultimately help each other in their projects and careers.