We Are Measuring Developer Productivity Wrong

Because of the nature of their work, developers face particular challenges when it comes to measuring productivity. Because developers deal with a variety of tasks on a daily basis, it can be challenging to implement a consistent productivity measure. In these circumstances, how can we measure developer productivity effectively?

What do businesses want?

Let’s begin by figuring out what businesses want from developers. It will be simpler to specify how to measure developers in a meaningful way once we understand this.

Why would companies want to gauge the productivity of developers? At first, speed is everything when a business is attempting to determine its product-market fit. Each new feature directly contributes to growth, developers create features quickly, and everyone is satisfied. The company begins hiring in order to maintain operations.

The growth curve, however, eventually flattens out. Development slows down, the team grows, and tech debt accumulates. The team is perceived by the company as being less “productive.” The management hopes to return to the era of lightning-fast feature development in order to reduce expenses and boost profits.

The company knows that the team isn’t working hard enough, but they don’t know why or how to make things better. They ask the CTO or VP of Engineering for metrics to demonstrate that the team is delivering value and operating efficiently. Leadership is under pressure to identify some metrics that will “prove” the team’s value and assist them in making performance-related decisions. In the end, the company desires:

• Quicker feature and project delivery to increase revenue.
• To identify top performers for rewards or promotions and underperformers for possible termination.
• Metrics that strongly correlate with company profit so they can effectively manage the team.

Organizations View Development Teams as Production Facilities

Developers are frequently seen by tech companies as being comparable to factory workers in an automobile manufacturing facility. Each feature is viewed as a profit-driven competitive advantage. It’s like opening a faucet that pours out money if the process of creating new features can be accelerated.

In order to produce more cars more quickly, manufacturing companies optimize by measuring each step, clearly mapping out the workflow, and continuously improving. The temptation is to approach software development with the same mindset.

“But software development isn’t like manufacturing; each feature is new and unique,” one may argue. You’re right, too. Since each feature is unique, we cannot just dissect it into small parts and accelerate each one separately. Iterative development means that after making one change, you may need to make three more adjustments, and occasionally you may need to completely reconsider the first change.

On the other hand, we adhere to certain standard processes. We conduct code reviews, testing, and other reviews on a regular basis. We always repeat certain upstream processes, such as planning, estimating, design, and ideation. Therefore, although each feature is distinct, the method by which it is used is not. What if we concentrated on quantifying these processes?

Issues with Typical Metrics

Let’s examine a few frequently used metrics: throughput, issue cycle time, story points per developer, deployment frequency, and lead time for changes. The majority of us have either used or heard of these metrics, so why not limit ourselves to the most widely used ones?

1. Limited Scope: Many of these metrics overlook other important steps and only concentrate on the development process. For instance, DORA metrics include mean time to recovery, change failure rate, deployment frequency, and lead time for changes. However, they frequently overlook product management, QA, and other phases.
2. Easy to Manipulate It is possible to game these metrics. To inflate the numbers, developers may simply divide the tasks into smaller ones or estimate them higher if you set a goal to double the story points completed in six months. The business outcomes stay the same, but all of a sudden, your team’s story points have doubled.
3. Lack of Diagnostic Power: While metrics such as throughput can indicate the existence of an issue, they are unable to pinpoint its exact location. Is the low throughput a result of sluggish developers, backlogged QA, or pending feature approvals?
4. Team-Specific Variability: Processes vary amongst teams. If your clients don’t require frequent deployments, metrics like deployment frequency may not matter. Without taking into account their particular situation, you cannot simply tell a team to alter their process in order to achieve a metric.
5. Weak Connection to Business Value: A lot of widely used metrics aren’t directly related to business outcomes. Technical performance, which doesn’t always equate to real profit, is frequently the main focus.

How About Engaging in Conversation?

To be honest, talking is the cheapest and easiest way to improve team processes; retrospectives, surveys, and one-on-ones are all excellent tools for this. However, there are also difficulties here:

1. Bias: Individuals are biased by nature. We have a tendency to assume that people share our opinions and actions more than they actually do (false consensus effect) and seek out information that confirms our preexisting beliefs (confirmation bias). It is difficult to obtain unbiased insights because of this.
2. Limited Perspective: People are only able to report what they observe. While QA may accuse developers of slow progress, developers may argue that code reviews take too long. However, a challenging release process could be the true bottleneck. When I implemented metrics, I found that 30% of our cycle time was just waiting for releases—something that was easier to automate than fixing code reviews, as the team pointed out.
3. Self-Protection: Individuals have a tendency to shield themselves and shift responsibility. Nobody wants to acknowledge that they caused a two-month delay in a project. Humans are naturally inclined to avoid feeling bad or like the “bad guy.”

This Is Not the End

A better approach to measuring and enhancing productivity must exist, one that takes into account the particulars of software development as well as the necessity for metrics in the business.

I’ll go over how to pick important metrics, how to apply them successfully, and how to use them to produce tangible outcomes for the team and the company in my upcoming post.