The Ghost in the Code: Debunking Myths About Engineering Metrics

By now you've seen the "~9.5% of software engineers do virtually nothing: Ghost Engineers (0.1x-ers)" paper from Stanford making the rounds this past week. It claims that nearly a tenth of software engineers are “ghost engineers" who contribute nothing to your organization and should be fired. This is a big claim, and it’s also very wrong and missing important context. Here’s a breakdown on why:

Measuring engineering productivity isn’t (and it shouldn’t be) about identifying hidden non-contributors, but about understanding how teams work together and can collaborate more effectively to benefit your organization. 

Software development is inherently collaborative and complex. Assigning certain software engineers the “ghost” label diminishes the true nature of engineering—teams solving tough problems as one unit.

1. "Ghost engineers"? Let's pump the brakes
2. When the data becomes the villain
3. Software engineers shine with the right metrics
4. How to actually measure your engineering team efficiency
5. From ghost stories to team success

The challenge with the “ghost engineer” claim lies in its reliance on metrics that isolate individual output, such as code commits, while overlooking other significant contributions like mentoring, debugging, and code reviews. These are critical to building a strong team, yet they don’t always show up in traditional measures of productivity.

The study itself leaves several key questions unanswered:

• How are “ghost engineers” identified? One of the paper’s authors, Yegor Denisov-Blanch, outlines research that aims to model and rate software contributions based on factors like time required to produce code, quality, and maintainability. However, the model was trained on an extremely small dataset—just 70 Java commits from 18 engineers—using expert assessments to guide its ratings. While correlations for certain dimensions, like “time required to produce,” were relatively strong (up to 0.86), others, such as “code maintainability,” fell as low as 0.30.

• What about the dataset? Although the researchers reference a larger dataset of 1.73 million commits from 50,935 engineers, this data wasn’t used to train the model itself, only to inform the broader research. The findings hinge on an extrapolation that concludes roughly 4,800 engineers (9.5%) perform below the “0.1x median engineer” level, yet details on methodology remain sparse.

• Where’s the peer review? The paper hasn’t undergone peer review, leaving its methodology, assumptions, and conclusions unverified by the broader academic community. Without transparency or external validation, its claims remain speculative at best.

Metrics without context can do more harm than good. Misinterpreted data leads to misplaced blame, eroded psychological safety, and increased burnout among engineers. Productivity tools should provide clarity, not create unnecessary pressure.

For example, a team’s velocity might drop during a period of onboarding new developers or addressing technical debt. While these activities are crucial to long-term success, trying to measure them using metrics without context can paint a misleading picture. The real goal should be insights, using tools like Flow,  that guide better teamwork, not metrics that penalize.

Flow takes a different approach, focusing on actionable insights that empower teams rather than isolating individuals. For example, identifying bottlenecks in PR reviews or balancing workloads across the team can significantly improve overall efficiency.

The goal is to shift the conversation from individual achievements to team collaboration, ensuring smoother workflows and better results.

The tech industry has long relied on outdated measures of engineering success. It’s time to prioritize metrics that promote developer happiness, psychological safety, and team engagement. Happy and engaged engineers create better software—it’s a win for everyone. Here are some of the biggest benefits on measuring your teams efficiency the right way: 

• Lower operational expenses: Optimized workflows and resource allocation lead to faster development, reducing both costs and time to market. Higher-quality software minimizes downtime, further driving cost savings.

• Better developer experience: Simplified processes allow developers to focus on coding, reducing interruptions and frustration. Clear communication within teams improves collaboration and overall satisfaction.

• Enhanced user satisfaction: Streamlined development processes ensure quicker delivery of features and improvements. This enables developers to concentrate on building meaningful, customer-centric experiences.

• Boosted Team efficiency: Setting clear priorities helps teams stay focused and organized, maximizing output while minimizing wasted effort. This makes room for more strategic work and faster progress.

• Increased Capacity for innovation: By automating routine tasks, teams can dedicate more time to exploring new ideas and refining features, driving creativity and progress across the board.

At Flow, the focus is on creating a culture of equity and trust, with tools that respect the complexities of engineering teamwork. Metrics should enable informed decision-making without compromising the well-being of the team.

It's time to move beyond narratives like "ghost engineers" which rely on outdated assumptions about software engineering. Instead, embrace a more thoughtful, data-driven approach to managing engineering teams. With smarter metrics and tools, teams can foster collaboration, transparency, and innovation without compromising their well-being.

Flow is dedicated to helping teams thrive by delivering insights that drive better collaboration and productivity. Learn how Flow can transform your team’s processes and help rewrite the story of engineering success.