Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Kirby Montgomery says monthly or quarterly monitoring must pair captured data with context, so non-technical investors understand why numbers moved. Code & Co. tells engineering teams it is not there for gotchas, but to translate their story; if the work is helpful, engineers lean in rather than game the metrics.

Related Insights

Top product teams like those at OpenAI don't just monitor high-level KPIs. They maintain a fanatical obsession with understanding the 'why' behind every micro-trend. When a metric shifts even slightly, they dig relentlessly to uncover the underlying user behavior or market dynamic causing it.

Kirby Montgomery says the first question about a good-looking engineering dashboard is who actually uses it and what decision it changes. He and Jim Milbury describe dashboards or cloud artifacts vibe-coded the night before diligence, hosted locally, and never shared with the C-suite or board.

Collecting data is insufficient; its true value is in enabling difficult conversations about operational bottlenecks. Many teams understand data's importance in theory but fail the last-mile work of making it meaningful. Data must be structured to spark discussion and diagnosis, not just reporting.

When leaders enforce memorizing every metric without a connecting narrative, teams resort to cherry-picking data to fit a story. This creates an illusion of data-drivenness while masking a lack of true strategic understanding and encouraging superficial analysis.

Metrics become poor measures once they become targets (Goodhart's Law). To effectively inform upper management, provide context and a 'gut feeling' through periodic demos and brief, 4-5 bullet point status reports that get read, rather than long reports that get ignored.

Data collection is only the first step. True value comes from evaluation (comparing results to objectives) and learning (generating insights to inform future strategy). Most organizations get stuck on measurement, creating backward-looking reports instead of forward-looking plans.

To get buy-in for developer experience initiatives, don't use generic metrics. First, identify leadership's primary concerns—be it market share, profit margin, or velocity. Then, frame your measurements and impact using that specific language to ensure your work resonates.

Unlike sales or marketing, engineering departments historically operated without clear, scientific KPIs. Decisions were based on approximations like story points, leading to opacity. AI now enables the same level of data analysis for engineering, creating a new "engineering intelligence" category.

Instead of ad-hoc reporting, a disciplined approach involves a comprehensive monthly metrics package prepared by RevOps. This "fact pack" enables leaders to dedicate focused time at the start of each month to analyze performance, uncover insights from combined metrics, and proactively investigate anomalies rather than reacting to problems later.

The most impactful quality metrics are not internal measures like bug counts but those directly linked to customer and business outcomes. QA professionals increase their influence by framing their findings in terms of business impact, financial exposure, and customer risk.