For 30 years, software product and engineering leaders have been asked the same questions by their business counterparts:

• What should we build?
• Why are we building this instead of that?
• When will the feature ship?
• How much will it cost?

For 30 years, our product and engineering leaders have been left to answer these questions with a mix of incomplete data, human intuition, and guesswork.

Why is software development the only modern business process that we excuse from being measurable and efficient? It’s an ironic paradox that many of us in the software industry think that software can optimize any process — except for the process of software development.

Consequently, shipping software on time and budget is not the norm. According to McKinsey, 78% of software projects are late, over budget, or don’t ship at all. We founded Bloomfilter so product leaders finally have the tools they need to reverse this statistic.

We must tear the lid off the black box.

Software development has evolved a lot over the years. We’ve broken projects into smaller pieces and made them iterative, created specialized roles, and made use of reusable components. On their own, these innovations have made software development faster and more efficient. But together, they’ve diffused accountability and made the process hard to understand, much less optimize.

Business doesn’t run in “agile.” Business leaders have quarterly sales goals, annual production targets, and multi-year capital plans. These plans have to be synchronized with the product team — leading to the constant questions around “what”, “when”, and “how much?” Business operates in the language of dollars and cents. Dashboards measuring sprints and story points are great for scrum masters and engineers, but they aren’t making business-product collaboration any easier.

Bridging this gap has fallen to the product leaders. We have stuck them into the business/product gap and said “go fix this… be our translator.” It’s an impossible job. We demand, in the same person, a product visionary, financial operator, and team builder.

Imagine a planning meeting where a product leader is asked to report on the status of a new feature. In most cases, they collect input from engineering managers to see if things are on track. They might even take a look at a project tracking tool. But ultimately, their projection is based on guesswork and the hope that everyone delivers on their commitments.

This is not a knock on product leaders. The ability to use data to spot risks and measure confidence levels and predictability would be invaluable, if it were available.

Once we understand how the sausage is made, we can improve the sausage making process.

The first step in improving a process is understanding it. Just like companies scrutinize their sales pipeline or marketing funnel, we must be able to see real-time project progress, roadblocks, and investments. We need objective data to help us understand where the process goes off the rails, what factors lead to a derailment, and what we can do to make the process more predictable and efficient. We need to be able to measure things in a language everyone understands.

At Bloomfilter, we use process mining to read time-series data from systems across the software development lifecycle. This includes core development platforms — project management tools, code repositories, CI/CD platforms, design tools — and other systems that contain business context (CRMs, ERPs, HCMs, etc). By stitching together and contextualizing this information, we can see how the process actually works, identify areas for improvement, and model future scenarios.

Armed with this data, everyone can have a shared view of objective reality. We can stop guessing about the status of projects. We can remove human bias from our predictions of whether we’ll meet deadlines. The need for human interpretation, collaboration, and decision-making remains, but we can do that from a place of shared truth and understanding. Shared truth radically transforms conversations between product teams and the business creating trust and better collaboration.

The limit does not exist.

We aren’t the first ones to recognize that we’ve left software development behind as we’ve optimized processes beyond our wildest imaginations. But judging by the fact that product managers are still piecing together status updates Excel and Powerpoint, the world has a long way to go.

It’s important to note, we don’t believe the solution is another tool that tries to monitor developer efficiency. We reject the notion that you can solve a process problem by babysitting individuals through Big Brother metrics like “hands on keyboard time” or “lines of code written”. Great people stuck in a bad process are always going to struggle.

Since we began this journey, we keep being told this is a really hard problem to solve — the interesting problems always are. That’s the reason we have assembled a team with the experience to meet the ambition of the challenge. But we also keep being told that if we can just move the needle a little bit, it will create massive amounts of good in the world. That’s what gets us up in the morning and gets us so excited.

We’ll know we’ve done our job when we have reversed the trend. Imagine a world where 78% of software projects ship on time and under budget. Imagine how many more problems we can solve as a society if our greatest single accelerator, the development of software, is accelerated.

We’d welcome you to join us on the journey. There’s a long path ahead, and we are just getting started.

Andrew, Erik, Blake, Chris & the Bloomfilter team