Improving testing operations while you automate testing

Automate the tests, instrument the testing.

Test automation speeds up the execution of tests. It does little for the system around them: how work arrives at QA, how long it waits, how defects bounce, and how release decisions get made. Most automation programs measure scripts written and coverage percentage, then wonder why cycle time and escaped defects barely move.

Bloomfilter, with Celonis as the process mining layer, makes the testing operation itself observable. We reconstruct how work actually flows from story to test to defect to release, using the event data already sitting in Jira, Azure DevOps, Git, CI/CD and your test management tools. That lets you point automation at the constraints that matter, prove the investment paid off, and keep improving after the scripts are written.

Why test automation programs stall

Automation removes the cost of running a test. Most of the delay in a testing operation lives somewhere else: in the queues, handoffs and rework that happen between test runs. When those stay invisible, automation just moves the bottleneck.

Where time and quality leak	What it looks like	Why automation alone doesn't fix it
Wait time before test	Stories sit in "Ready for QA" for days. Environments and test data aren't ready.	Faster execution doesn't shorten a queue.
Defect ping-pong	Defects reopen, bounce between dev and QA, or get rejected as "works as designed".	The loop is a communication and requirements problem.
Late testing	Most test activity bunches up at the end of the sprint or just before release.	Scripts get run late if work arrives late.
Flaky and low-value suites	Reruns, quarantined tests, red builds people ignore.	More tests can mean more noise and slower pipelines.
Misaligned coverage	Automation targets what's easy to script, not where escaped defects come from.	Nobody has linked defects back to process paths.
Release by negotiation	Go or no-go relies on status meetings and spreadsheets.	Test results exist but aren't tied to risk and flow.
Unprovable ROI	Leadership asks what the automation program returned.	Script counts and coverage don't show a business outcome.

AI coding agents make this sharper. As code volume rises, testing becomes the constraint, and a team that can't see its own flow will automate in the wrong places faster.

What process intelligence sees that test tools can't

Test tools report on tests. Process intelligence reports on the work moving through testing. Every status change, pull request, pipeline run, test execution and defect transition is a timestamped event. Stitched together, they show the real path each story took and where it waited.

The rework loop under "In test" is where most programs lose time, and it is the one part of the flow no test tool reports end to end.

That view answers questions a QA dashboard can't:

Where does work wait before and during test, and for whom?
Which teams, components or requirement types generate the most defect loops?
Which escaped defects came through paths with low automated coverage?
Are automated tests catching defects earlier, or only re-running what manual testers already cover?
Did the automation investment shorten lead time and cut escapes, measured before and after?

Where Bloomfilter and Celonis each fit

You keep your test automation stack. Bloomfilter and Celonis sit above it as the measurement and improvement layer, so nothing you have built gets replaced.

Layer	Who	Role in testing operations
Test execution	Your tools (Tricentis, Selenium or Playwright, Xray, Zephyr, qTest and others)	Write and run tests. Unchanged.
Process mining engine	Celonis	Enterprise process mining platform, object-centric data model, action flows.
SDLC context layer	Bloomfilter	Connectors and a data model for SDLC systems, an automatic ontology of how your teams actually work, and benchmarks and KPIs for engineering and testing.
Agent layer	Bloomfilter Flora and agent mining	Answers "why is this release late?" in plain language, and measures what AI coding and testing agents actually do in the flow.

What we are not. Bloomfilter is not a test automation platform. We don't write, run or maintain tests. We show where testing work waits, loops and leaks, and whether automation is fixing it. That makes the automation you are already paying for easier to aim and easier to prove.

If you already run Celonis for business processes, extending it into the SDLC is the same muscle on a new process. It also gives any AI-in-testing experiment what it needs to be credible: a baseline of the flow you are changing.

Six use cases and what we measure

Six use cases cover most of the value. Each one ties an operating question to metrics we can baseline from your existing data before any automation changes, then track as automation rolls out.

Use case	Question it answers	Metrics we baseline and track
Target automation where it pays	Which flows, components and teams drive the most test effort, rework and escapes?	Test effort by path, defect density by component, escaped defects by path
Remove queue time around test	Where does work wait for QA, environments or data?	Wait time in "Ready for QA", environment wait, test cycle time
Break the defect loop	Why do defects reopen or bounce?	Reopen rate, dev-QA handoffs per defect, rejected-defect rate, time to fix
Shift testing left	Is testing happening earlier in the sprint, or still bunching up at the end?	Share of test activity by sprint day, defects found before and after merge
Keep the suite healthy	Which tests are flaky, slow or never catch anything?	Flaky rerun rate, pipeline time spent in test, tests with no catches over a set number of runs
Prove the ROI	Did automation shorten lead time and cut escapes?	Lead time for change, change failure rate, escaped defects, manual test hours, before and after

Lead time for change and change failure rate are two of the DORA software delivery metrics, so the results read in the same terms your engineering leadership and finance already use.

How an engagement works

Measure first, then automate, then prove it. The durations below are a typical starting point, not a fixed schedule. We set the actual timeline with you once we know your scope and systems.

Baseline (about four weeks). Connect Jira or Azure DevOps, Git, CI/CD and your test management tool. Reconstruct the testing flow for two or three product teams. You get where time and quality leak today, with numbers.
Target (about two weeks). Rank automation candidates by measured rework, wait time and escape risk instead of by ease of scripting. Agree the three to five KPIs that define success.
Automate and observe (ongoing, alongside your program). Your team and your test partner build the automation. Bloomfilter tracks the KPIs per team and flags where automation isn't moving the flow.
Prove and scale (typically quarterly). A before and after readout to leadership. Then extend to more teams, and to AI testing agents, where agent mining shows what the agents actually did.

See the flow, then automate

Automation makes each test cheaper to run. It does not, on its own, make the testing operation faster or the release safer. Seeing the flow is what tells you where to automate next, and whether it worked.

If you want to see where your testing work waits and loops before you automate more of it, talk to us.