Process Observability for Your Business

The promise of digital transformation has been to leverage technology in ways that simplify and streamline our work lives. We love Amazon’s “click to buy” button, which sends nearly any product we can think of to our doorsteps in a matter of hours. In our work lives, we can now collaborate on documents in real time, take video conferencing for granted, look to AI bots to record our action items, and more.

Luckily, as consumers, we rarely have to look underneath the surface, onto the complexity required to make these systems function. Businesses aren’t that lucky. As business processes become more and more digitized, it is more important than ever to understand the impact of operational systems on business outcomes and to align business strategy with IT architecture.

It’s like watching your favorite professional sports team – everyone can be an armchair expert, but fielding a winning team week after week and year after year takes consistent and ongoing effort. Coaches and players need to work together, pulling in data from various sources and applying creative, exploratory analysis to discern the best strategy, mental and physical fitness, and tactics needed to win a particular match. Every business decision is like a new match and the schedule has become more crowded and the stakes higher.

In the same way, businesses need to take a holistic team view that looks at the health of the overall business, identifies key processes and functions, and creates the opportunity for improvement and change. This team view requires understanding end-to-end business processes, the operational systems they touch, and how well they meet business goals as well as the needs of customers and employees. As businesses look to drive complicated business and technology transformation faster, at scale, and with more confidence, they are turning to the emerging practice of process observability.

What Is Process Observability?

At its most basic level, ‘observability’ is the ability to measure the internal states of a system by examining its outputs. It also implies an ability to separate critical information from routine information. A ‘process’ is simply a description of how something gets done. So ‘process observability’ provides a better understanding of the state and performance of how work is getting done within an organization. A process is observable if stakeholders and systems understand how it operates – where the process outcomes are known – and actions can be derived to improve or change these outcomes.

Of course, anything that sounds that simple is usually extremely difficult to execute. Here, the challenge is ensuring that the right observations based on the right data are being collected and that the right people have the right tools to analyze that data in real or near-real time. Identifying patterns, trends, anomalies, and potential issues in this way requires strong alignment of business strategy and IT architecture, as well as the ability to directly connect business processes to the underlying IT landscape they rely on.

Where to Start

Supporting better business outcomes must always be the driving force behind ongoing process analysis, insights, and change. Increasing process observability is not about vanity metrics for executives but the alignment needed to make change easier. This depends on three key pillars – data, models, and benchmarks.

Obviously, a key part of observability is accessing the data that tells you what is happening at a granular level. Every business process produces data in the form of event logs, metrics, and traces. There’s also data from application monitoring and data on the cycle times or costs for processes. Experience data can be gathered from customers, employees, or other stakeholders. In other words, there’s no lack of data sources.

The bad news is that a recent research study with IDC* found that simply having access to multiple sources of insight was not enough by itself. In fact, when organizations leverage multiple data sources but do not integrate them, they encounter significant challenges. The good news is that the organizations that perform best are significantly more likely to integrate different data sources to analyze processes than the organizations performing less well. 

Just like the pre-game stratagems of your favorite team, business process models are more of an idealized version of reality that rarely survives unscathed at first contact with the reality of the opposing team or the reality of the business.  Because of this, it is necessary to bridge the gap between the results of modeling and what is happening in real life. This requires feedback from human experts about underlying issues that cannot be captured by the data alone. Keep in mind that perfect observability is an unattainable goal. It is important to avoid analysis paralysis by understanding the interplay of process models and data, and make conscious, reasonable trade-offs between maximizing process observability and making efficient use of limited time and resources. Don’t let perfect be the enemy of good when you see a clear goal in front of you. You can take one right step at a time.

Benchmarking is a great way to take one such step towards increased overall observability of your processes. Comparing your outcomes to best-in-class performers in similar organizations provides useful insights on its own. In addition, it can help identify which processes can provide the most competitive differentiation for your business. Benchmarks are most useful when they show us more than just our destination – they need to provide content and guidance on the best way to arrive at that destination.

At the End of the Day, It’s about People

For decades, business leaders have been waiting for technology to fully deliver on the promise of seamless and delightful customer experience; ubiquitous, clean, and secure data; robust and complete applications; and a way to easily test and embed new technologies. In other words, they want to know how technology is supporting business outcomes. This is the promise and the challenge of process observability.

As in any team sport, communication – frequent, data-driven, curious – is vital for success. Historically, one of the toughest communication gaps has been linking business strategy with technology architecture, mapping business outcomes with underlying operational systems. Bridging this gap is critical to success. The good news is that we are hearing from our most innovative customers that their teams are finally starting to have the right conversations. Instead of IT reporting to the business about system performance and the like, the conversation is now centered on how processes are performing and what needs improving.

*IDC White Paper, sponsored by SAP, Business Process Observability: A Collaborative Approach to Transformation Enablement, doc #EUR251308223, November 2023.

Dee Houchen is global lead for the Market Impact Team at SAP Signavio.