Enabling data-driven decisions through data products
Nathi Dube, director, PBT Innovation at PBT Group.
There is no doubt that customer insights derived from high-quality data helps businesses to make more accurate decisions, which in turn gives them a competitive advantage.
As more businesses seek to understand their customers better, the way they use available data to achieve this becomes a determining factor as to whether they ‘win’ or ‘lose’ at sustaining customer satisfaction and in the long-term, customer loyalty.
The ever-changing customer needs can best be met through effective ongoing analysis of customer data to create more personalised experiences. Customer data analytics helps drive customer-led innovation that puts the client at the centre of a business strategy.
This is commonly referred to as hyper-personalisation, which is rapidly becoming the new standard for client engagement/experience that ultimately supports sustaining customer satisfaction and loyalty.
To effectively implement such a strategy, there is a need to re-organise data teams, revise the underlying architectures and redefine the way businesses relate to data.
Waterfall data projects with fixed timelines and deliverables are no longer serving business needs well. The idea that you can build a data solution today that will still serve the needs of the business in five years simply does not work.
The idea that you can build a data solution today that will still serve the needs of the business in five years simply does not work.
Today, there is a paradigm shift that introduces product thinking to data, resulting in domain-specific data products that evolve with changing business needs. In this piece, I explore a few key aspects related to enabling data products for businesses to make more data-driven and informed decision-making.
There are a few aspects that need to be considered and put into place in order to build data products. These include:
Create a data culture: Ensure there is a deliberate and concerted effort to treat data as part and parcel of the project build process and not its by-product. When building new system features, data owners should constantly ask themselves: what insights do we want to gain from this feature; is it possible to generate all data elements required for insights; do we have the tools to process and analyse data; is the data delivered on time?
Adopting DataOps: DataOps is a data management approach that borrows from DevOps − it combines development and operations together to shorten time to value. It allows for quicker response to user demands, while adhering to data governance principles. DataOps is well-suited for building data products because of its principle of continuous development and collaboration with stakeholders.
Domain team composition: This must be such that the process to build and deliver data products is efficient and frictionless. Domain data teams must be self-contained, comprising cross-functional resources with complementary skills to deliver data solutions. Teams must have autonomy of how they manage and use data that falls under their domain.
Ownership: In a data mesh, a single team with expertise in a particular domain creates and manages data products. This team carries the responsibility of delivering the required levels of service and data quality. The domain’s product owner is the dedicated contact person within the domain and is also responsible for all aspects of the data product. Product owners are responsible for the lifecycle and continuous improvement or changes of data products, considering domain-specific analytical requirements and the needs of data consumers where applicable.
Data platform team: These teams must create data architecture patterns that satisfy business requirements – whether there is a need for real/near time data analysis, or data is accumulated and then processed in batches. They must provide an enabling environment to innovate and build data products that speak to the specific domain requirements.
Data architectures that support data products
Data mesh is often contrasted with data fabric. While both architecture styles seek to democratise data access and usage, data fabric is centralised as it focuses on automating the process to bring data from various sources in one place for analysis.
On the other hand, a data mesh is a domain-oriented data architecture that distributes data ownership among domain owners, granting each domain greater control over the ingestion and processing of data within the data platform.
The decentralised nature of the data mesh architecture introduces a new paradigm in data management, as it shifts the focus away from IT teams to the business domains who own the data. Domain owners are responsible for creating their own data products, which are either consumed within the domain or provided as input to other domains.
A data mesh architecture supports and promotes a culture of experimentation and continuous improvement within the limits of data governance and security controls.
As business domains have intimate knowledge of their data, they can analyse data more efficiently to derive valuable insights from it, which they can then share with the wider audience within the organisation. They can give an authoritative voice to the data being shared as they know it better.
The iterative nature of data product development ensures refinements are constantly being made, which further improves the quality of insights derived over time.
When a business decision needs to be made on a particular subject, there would typically be a specific data product that can be used to help make an informed decision and that data product would have been vetted by the data owners and signed off before it can be used.
Domain owners can share their data products with other domains, which introduces efficiencies in the overall business process, as other teams do not have to re-invent the wheel and they can trust the data they consume as it comes from the authoritative source.
Business domains can use data to make timely and well-informed decisions that impact their domains when they have control over their data ingestion and processing. An organisation is truly data-driven if every business domain inside the organisation uses data to make decisions.
