The Happy Data Platform: A Personified Perspective (Part II)

“True happiness comes only by making others happy.” - David O. McKay

Background and Context

As in the quote above, a data platform can only be truly “happy” if it can make others happy. The others in this context are the actors/teams with whom the data platform interacts, including:

• Data Engineers
• Data Consumers
  • Data Analysts
  • Data Scientists/Machine Learning Engineers
  • External Data Consumers like partners & data buyers
• DataOps Engineers
• Data Stewards & Admins (for Data Governance)

While Part I of this blog series covered the perspectives of Data Engineers and Data Consumers, in this second and final post you’ll find the perspectives of DataOps Engineers and Data Stewards & Admins.

Great (User) Expectations

Data Governance

End-to-end data governance encompasses aspects of Data Quality, Data Security and Data Compliance.

It also includes data cataloging, enabling us to discover the data and data lineage. This furthers our understanding of its origins, process, and journey to its current state.

Data stewards want to make sure that all the above aspects are taken care of by the data platform; that all of the data quality, security, and compliance requirements are converted into policies, controls and processes.

Overarching management processes, policies, and control are needed around aspects including the following:

• Defining the tags and tagging the data.
• Establishing communication and collaboration around the data and the data platform assets.
• Monitoring and managing the usage of these assets.
• Monitoring and managing the effectiveness and efficiency of these assets.
• Defining, monitoring, and managing policies and controls.
• Establishing a continuous improvement process of managing feedback/changes such as regulatory ones.
• Analysis and improvements across all the three dimensions of quality, security, and compliance.
• Implementing workflows to allow search and request for access to existing data assets, as well as requesting the availability of new data.

Enabling enterprise data governance has to be a step-by-step process, and it needs to start with defining MVP based on the specific needs and context. You can keep adding features based on specific priorities, rather than trying to do a “big bang” implementation covering everything all at once.

Data Ops

While DataOps might sound similar to DevOps for data projects, it is much more than that.

DevOps typically covers aspects such as software and services setup, environment provisioning, infrastructure as code, and continuous development, continuous testing, continuous integration, continuous deployment, continuous monitoring, etc.

While DevOps is involved as part of the DataOps, it is only a subset of the whole gamut of aspects that the SuperSet DataOps includes. The high-level objective of DataOps is to make sure that the appropriate data is available to the right consumers, at the right time.

DataOps covers an end-to-end, 360-degree aspect of orchestrating data and converting it into value. It is about enabling an ecosystem of technology, tools, processes, policies, and people to effectively and efficiently ingest, process, store, discover, and consume the data in a secured and compliant manner.

DataOps also involves enabling end users’ trust in the data assets made available to them (which encompasses the aspects covered in Part I of this series).

Accelerate Your Own Data Journey

The objective of a data platform is to eventually enable purposeful, actionable insights that can lead to business outcomes. Additionally, if the data platform puts the right emphasis on the journey and process (i.e., how it can make the job easier for its key actors while delivering the prioritized projects), then it will deliver an ecosystem that is fit for purpose, minimizes waste, and enables a “reuse” mindset.

At GlobalLogic, we are continuously improving our Data Platform Accelerator based on a similar approach. This digital accelerator enables enterprises to immediately manifest a solution that can gather, transform, and enrich data from across their organization. We are excited to work with our clients to accelerate their data journeys and would be happy to discuss your needs with you. Please get in touch using the contact form below.

Related Blogs - The Happy Data Platform - A Personified Perspective (Part I)