BigDataLDN returned to London with one of the most talked about data trends in key focus. The usual companies were front and center, with talks around data engineering, the modern stack, analytics, fast data and AI & MLOps all expected given past years.
One ‘theatre’ was brand new to BigDataLDN for 2022. Its inspiration came from a paper published back in 2019, highlighting 4 principles to help companies move past a “monolithic data lake” and progress to a distributed architecture where data is finally treated like a product. This idea, Data Mesh, has seemingly become both the buzzword and strategic direction of much of the data community.
Data Mesh, and its conceptual founder Zhamak Dehghani, brought some interesting and thought-provoking conversation during a panel also including individuals from the BBC, Thoughtspot, Starburst, Roche and Eckerson.
Zhamak Dehghani, Data Mesh’s conceptual founder, alongside Jules Marshall (BBC), Cindi Howson (Thoughtspot), Justin Borgman (Starburst), Omar Khawaja (Roche) and Kevin Petrie (Eckerson), dived into the heart of what they think you should know - and where they believe the missteps on the road to Data Mesh progress will be. While smaller elements were around a need to keep business value, starting small and scaling fast, and engineering availability and connectivity issues, the core topic touched on issues from recent data breaches.
If you are a data leader, product owner or engineer, thanks to regulations like GDPR, CCPA and the US federal privacy regulation, access controls will need to be centralized and high on every priority list. Fine-grained data access controls will have to function smoothly and flexibly to accommodate scaling data teams,
Jules Marshall, the BBC’s Director of Product Data, echoed that data access management and requirements for linking controls to governance and policy would be key for success. This means creating actionable data mesh security while keeping openness to data and open trust ensuring that people only have access to the data they need to have access to.
A solid framework and execution for data access control will undoubtedly lead to companies moving faster, dealing with key use cases like enterprise-scale data sharing that can quickly multiply business value.
So, jumping on the data mesh train may not be as easy as it seems. If you can take one thing from the experts, it is that good frameworks and ways of working will be critical, but who will build these well for scale? In the end, many will claim to have reached Data Mesh…but few will initially manage to do it well.
How mature is data access management? And how fast is it moving? Give your anonymous answers to our 8-question survey here: https://forms.gle/j6JwSy6JYAmkFjXBA
