What are our data predictions for 2023? Organizations have long since discovered the potential of data as a competitive asset, but are still missing the frameworks to extract it efficiently and safely. We believe that companies will continue to invest in data and analytics, albeit in a more governed and cost efficient way.
So, here are our combined big 5 predictions for 2023.
Back in October, when we were at the Coalesce event in London, we noticed a surprisingly high number of data governance related questions. It was an interesting experience. Data Governance was long considered outdated, if not dead. Something only big corporations do, and only because they have to. Yet, the need for data governance was unequivocally felt during the event. The Modern Data Stack has truly democratized access to data and analytics which leads to an explosion in data teams across organisations of all sizes in all industries. They are now realising that you need data governance to truly get value from your data products. With the economic crisis and an increased focus on cost management, it will be even more important to optimize the value out of your data products.
2023 will also see a lot of regulatory change that will continue to shape data governance across all sectors and industries. Regulations such as the CPRA and DORA, and supervisory audits of any kind will require more transparency in and control over how data is stored and processed.
In the same vein, organizations are increasingly becoming aware of the reputational damage of privacy and security breaches and what it can mean for a company’s bottom line. The recent breaches at TikTok, LastPass, and Optus show organisations have to better govern their data if they want to prevent the business impact of privacy and security concerns.
In 2022 we saw the concept of data contracts taking shape in an attempt to bring much needed maturity to the data landscape. Conceptually close to API integrations, data contracts are one of the ways data teams are applying software engineering best practices to data. We saw several successful implementations in the second half of 2022, and we expect data contracts to really take off in 2023. Next to offering a technical standard, data contracts will describe the data products and increase the reliability, and (re-)usability of data, creating a trusted relationship between data consumers and data producers.
We expect companies to continue to treat data as a competitive asset. Companies will continue to look for ways to get the most out of existing and future data, and will rely on Self Service Analytics for this. For this to work, Data will have to be treated as a Product, which is one of the tenants of the Data Mesh framework. This also means that we’ll see data teams increasingly borrow best practices from DevOps in the relatively new framework of DataOps, such as the aforementioned data contracts. Combined with the first point, this means that Data Governance will have to be pushed left and be automated as much as possible.
In the past, most companies within traditional industries were quite hesitant to adopt open source technologies. Nowadays, we notice that larger companies are increasingly adopting and even contributing to open source technology. This implies a significant shift in the data landscape, from proprietary vendor tooling to community driven technologies across all components of the data stack. Driven by widespread adoption, open source libraries such as Airflow and DBT have become the de facto standards in the data stack. Note, however, that due to the large amount of open source initiatives, we don’t expect a consolidation to happen anytime soon. Organisations will still have to work with a wide-spread collection of tools which makes data governance, and specifically data access management, challenging.
From the many conversations we’ve had it is clear that the move to the cloud is not losing steam. Companies make these decisions on a 10-15 year horizon, and the current slowdown hardly affects these decisions. With the US defense department and the London Stock Exchange, we also had some high profile examples of actors in very conservative sectors migrating to the cloud, showing that the cloud is mature enough to host these extremely demanding organizations. We also expect that the ongoing efforts to implement a new Privacy shield will bear fruit in 2023, removing the regulatory uncertainty in Europe after a long hiatus following Schrems II.
So there you have it. Our 5 thoughts leading into 2023.
Image source: https://twitter.com/emily_murnane/status/1604512050180300800
Photo by Moritz Knöringer on Unsplash
