There’s no one size fits all approach to sharing your data

Organizations now have a much more convenient way to store data

News By Bart Koek published 15 November 2024

It’s almost become cliché to start articles about data with an acknowledgement of the sheer volume of data that modern economies create and, with the increasing prominence of AI, the demand for all of this data is simply going to continue to increase.

Thanks to the rise in business cloud storage platforms, organizations also now have a much more convenient way to store these huge quantities of data, but that has come at a cost: this data is decentralized like never before. That means, for a modern data consumer, it’s not just tough to find and access the data they’re looking for; they can’t even be sure it exists at all.

Collecting, cataloguing and governing the large amounts of data stored by an organization is now paramount to getting the most out of it. In an effort to accelerate this move towards ensuring data is both discoverable and usable, though, businesses are increasingly adopting a one size fits all approach via a data catalog alone. Whilst this approach is being praised as a silver bullet for businesses, could it actually be holding them back?

Bart Koek

Field CTO, Immuta.

Uniting your data

To overcome the challenges associated with large data lakes, which are increasingly becoming more like data oceans, many data professionals have adopted data catalogs to help bring some order to their large quantities of data. But what exactly are data catalogs, and why might they not be the sole solution to the challenge of uniting an organization's data?

Data catalogs provide the frameworks and interfaces to manage and collect metadata, or information on the lineage, reliability, quality and sensitivity of data and data products. Put more simply, they allow users to see all of the information they need about data stored by an organization, which is an incredibly powerful tool for, say, the data engineers who are building their organization's data products – think of the data catalog like a large inventory list at a builders yard, allowing data product builders to find all of the right materials for their next project at a glance.

Data catalogs are, therefore, incredibly important to an organization serious about getting the most from its data. They are not, though, a one size fits all solution to the challenge of making data discoverable, governable and accessible.

The data marketplace

As data consumers become an increasingly important part of a modern data strategy, businesses need to look beyond a catalog-only approach to their data tools, and begin considering an internal data marketplace. But what exactly is a data marketplace, and what can it offer data consumers that they can’t achieve with a data catalog alone?

Are you a pro? Subscribe to our newsletter

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

Contact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsors