U:: [[Data Engineering]] ## Big data is no longer the concern; observability and metadata is - Compute over big data has been less and less relevant of a term because most enterprises have data that can be stored on a giant data lake but processed in isolation on machines or smaller compute units. With the advent of [[Cloud Data Warehouses]] like [[Snowflake]], [[BigQuery]] (I'm not including [[Redshift]] because there is no separation of compute and storage), compute over large datasets becomes trivial for everyone except some data unicorns. - Instead it's more important to figure out how best to observe and add metadata to make them more useful. ## Who needs big data? ![[Big Data Is Dead#^f2406b]] ## References - [[Big Data Is Dead]]