In this episode Andrew and Danny talk about modern data warehouses – what they are, how they differ from old-fashioned data warehouses, and why you might want one.
What is a modern data warehouse?
- A modern data warehouse is a database and analytics technology that helps you process big data using all the scale of the cloud.
- We are talking about products such as Azure Synapse Analytics and Snowflake.
How do they differ from older generations of data warehouses?
- Older generations of data warehouse were built on relational database technology, and therefore constrained by the performance of individual machines.
- A next generation introduced MPP – Massively Parallel Processing – which gave scale to data warehouses.
- Modern data warehouses separate compute from storage. Compute is the expensive part, storage is cheap (relatively). You can scale compute up and down as demand requires, whilst benefitting from massive-scale cloud storage.
- Modern cloud data warehouses are as much a system for managing cloud resources as they are a single product.
Why you might want a modern data warehouse?
- Increasingly, we see companies adopting SaaS products for major line of business systems.
- This is great for many businesses – lower operating and support costs, functionality develops over time, can get industry-specific solutions that fit with their businesses.
- Downside is that this creates data silos – isolated pockets of data within the SaaS apps.
- You need to own your own data, don’t rent it. Build a data warehouse and ingest the data from your SaaS apps so that you have a single view of the data your organisation relies on.
- This single view of data allows you to combine and crunch datasets together, which is simply not possible when your data lives in separate SaaS apps.
Watch the video here:
Listen to the audio here:
Episode 30: Why IoT Might Be Great for Your Business
In this episode we take a first look at IoT and cover off some of the common scenarios where IoT is a great solution. IoT
IoT: why collecting data isn’t enough
The Internet of Things (IoT) has been given a bad name by those who think that the technology is limited to expensive white goods in
Episode 29: Why You Need a Modern Data Warehouse
In this episode Andrew and Danny talk about modern data warehouses – what they are, how they differ from old-fashioned data warehouses, and why you
Episode 16: Data Science South Coast – Introduction to Data Lakes
In this episode of the podcast we are again at the Data Science South Coast meetup. This time I’m presenting an Introduction to Data Lakes.
Episode 14: How to Manage a Data Lake
In this episode Andrew and Paul discuss the finer points of managing a data lake. Things you need to look out for in data lakes:
Episode 13: What is a Data Lake?
In this episode Andrew and Paul discuss what a data lake is, what you put in it and why you want one. Check out the
