In this episode Andrew and Danny talk about modern data warehouses – what they are, how they differ from old-fashioned data warehouses, and why you might want one.
What is a modern data warehouse?
- A modern data warehouse is a database and analytics technology that helps you process big data using all the scale of the cloud.
- We are talking about products such as Azure Synapse Analytics and Snowflake.
How do they differ from older generations of data warehouses?
- Older generations of data warehouse were built on relational database technology, and therefore constrained by the performance of individual machines.
- A next generation introduced MPP – Massively Parallel Processing – which gave scale to data warehouses.
- Modern data warehouses separate compute from storage. Compute is the expensive part, storage is cheap (relatively). You can scale compute up and down as demand requires, whilst benefitting from massive-scale cloud storage.
- Modern cloud data warehouses are as much a system for managing cloud resources as they are a single product.
Why you might want a modern data warehouse?
- Increasingly, we see companies adopting SaaS products for major line of business systems.
- This is great for many businesses – lower operating and support costs, functionality develops over time, can get industry-specific solutions that fit with their businesses.
- Downside is that this creates data silos – isolated pockets of data within the SaaS apps.
- You need to own your own data, don’t rent it. Build a data warehouse and ingest the data from your SaaS apps so that you have a single view of the data your organisation relies on.
- This single view of data allows you to combine and crunch datasets together, which is simply not possible when your data lives in separate SaaS apps.
Watch the video here:
Listen to the audio here:
How we’re trying to shape the culture here at 345 Technology
The first time I tried to explain a “Culture” I was staring at something growing in a petri dish in a biology class. Now that
How reshaping data leads to better reports
If you’ve ever used a tool such as Power BI… …and struggled to extract meaningful reports from your databases, you’ll know the frustration of having
Episode 32: Review of 2020 – the Good, the Bad and the Ugly
What a year it’s been! Thinking back to January, we were dealing with the impact of the impending IR35 rule change. As with all software
Data platform engineering to help Novatia support more schools
Novatia is an ICT consultancy that has specialised in the UK education space since being founded in 2003. Its Questa product is a data aggregation
Episode 31: Automated Monitoring and Control Technologies
In this episode Andrew and Danny look into why measuring the temperature in supermarket fridges is really important, and it turns out to be amazingly
Redesigning a UK banking institution’s payments platform
A UK household name and one of the world’s largest mutual financial institutions, our client has over 14 million members and a history that stretches back