Migration is hard stuff to accomplish, especially big amount of volume. I am seeing articles in LinkedIn which migrate X to BigQuery or migrate X to Snowflake. I’ve not used Snowflake and I don’t think I will use it near future but I can talk about BigQuery migration… And more specifically, on-premise database to BigQuery migration. You can also use these methods on managed databases in every cloud provider.
Currently, I am working for Zattoo and I am tech lead in data team — means BI + Data Engineering colleagues. As the responsibility of my role, I am driving Zattoo’s data future and deciding architectures and methods to be alive at least the next 5 years. As we are growing explosively, our data is growing so too. I have already replaced our legacy data pipelines and it is time to change the wrong processes about data.
As I am seeing a lot of architectural and process discussions in LinkedIn, sometimes it is really weird to see how people make things so complicated. I am a big fan of KISS method: Keep-it-simple-stupid! So complication is not a design for me and never will be — It is not about me, none of us like complicated things.
