{"id":201,"date":"2025-08-21T07:17:22","date_gmt":"2025-08-21T07:17:22","guid":{"rendered":"https:\/\/themes.dynamiclayers.net\/redias\/project\/global-data-economy-copy\/"},"modified":"2025-10-30T16:40:14","modified_gmt":"2025-10-30T16:40:14","slug":"pipelines-elt-evolutifs","status":"publish","type":"dl-project","link":"https:\/\/viasservices.com\/en\/project\/pipelines-elt-evolutifs\/","title":{"rendered":"Scalable ELT pipelines"},"content":{"rendered":"<div data-elementor-type=\"wp-post\" data-elementor-id=\"201\" class=\"elementor elementor-201\" data-elementor-post-type=\"dl-project\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d51efeb elementor-section-full_width elementor-section-height-default elementor-section-height-default\" data-id=\"d51efeb\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-no\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-9f76eda\" data-id=\"9f76eda\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4432db3 elementor-widget elementor-widget-text-editor\" data-id=\"4432db3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>This project involved designing and deploying robust, scalable ELT pipelines for a leading German agency. The goal was to centralise and automate data ingestion, transformation and analysis via Snowflake, ensuring speed and reliability in processing massive volumes.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-551e0dd elementor-section-full_width elementor-section-height-default elementor-section-height-default\" data-id=\"551e0dd\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-no\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-b9bbcb2\" data-id=\"b9bbcb2\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-d939178 dl-align-left elementor-widget elementor-widget-redias-section-heading\" data-id=\"d939178\" data-element_type=\"widget\" data-widget_type=\"redias-section-heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t        <div class=\"dl-section-heading\">\n                                    <h2 class=\"dl-heading\" data-bg-text=\"\">\n                Project challenges            <\/h2>\n                                        <p class=\"dl-desc\">Ensure smooth and reliable data flow.<\/p>\n                    <\/div>\n        \t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e6daad3 elementor-icon-list--layout-traditional elementor-list-item-link-full_width elementor-widget elementor-widget-icon-list\" data-id=\"e6daad3\" data-element_type=\"widget\" data-widget_type=\"icon-list.default\">\n\t\t\t\t\t\t\t<ul class=\"elementor-icon-list-items\">\n\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Integrate multiple sources in various formats.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Manage the growing volume of data sets.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Maintain information consistency after transformation.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Ensure the security and traceability of flows.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t<\/ul>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-b3cc9e5\" data-id=\"b3cc9e5\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0e3cfce dl-align-left elementor-widget elementor-widget-redias-section-heading\" data-id=\"0e3cfce\" data-element_type=\"widget\" data-widget_type=\"redias-section-heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t        <div class=\"dl-section-heading\">\n                                    <h2 class=\"dl-heading\" data-bg-text=\"\">\n                Solutions implemented            <\/h2>\n                                        <p class=\"dl-desc\">Implementation of an optimised and scalable Snowflake architecture.<\/p>\n                    <\/div>\n        \t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c797dcf elementor-icon-list--layout-traditional elementor-list-item-link-full_width elementor-widget elementor-widget-icon-list\" data-id=\"c797dcf\" data-element_type=\"widget\" data-widget_type=\"icon-list.default\">\n\t\t\t\t\t\t\t<ul class=\"elementor-icon-list-items\">\n\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Complete automation of ELT ingestion processes.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Optimised SQL queries for fast processing.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Caching and intelligent data partitioning.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"la la-hand-point-right\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Continuous monitoring to ensure performance.<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t<\/ul>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-9e901ec\" data-id=\"9e901ec\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Ce projet a consist\u00e9 \u00e0 concevoir et d\u00e9ployer des pipelines ELT robustes et \u00e9volutifs pour une agence allemande de premier plan. L\u2019objectif \u00e9tait de centraliser et d\u2019automatiser l\u2019ingestion, la transformation et l\u2019analyse des donn\u00e9es via Snowflake, garantissant rapidit\u00e9 et fiabilit\u00e9 dans le traitement de volumes massifs. D\u00e9fis du projet Assurer la fluidit\u00e9 et la fiabilit\u00e9 [&hellip;]<\/p>","protected":false},"featured_media":1893,"comment_status":"open","ping_status":"closed","template":"","project-category":[34],"class_list":["post-201","dl-project","type-dl-project","status-publish","has-post-thumbnail","hentry","project-category-ingenierie-de-donnees"],"acf":{"project_info_heading":"Informations sur le projet","project_info_desc":"Optimisation et automatisation des pipelines ELT Snowflake.","project_info":[{"type":"text","label":"Client","value":"Agence allemande","pick_date":null,"rating":5},{"type":"date","label":"Date d'ex\u00e9cution","value":"","pick_date":"20240304","rating":5},{"type":"text","label":"Localisation","value":"Berlin, Allemagne","pick_date":"20230810","rating":5}],"based_on":"category","related_projects":[141,197,205,195],"dl_description":"","dl_pinterest_image":"","dl_hashtags":""},"_links":{"self":[{"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/201"}],"collection":[{"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project"}],"about":[{"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/types\/dl-project"}],"replies":[{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/comments?post=201"}],"version-history":[{"count":9,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/201\/revisions"}],"predecessor-version":[{"id":2078,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/201\/revisions\/2078"}],"acf:post":[{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/195"},{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/205"},{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/197"},{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/dl-project\/141"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/media\/1893"}],"wp:attachment":[{"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/media?parent=201"}],"wp:term":[{"taxonomy":"project-category","embeddable":true,"href":"https:\/\/viasservices.com\/en\/wp-json\/wp\/v2\/project-category?post=201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}