1 research outputs found
Modern Trends in Automating ETL Pipelines in Azure
No full textThe study is aimed at systematizing and analyzing contemporary trends in the automation of ETL pipelines within the Microsoft Azure cloud ecosystem. The objective of the work is to identify key paradigms, toolsets and architectural approaches, as well as to develop a scientifically grounded model for selecting an optimal technology stack depending on the specifics of business tasks. The methodological basis includes a comprehensive analysis of current scientific publications, technical documentation and industry reports, as well as a comparative evaluation of the leading Azure services: Data Factory, Databricks and the newest Microsoft Fabric platform. As a result of the study, the dominant trends have been identified: the shift from classical ETL to ELT, large-scale adoption of serverless architectures, active development of low-code/no-code solutions and the emergence of the Data Lakehouse concept as a universal data repository. Within the framework of the work, a decision matrix for selecting an automation tool is proposed, based on the criteria of transformation complexity and the need for an integrated analytics platform. It is concluded that the evolution of automation tools in Azure is progressing from a set of disparate services toward fully integrated platform solutions, which fundamentally changes the methodology of data lifecycle design and management. The results of the study are of practical value for data architects and engineers, as well as for IT department leaders responsible for developing and implementing data management strategies in a cloud environment
