Design, develop, and maintain data pipelines using Azure Data Factory for efficient data movement and transformation. Azure Synapse Analytics Implementation: Implement and optimize large-scale data warehousing solutions using Azure Synapse Analytics. Data Transformation and Processing: Utilize Azure Databricks for data transformation and processing, ensuring the scalability and efficiency of ETL workflows. Database Design and Management: Design and manage databases on Azure, optimizing for performance, scalability, and data integrity. Integration with Power BI: Integrate data from various sources into Power BI for creating insightful reports and dashboards.
Leverage big data technologies, including Apache Spark, for distributed and scalable data processing. Query Optimization: Optimize SQL queries to enhance database performance and ensure efficient data retrieval. Data Security and Compliance: Implement data security measures and ensure compliance with regulatory standards in Azure data solutions.
Develop and implement data quality checks and validation processes to maintain high-quality data. Scripting and Automation: Use scripting (Python, PowerShell) for automation of data-related tasks, enhancing efficiency and repeatability. Collaboration and Documentation: Collaborate with cross-functional teams, providing technical expertise and insights. Document data engineering processes, solutions, and best practices. Continuous Learning: Stay updated on the latest Azure data services, tools, and best practices. Continuously improve data engineering processes and solutions.
Proficiency in working with Azure data services, including Azure Data Lake Storage, Azure SQL Database, and Azure Blob Storage. Knowledge of Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics (formerly SQL Data Warehouse), and other Azure data tools. Data Modeling and Database Design: Experience in designing and implementing data models. Expertise in creating and optimizing databases for performance and scalability. Data Integration: In-depth knowledge of Azure Data Factory for orchestrating and automating data workflows. Integration of diverse data sources into a unified and coherent data ecosystem. Data Transformation and Processing:Proficiency in data transformation using tools like Azure Databricks. Implementation of data processing pipelines for ETL (Extract, Transform, Load) tasks.
Scope: Azure services: Azure Data Factory (ADF) Azure Databricks Azure Synapse Analytics (formerly Azure SQL Data Warehouse) Azure Blob Storage Azure Data Lake Storage Azure Cosmos DB Azure Event Grid Data processing and analytics: Apache Spark ...
... as a Database Engineer or similar role (minimum 3 years). Strong understanding of data storage solutions, such as Azure SQL Database, Azure Data Lake Storage. Experience with big data technologies, including Azure Data Lake Analytics, Azure Databricks, and ...
... as a Database Engineer or similar role (minimum 3 years). Strong understanding of data storage solutions, such as Azure SQL Database, Azure Data Lake Storage. Experience with big data technologies, including Azure Data Lake Analytics, Azure Databricks, and ...
... in a dedicated Data & Analytics department. At the moment, ... identifying key data problems challenges development Data strategy, based on Data Governance Framework implementing Data Governance Data Management supporting of the data projects acting as a PMO ...
... Data Engineer plays a critical role in data processing, responsible for designing and ... data warehousing and processing such as AWS, GCP or Azure - Ability to design, build and maintain robust and scalable data pipelines - Interacting with other departments ...
... API, GraphQL, DBT, etc. ) and data pipeline orchestration (e.g., Airflow, ... and transforming large volumes of data, data quality and data validation Experience in SQL (SQL ...
... -intermediate), Shell, OOP, Algorithms and Data Structures, Web Development, etc.Data Engineering: Python (upper-intermediate), SQL ( ... advanced), ELT ETL, Query Optimization, Data Architecture, Data Analysis, etc. Кандидат должен обладать ...
от 4000 USD... and competitive market data by working directly with external data miners to obtain data needed to build research. Participate ... Experience as analyst data science preferably in b2b b2c ... market pricing and competitive data develop an understanding of where ...
Вакансия: Data Engineer Мы ищем опытного Data Engineer для присоединения к нашей команде VILED Если вы готовы взять на себя ответственность за поддержание и развитие нашего Data Warehouse (DWH), то эта вакансия ...
... the stakeholders to understand the data-analysis steps, enabling them to ... - this includes local movements in Data Warehousing, Visualization, and Infrastructure platforms ...
... to the establishment of the Data Hub at the Graduate School ... key terms and concepts of Data Science and Machine Learning. Develop ... years of experience working in data analytics or related area and ...
... for a skilled Senior Data Scientist Machine Learning Engineer to join our team, with a focus on finance-related machine learning processes.Our candidate should have experience in data science, machine learning, MLOps and data engineering, enabling them to ...
... др.) проектов и решений BI & Data (PowerBI, Azure Data Platform, Azure AI Services, DWH, MDM, BI, ... 3-х лет (BI, Data, DWH, Master Data Management, DataLake, AI, ML, OpenSource, ... минусов реализации проектов BI & Data в облачной и локальной инфраструктуре; ...
... to present company data. Data analysis: Researching and analyzing data using Power BI to identify trends, patterns, and insights. Data modeling: Building data models in Power BI to ... that arise during data analysis. Creating user documentation: Developing ...
... Вас: Опыт работы на позиции Data Engineer не менее 2 лет; Опыт ...
В команду разработки ищем Data Engineer-а на проекты по развитию ...
... Pandas будут плюсом), опыт построения data pipeline’ов при помощи Airflow. Владение ... - знание технологий DBT, Greenplum, Spark, Data Lake, k8s. Мы предлагаем: Сильная ...
... WCCE for Organized Trade with Data, Shopper Category Customer opportunities & insights ... methodologies and techniques (retail audit data, qual&quant research methodologies etc); ... the use of my personal data and resume by the companys ...
... процессов; Опыт в проектах внедрения Data Quality Знание принципов построения DWH ... работы с инструментами: Informatica Enterprise Data Catalog, Informatica Axon Data Governance, Informatica Data Quality Умение презентовать и выступать ...