Design, develop, and maintain data pipelines using Azure Data Factory for efficient data movement and transformation. Azure Synapse Analytics Implementation: Implement and optimize large-scale data warehousing solutions using Azure Synapse Analytics. Data Transformation and Processing: Utilize Azure Databricks for data transformation and processing, ensuring the scalability and efficiency of ETL workflows. Database Design and Management: Design and manage databases on Azure, optimizing for performance, scalability, and data integrity. Integration with Power BI: Integrate data from various sources into Power BI for creating insightful reports and dashboards.
Leverage big data technologies, including Apache Spark, for distributed and scalable data processing. Query Optimization: Optimize SQL queries to enhance database performance and ensure efficient data retrieval. Data Security and Compliance: Implement data security measures and ensure compliance with regulatory standards in Azure data solutions.
Develop and implement data quality checks and validation processes to maintain high-quality data. Scripting and Automation: Use scripting (Python, PowerShell) for automation of data-related tasks, enhancing efficiency and repeatability. Collaboration and Documentation: Collaborate with cross-functional teams, providing technical expertise and insights. Document data engineering processes, solutions, and best practices. Continuous Learning: Stay updated on the latest Azure data services, tools, and best practices. Continuously improve data engineering processes and solutions.
Proficiency in working with Azure data services, including Azure Data Lake Storage, Azure SQL Database, and Azure Blob Storage. Knowledge of Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics (formerly SQL Data Warehouse), and other Azure data tools. Data Modeling and Database Design: Experience in designing and implementing data models. Expertise in creating and optimizing databases for performance and scalability. Data Integration: In-depth knowledge of Azure Data Factory for orchestrating and automating data workflows. Integration of diverse data sources into a unified and coherent data ecosystem. Data Transformation and Processing:Proficiency in data transformation using tools like Azure Databricks. Implementation of data processing pipelines for ETL (Extract, Transform, Load) tasks.
Scope: Azure services: Azure Data Factory (ADF) Azure Databricks Azure Synapse Analytics (formerly Azure SQL Data Warehouse) Azure Blob Storage Azure Data Lake Storage Azure Cosmos DB Azure Event Grid Data processing and analytics: Apache Spark ...
... ; - 2 years of experience creating data models, transformation logic, data quality rules, and test cases; - ...
от 4000 до 6000 USD... to the establishment of the Data Hub at the Graduate School ... key terms and concepts of Data Science and Machine Learning. Develop ... years of experience working in data analytics or related area and ...
... for a skilled Senior Data Scientist Machine Learning Engineer to join our team, with a focus on finance-related machine learning processes.Our candidate should have experience in data science, machine learning, MLOps and data engineering, enabling them to ...
... be the full-time Industrial Data Coordinator in Astana we’re looking ... coordinate a team of 2 data engineers and ensure all the data in Information systems is up- ...
... be the full-time Industrial Data Coordinator in Astana we’re looking ... coordinate a team of 2 data engineers and ensure all the data in Information systems is up- ...
... experienced AI ML Automation Engineer to join our team. The ... fill in all the data (name, description, attributes, cost). Currently, ... . Experience of building data processing automation on top of ... to connect external data sources for model training. Strong ...
... with all teams in the Data Department to optimize data flows, control data quality and integrity, and create data marts for the product. Qualifications:— ...
... equipment by reviewing TCMS data Ability to propose improvement of ... correct completion of the data. Execution phase: Monitoring of the ... support to technical documentation engineer to edit and modify the ... and reflecting these data in the project monthly and ...
... countries?Join our community of Data professionals and become a part of Data Products pipelines migration from Oracle ... Do Develop, monitor, and operate data pipeline - Sales Order Data (with Post-order information: shipment, ...
... and active Software Architect (.Net Azure) to become part of our ... architectures leveraging .NET Core and Azure technologies.- Defining and detailing solution ... experience of cloud platforms (Microsoft Azure is mandatory, AWS and Google ...
... cars. POSITION OVERVIEW: Design Engineer is an Engineer responsible for carrying out the ... processes using tools and data available; Providing detailed specifications analysis ... ; Recording and evaluating testing data, altering designs as necessary to ...
... : Karate + Java, GIT, AWS environment, Azure DevOps Day to day duties ... Karate framework Pipeline configuration in Azure DevOps Test Reporting in Azure DevOps Monitoring existing and newly ...
от 5000 до 5800 USD... implementing cloud-based solutions on Azure, including storage, networking, security, and ... to date with the latest Azure features and technologies, and incorporating ... operating systems running in Microsoft Azure Strong oral and written communication ...
... -time Project Configuration and Change Engineer in EKZ, Astana we’re looking ... and completeness of configuration items data all along project life cycle; ... . for sub-system target configuration data and ensures consistency. Ensure the ...
... accountabilities: Collect, consolidate and verify data for internal and external purposes ... years of experience in sustainability data audit and or reporting Strong ... to analyse and systematise large data sets Solid knowledge of sustainability ...
... сейчас в поиске Data Scientist. Обязанности: Работа с сырыми обработанными телеком. данными, веб-траффиком; Exploratory Data Analysis; Профилированием пользователей; Кластеризацией динамических ... Знание и понимание технологии Data Science, в том числе ML ...
... телеком. данными, веб-траффиком; Exploratory Data Analysis; Профилированием пользователей; Кластеризацией динамических ... знать? Знание и понимание технологии Data Science, в том числе ML ...
Henkel is seeking an Application engineer for ACM (Adhesives Business Unit), ... , and usage of your personal data.