Job Description & Requirements

The Senior Data Engineer plays a criticalrole in designing, developing, and maintaining scalable data infrastructure and pipelines. This position supports data-driven decision-making across the organization by ensuring data quality, accessibility, and performance.

Roles and Responsibilities

Data Pipeline Development: Design, build, and maintain robust, scalable data pipelines using tools like Apache Airflow, Kafka, and Spark.
Data Architecture: Develop and optimizedata models, data warehouses, and ETL processes to support analytics and machine learning initiatives.
Collaboration: Work closelywith data scientists, analysts, and software engineers to understand data requirements and deliver tailored solutions.
Data ǪualityG Governance: Ensure data integrity, implement validation checks, and enforce data governance policies.
Performance Optimization: Monitor and improvedata processing speed,system uptime, and query performance.
Leadership G Mentorship: Guide junior engineers, promote best practices, and foster a culture of continuous learning.
Innovation G Research: Stay updated with emerging technologies in AI, automation, and data streaming; lead initiatives to integrate them into existing systems.

Profile

Bachelor's or Master's in computer science, data science, information technology, or a related field, with 6+ years of experience.
Proficiency in SǪL, Python, and ETL/ELT tools
Experience in Microsoft solutions are as follows:
Azure Data Factory
Microsoft Fabric
Azure Synapse Analytics
Azure Databricks
Power BI
Azure SǪL Database/ SǪL Server
Azure Purview
Experience with big data technologies (Hadoop, Spark)
Familiarity with cloud platforms(Azure, AWS, and GCP)
Strong understanding of data modelling, data warehousing, and stream processing
Knowledge of data security, compliance, and governance
Knowledge of CI/CD pipelines using Azure DevOps
Understanding of data security and compliance(e.g., GDPR, HIPAA)

Data Engineer - Microsoft Fabric

카테고리

기술

직무 설명