Azure Data Engineer

We are looking for a highly skilled Azure Data Engineer for an engagement to build scalable, production-ready data solutions.The ideal candidate will have strong hands‑on experience with Python, PySpark, Azure Databricks, and Azure Data Factory, along with a solid understanding of modern data engineering best practices.Key ResponsibilitiesDesign, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, and Spark SQLBuild and optimize workloads on Azure Databricks (notebooks, job clusters, workflows)Implement Delta Lake best practices (schema evolution, partitioning, performance tuning, merge strategies)Develop robust ingestion frameworks integrating with Azure Data Lake Storage Gen2 and Azure SQLImplement data validation, transformation logic, and quality control frameworksEnsure performance optimization and cost‑efficient executionSupport production deployments, monitor pipelines, and resolve incidentsMaintain version control using Git and follow CI/CD best practicesCollaborate with architects, analysts, and business stakeholdersMandatory SkillsStrong hands‑on experience in PySpark and Spark SQLProven experience in Spark performance tuning & optimizationStrong knowledge of Delta Lake architectureExperience working with Azure Data Lake Storage Gen2Git‑based version control experienceProduction support and troubleshooting experienceGood to HaveExposure to data governance, data security, and compliance frameworksExperience integrating data from SAP ERP systemsKnowledge of Azure Data Factory (ADF)Understanding of DevOps and CI/CD pipelines#J-18808-Ljbffr