Data Engineer - Consultant PySpark ADF SQL 6 months contract 2 to 4 years experience required 1[...]

<div><h3>Job Overview</h3><p>Its a 6 months contract role extendable further based on client discretion.</p><p>Minimum 2 years of experience is required.</p><p>Budget 10k to 12k AED. Visa, Medical Insurance, Work Permit.</p><h3>Responsibilities</h3><ul><li>Design, develop and maintain data pipelines for ingestion, transformation and loading of data into the data warehouse.</li><li>Design, develop and maintain data pipelines using<b>PySpark</b>and<b>Azure Data Factory (ADF)</b>.</li><li>Implement data governance frameworks and ensure data quality, security and compliance with industry standards and regulations.</li><li>Develop complex SQL queries and manage relational databases to ensure data accuracy and performance.</li><li>Establish and maintain data lineage tracking within the data fabric to ensure transparency and traceability of data flows.</li><li>Implement ETL processes to ensure the integrity and quality of data.</li><li>Optimize data pipelines for performance, scalability and reliability.</li><li>Develop data transformation processes and algorithms to standardize, cleanse and enrich data for analysis, applying data quality checks and validation rules to ensure the accuracy and reliability of data.</li><li>Mentor junior team members, review code and drive best practices in data engineering methodologies.</li><li>Collaborate with cross‑functional teams including data scientists, business analysts and software engineers to understand data requirements and deliver solutions that meet business objectives; work closely with stakeholders to prioritize and execute data initiatives.</li><li>Maintain comprehensive documentation of data infrastructure designs, ETL processes and data lineage; ensure compliance with data governance policies, security standards and regulatory requirements.</li></ul><h3>Qualifications</h3><ul><li>Strong proficiency in<b>SQL</b>and at least one programming language (e.g. Python) for data manipulation and scripting.</li><li>Strong experience with<b>PySpark</b>,<b>ADF</b>,<b>Databricks</b>and<b>SQL</b>.</li><li>Preferable experience with MS Fabric.</li><li>Proficiency in data warehousing concepts and methodologies.</li><li>Strong knowledge of Azure Synapse and Azure Databricks.</li><li>Hands‑on experience with data warehouse platforms (e.g. Snowflake, Redshift, BigQuery) and ETL tools (e.g. Informatica, Talend, Apache Spark).</li><li>Deep understanding of data modeling principles, data integration techniques and data governance best practices.</li><li>Preferable experience with Power BI or other data visualization tools to develop dashboards and reports.</li></ul><h3>Additional Details</h3><p><b>Remote Work:</b>Yes</p><p><b>Employment Type:</b>Fulltime</p></div>#J-18808-Ljbffr