AWS Glue Data Engineer

OverviewDeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking.ResponsibilitiesBuild scalable ingestion pipelines using AWS Glue, PySpark, and ETL pipeline development.Implement AWS Glue jobs for Bronze layer ingestion, applying defined standards and templates.Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.Optimize Glue job performance (DPU allocation, parallelization, partitioning) according to best practices.Collaborate with platform teams to ensure tooling and optimization alignment.Automate migration of source tables to Bronze layer, leveraging AI-enabled acceleration.Version-control jobs and automate production deployment via Git and Terraform.Establish source system connectivity into CDP in collaboration with source system owners.Ensure jobs comply with data contracts and monitoring standards.Prepare documentation and handover to operational support teams.Work closely with Data Architect for ingestion patterns and standards.Coordinate with Data Assurance Lead to apply quality checks across all jobs.Partner with platform engineers for tooling and optimisation.Qualifications5+ years’ experience in data engineering roles, with hands‑on experience in AWS Glue.Proficiency in AWS Glue, PySpark, and ETL pipeline development.Substantial knowledge of Lakehouse architecture and Medallion design principles.Familiarity with CDC, delta loads, and historical data ingestion strategies.Experience with AWS services: Glue, S3, Athena, Lambda.Competence in Git and Terraform for CI/CD automation.Knowledge of data quality frameworks (e.g., Soda Core).Strong communication skills and ability to engage with various stakeholder levels.Experience working in fast‑paced environments and delivering aggressive migration targets.Familiarity with Jira and agile methodologies.Benefits & Growth OpportunitiesCompetitive salary and performance bonusesComprehensive health insuranceProfessional development and certification supportOpportunity to work on cutting‑edge AI projectsFlexible working arrangementsCareer advancement opportunities in a rapidly growing AI companyWe are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions. All candidates are encouraged to request reasonable adjustments to the application process, should the need arise.#J-18808-Ljbffr