Senior Data Engineer

We are seeking a highly skilled Senior Data Engineer to join our Data Engineering team. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and enabling advanced analytics and machine learning use cases. This role requires strong expertise in big data technologies, data processing frameworks, and feature engineering to support ML models in a production environment.Key ResponsibilitiesGather and analyze business and technical requirements for data solutionsPerform Exploratory Data Analysis (EDA) to understand data patterns and qualityDesign, build, and maintain scalable and high-performance data pipelinesIngest structured and unstructured data from multiple sourcesTransform and process large datasets using PySpark and PythonImplement feature engineering techniques for machine learning modelsOptimize Spark jobs for performance and cost efficiencyEnsure data quality, integrity, and security across pipelinesCollaborate with Data Scientists, Analytics Teams, and Delivery LeadsWork in an Agile environment with cross-functional teamsCommunicate effectively with stakeholders and provide technical insightsMaintain version control and CI/CD workflows using GitRequired Skills & ExpertiseStrong experience in Python and PySparkHands-on expertise in Apache Spark (performance tuning & optimization)Solid knowledge of Hadoop ecosystemAdvanced proficiency in SQLExperience in building end-to-end data pipelines and ML pipelinesStrong understanding of data modeling and data warehousing conceptsExperience with feature engineering for ML use casesFamiliarity with Git and version control systemsExposure to cloud platforms (AWS/Azure/GCP) is a plusQualificationsBachelor's or Master's degree in Computer Science, Engineering, or related field8-10+ years of experience in Data Engineering / Big DataPrior experience working in banking or financial services is preferredSoft SkillsStrong problem-solving and analytical thinkingExcellent communication and stakeholder management skillsAbility to work independently and in a team environmentAdaptability to fast-paced Agile workflows#J-18808-Ljbffr