Data Engineer

Location: Offshore- Chennai/BengaluruExperience: 4+ YearsEducation: Bachelor's/Postgraduate degree in Computer Science or related field (or equivalent industry experience)Role OverviewWe are looking for a skilled Data Engineer to design, build, and maintain scalable, real-time data processing systems. The ideal candidate will have strong experience in event-driven architectures, streaming technologies, and distributed systems, along with a solid understanding of performance optimization, security, and data governance.You will play a key role in developing robust data pipelines and ensuring high availability, reliability, and compliance across enterprise data platforms.Key ResponsibilitiesData Engineering & ArchitectureDesign and develop real-time data pipelines using Kafka and streaming frameworks such as Flink, Spark Streaming, or Beam.Build and maintain event-driven architectures for scalable and high-throughput data processing.Implement distributed data systems ensuring performance, resilience, and fault tolerance.Work with SQL and NoSQL databases to manage structured and unstructured data efficiently.Performance & OptimizationPerform JVM tuning and performance optimization for high-volume data processing applications.Monitor and improve system performance, ensuring low latency and high throughput.DevOps & InfrastructureContainerize applications using Docker and orchestrate using Kubernetes.Implement and maintain CI/CD pipelines using tools like Jenkins and GitHub.Work in Linux environments, leveraging shell scripting for automation and operational efficiency.Security & ComplianceDesign and implement secure data processing systems, ensuring compliance with enterprise and regulatory standards.Identify security gaps in system design and recommend improvements.Implement data protection mechanisms including encryption, anonymization, and secure data transfer processes.Ensure compliance with data governance policies, including handling of sensitive data and logging standards.Monitoring & ReliabilityImplement monitoring and alerting for distributed systems and streaming platforms.Ensure system resiliency and fault tolerance in production environments.Troubleshoot and resolve issues in distributed services ecosystems.Work in an Agile environment, collaborating with cross-functional teams.Follow best practices in coding, testing, documentation, and security.Research and benchmark new technologies to enhance system capabilities.Technical SkillsCore TechnologiesStrong programming skills in Java or ScalaHands-on experience with Kafka and streaming frameworks (Flink, Spark, Beam)Experience with event-driven architecture and distributed systemsInfrastructure & DevOpsDocker, Kubernetes for containerization and orchestrationCI/CD tools such as Jenkins, GitHubLinux OS and shell scriptingData & StorageExperience with SQL and NoSQL databasesFamiliarity with caching systems (Redis preferred)Security & NetworkingKnowledge of data security, encryption, and compliance standardsUnderstanding of networking basics (DNS, Proxy, ACLs)Experience with secure data transfers and ETL processesFunctional SkillsExperience working in Agile development environmentsAbility to ensure high-quality architecture and system designStrong analytical skills to evaluate and benchmark technologiesExperience in Banking, Financial Services, or Fintech domains is preferredSoft SkillsStrong communication and collaboration skillsAbility to work independently and drive initiativesPositive, proactive attitude with strong problem-solving abilitiesAbility to influence teams and contribute to technical decision-makingWe craft, deploy, and manage bespoke services in CRM, data and AI, cybersecurity and consulting.#J-18808-Ljbffr