Data Engineer (Temporary position)

Job Summary:We are looking for a Mid-level Data Engineer to join a small, focused team building and maintaining a production-grade financial data processing platform for the Abu Dhabi Securities Exchange (ADX).The platform ingests daily stock trading data from ADX, processes it through a series of ETL pipelines, tracks shareholder balances and positions, and powers downstream analytics via PostgreSQL and Elasticsearch. The codebase is Python-first, well-structured, and held to high quality standards (strict typing, clean architecture, comprehensive tests).You will own the data and application layer — designing pipelines, evolving the schema, writing business logic services, and keeping the system accurate and reliable. The CTO handles all infrastructure and deployment concerns. We are looking someone extremely curious and likes to connect the dots with data and understand sentiment. The ideal candidate will be a good storyteller with data and statistics Problem-solving, adaptability in fast-paced environments, and familiarity with financial and trading platform databases.Job Responsibilities:Build and maintain ETL pipelines that ingest daily Excel/CSV trade files, transform and validate the data, and persist it to PostgreSQLDesign and evolve database schemas using SQLAlchemy ORM and Alembic migrationsImplement business logic services: shareholder balance tracking, sharebook reconciliation, movement detection, classification rulesMaintain and extend Elasticsearch sync services (incremental and full resync workflows)Support business stakeholders with data questions using the ELK stack — writing and running queries, interpreting results, and surfacing relevant data from KibanaParse and process structured financial data files (Excel, CSV) using Pandas, including data validation and deduplicationWrite robust, type-safe Python following project standards (mypy strict mode, Black, isort)Write and maintain automated tests with pytest — unit tests, repository mocks, snapshot testsCollaborate with the CTO on data flow design and review migration strategies before deploymentRequired Skills and Qualifications:3–5 years of professional software/data engineering experienceAt least 2 years working with Python in a production environmentPython 3.8+ — 3+ years of production experience; comfortable with type hints, async/await, and clean OOP designPostgreSQL — schema design, indexing strategies, query optimization, migrationsSQLAlchemy 2.0 — declarative ORM models, session management, Alembic for migrationsPandas — data transformation, Excel/CSV parsing, validation pipelinesFastAPI — building and maintaining REST APIs with dependency injection and async handlersPytest — writing unit and integration tests, using fixtures and mocking patternsELK Stack (Elasticsearch, Kibana) — working knowledge of Elasticsearch indexing and querying; able to write queries and navigate Kibana to answer business questions (dashboard building not required; Logstash not used)Code quality discipline — strong typing, Black formatting, readable and maintainable codeNice to HaveWeb scraping — familiarity with Playwright or similar browser automation toolsFinancial data domain — understanding of stock trading concepts (OHLCV data, shareholder registers, portfolio performance metrics such as XIRR or TWR)Financial APIs — experience with finance or similar market data sources#J-18808-Ljbffr