Data Acquisition Research Engineer

OverviewWe are looking forDataAcquisitionResearchEngineers. They will develop and maintain large-scale web scraping and data acquisition systems. The role focuses ondesigning,developingand constantlymaintainingresilientdata collectionand acquisition systems andscrapers for complex web platforms. Thisincludesenvironments whereadversarialand reverse engineering may berequired.You will develop robust datapipelines andensure reliable collection of large volumes of structured and unstructured data.ResponsibilitiesDevelop andmaintaindata acquisition systems: web scrapers and automated data collectionDevelop data extraction methods that work with data repositories such asWebsites, APIs, andGraphQLendpointsResearchand solve data collection challenges inadversarial environments(rate limits, blocking, anti-bot systems,...)Developand integratescalable and reliable data pipelinesMonitor scrapers and ensuredata qualityand pipeline stabilityWork closely withother teams to transfercollecteddata intodownstream systemsRequirementsExperience withweb scraping frameworks(Scrapy, Playwright, Selenium, Puppeteer, etc.)Familiarity withGraphQLAPIs and REST APIsExperience handlinganti-bot protections and detection mechanismsExperience withCI/CD workflowsfor testing and deploying scraping infrastructureAbility to parse and processHTML, JSON, and dynamic web contentSoft SkillsPatience and persistence in handling complex data acquisition challengesCompetitive, results-driven mindset with a strong drive to solve difficult problemsAttention to detail and commitment to data qualityAbility to adapt quickly to changing platforms and adversarial environmentsGood communicationand collaboration skillsNice to HaveExperience buildingdistributed scraping systemsFamiliarity withcontainerization and cloud infrastructureExperience withdata processing pipelines or large-scale data systems#J-18808-Ljbffr