Product Manager – Data & ML Evaluation (Enterprise) [UAEN]

Full time | Black Pearl Consult | United Arab EmiratesPosted On 03/26/2026Job InformationTechnologyNumber of Positions 1City Abu Dhabi51133Job DescriptionOur client is seeking a highly analytical and technically strong Product Manager – Data & ML Evaluation to lead the strategy and execution of product data, experimentation, and machine learning evaluation.In this role, you will act as the product owner for data and evaluation frameworks, ensuring that AI-driven products are measurable, reliable, and continuously improving. You will work cross-functionally with Product, Engineering, Data Science, and business stakeholders to deliver high-impact, data-driven solutions.Key Responsibilities:Own and manage the end-to-end product data ecosystem, including telemetry, tracking plans, event schemas, data quality, and warehousingDefine and implement a product metrics framework (north star metrics, leading indicators, input metrics)Enable stakeholders with accessible dashboards and actionable insights across products and platformsML Evaluation & ExperimentationDesign and own machine learning evaluation frameworks, including test set curation and KPI definition (quality, safety, bias, latency, cost)Run and optimize offline and online evaluations for AI models and featuresBuild scalable and reproducible evaluation pipelines integrated with CI/CD workflowsLead A/B testing and experimentation strategies, including designing, analyzing, and scaling testing frameworksDrive data-informed pricing strategies by analyzing usage patterns, feature value, and cost structuresEstablish and enforce data governance standards, including event naming, privacy, PII handling, and data retention policiesCollaborate with Security, Legal, and business teams to ensure compliance and best practices.Requirements5–8+ years of experience in Product Management with a strong background in data, analytics, or engineeringProven experience owning product analytics, instrumentation, and experimentation frameworksStrong proficiency in SQL and data analysis, with the ability to validate and interpret experiments independentlyHands‑on experience working with machine learning products, including model evaluation and monitoringStrong understanding of A/B testing methodologies and statistical concepts (e.g., power, MDE, guardrail metrics)Excellent stakeholder management and communication skills across technical and business teamsAbility to operate effectively in fast‑paced, cross‑functional environmentsPreferred QualificationsExperience with product analytics tools (e.g., Amplitude or similar) and experiment tracking platforms (e.g., MLflow)Exposure to LLM / RAG evaluation frameworks, safety metrics, and AI model testing methodologiesFamiliarity with data modeling tools (e.g., dbt), feature stores, and observability platformsExperience working in regulated or privacy-sensitive environmentsComfort leveraging AI tools to prototype analyses, dashboards, or lightweight data utilitiesFollow the Black Pearl channel on WhatsApp.Disclaimer: Black Pearl will never ask for money or any form of charge our candidates just to process or consider their application for any of our available vacancies. If you happen to receive such a request from any members of our staff or other individuals claiming to be part of Black Pearl, please do call our office at +9712 622 55 03 or drop us a message on our website - www.blackpearlconsult.com.#J-18808-Ljbffr