AI Application Test Enginee (m/f/d)

We are looking for an experienced AI Application Test Engineer to join our growing team. You will focus on testing AI-powered applications, including conversational AI and RAG-based systems, ensuring they meet quality standards and perform reliably at scale.Key ResponsibilitiesDesign and execute test cases for AI/ML applications, with emphasis on conversational AI and RAG architecturesPerform end-to-end testing of AI applications across development, staging, and production environmentsTest LLM-based applications for accuracy, consistency, and appropriate responsesValidate retrieval systems, vector search functionality, and document ingestion pipelinesConduct functional, regression, and integration testing for AI componentsIdentify and document bugs, edge cases, and model behavior issuesCollaborate with cross-functional teams to ensure quality throughout the AI development lifecycleContribute to the development of AI testing best practices and standardsRequired Technical SkillsAI/ML Testing: 3+ years of hands‑on experience testing AI applications, chatbots, or RAG-based solutionsAzure Platform: Strong experience with Azure AI services including Azure OpenAI, Azure ML, Cognitive Services, and Azure AI SearchTest Automation: Proficiency in automation frameworks and tools (Selenium, Playwright, PyTest, etc.)Programming: Good coding skills in Python for test automationAI Concepts: Understanding of LLM behavior, prompt testing, and conversational AI patternsAPI Testing: Experience testing REST APIs, especially AI model endpointsCloud Testing: Familiarity with testing cloud‑native applications on AzureVersion Control: Experience with Git and collaborative development workflowsPreferred QualificationsExperience with Azure DevOps and CI/CD pipelinesKnowledge of data quality testing and validation techniquesFamiliarity with vector databases (e.g., Pinecone, Weaviate, Azure AI Search)Understanding of AI model evaluation metricsExperience in Agile environments using Jira or similar toolsExposure to MLOps practicesHalian GroupWith over 28 years of experience, we have come to understand that innovation is the only way to provide agile, practical solutions that transform businesses and careers. Our resourcing and smart services help you to realize tomorrow’s potential. Discover the amazing things possible when you bring the right people and the right technologies together. At Halian, we recognize that diversity, equity, and inclusion (DEI) are essential to building high-performing teams for our clients. We are committed to connecting organizations with top talent from all backgrounds, ensuring that every individual feels valued, respected, and empowered to contribute their unique perspectives. We encourage applications from all qualified candidates, regardless of race, gender, disability, or any other characteristic that makes them unique. By fostering diverse and inclusive workplaces, we help our clients drive innovation, enhance collaboration, and better reflect the communities they serve.#J-18808-Ljbffr