Be the first to apply!
QA Engineer – AI Testing & Automation
HealthRecon Connect LLC
Colombo •
Full-time
Job Description
Are you passionate about quality assurance, automation, and the rapidly evolving world of AI? Join HRC Labs and play a key role in testing and validating cutting-edge AI solutions.
Key Responsibilities Include:
- Test Al-driven products including LLMS, RAG systems, Al assistants, chatbots, and ML prediction features
- Validate Al outputs for accuracy, relevance, consistency. hallucination, bias, safety, and business correctness.
- Test prompt behavior, system instructions, guardrails, model responses, and Al workflow quality.
- Work with local LLM tools such as Ollama, LM Studio, LocalAl or similar platforms
- Create Al test datasets, prompt regression suites, golden test sets, and evaluation rubrics.
- Perform functional, regression, integration, API, UI, database, and end-to-end testing.
- Build and maintain automation scripts using Selenium, Playwright Cypress, REST Assured, PyTest, or similar tools
- Use Java, Python, or JavaScript to support automation and technical validation
- Validate APIs, databases, logs, and backend workflows.
- Use Al-powered QA tools to improve test case creation, automation support, defect analysis, and QA productivity.
- Identify risks, requirement gaps, and quality issues early. Work independently with minimum guidance and take ownership of assigned deliverables.
- Collaborate with developers, business analysts, product owners, and QA team members.
Qualifications/Criteria:
- Bachelor's degree in computer science, Information Technology, or a related field
- Minimum of 3+ years of hands-on experience in manual and automation testing.
- ISTQB or Al testing certification will be considered an added advantage.
- 1 year of experience testing Al, ML, LLM, chatbot, automation, or data-driven products is preferred.
- Good understanding of AU/ML/LLM concepts. Knowledge of RAG systems, embeddings, vector search, prompts, tokens, and hallucination risks
- Hands-on experience with API testing and SQL
- Automation experience using Selenium, Playwright, Cypress, REST Assured, PyTest, or similar tools. Programming knowledge in Java, Python, or JavaScript
- Ability to analyze logs, troubleshoot issues, and validate backend data
Strong written and verbal communication skills. - Ability to work independently and responsibly.
- Experience with vector databases.