Be the first to apply!

QA Engineer – AI Testing & Automation

HealthRecon Connect LLC   Colombo • Full-time

Job Description

Are you passionate about quality assurance, automation, and the rapidly evolving world of AI? Join HRC Labs and play a key role in testing and validating cutting-edge AI solutions.

Key Responsibilities Include:

  • Test Al-driven products including LLMS, RAG systems, Al assistants, chatbots, and ML prediction features
  • Validate Al outputs for accuracy, relevance, consistency. hallucination, bias, safety, and business correctness.
  • Test prompt behavior, system instructions, guardrails, model responses, and Al workflow quality.
  • Work with local LLM tools such as Ollama, LM Studio, LocalAl or similar platforms
  • Create Al test datasets, prompt regression suites, golden test sets, and evaluation rubrics.
  • Perform functional, regression, integration, API, UI, database, and end-to-end testing.
  • Build and maintain automation scripts using Selenium, Playwright Cypress, REST Assured, PyTest, or similar tools
  • Use Java, Python, or JavaScript to support automation and technical validation
  • Validate APIs, databases, logs, and backend workflows.
  • Use Al-powered QA tools to improve test case creation, automation support, defect analysis, and QA productivity.
  • Identify risks, requirement gaps, and quality issues early. Work independently with minimum guidance and take ownership of assigned deliverables.
  • Collaborate with developers, business analysts, product owners, and QA team members.

Qualifications/Criteria:

  • Bachelor's degree in computer science, Information Technology, or a related field
  • Minimum of 3+ years of hands-on experience in manual and automation testing.
  • ISTQB or Al testing certification will be considered an added advantage.
  • 1 year of experience testing Al, ML, LLM, chatbot, automation, or data-driven products is preferred.
  • Good understanding of AU/ML/LLM concepts. Knowledge of RAG systems, embeddings, vector search, prompts, tokens, and hallucination risks
  • Hands-on experience with API testing and SQL
  • Automation experience using Selenium, Playwright, Cypress, REST Assured, PyTest, or similar tools. Programming knowledge in Java, Python, or JavaScript
  • Ability to analyze logs, troubleshoot issues, and validate backend data
    Strong written and verbal communication skills.
  • Ability to work independently and responsibly.
  • Experience with vector databases.