Hi there! AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
Why join us If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you! :)
About the role We are looking for a Senior QA Engineer to perform manual and automation testing of web applications, validate AI-generated test cases and LLM outputs, and contribute automation scripts to a stable existing framework under the guidance of a dedicated Automation Lead. You will design and execute test plans in TestRail, conduct API testing with Postman, document defects in Jira, and collaborate across Agile sprint cycles. The role suits an engineer with a strong manual background who is actively growing their automation skills using tools including Claude and GitHub Copilot.
What you will do ● Perform manual and automation testing of web applications to identify bugs, glitches, and inconsistencies; ● Take ownership of automation tasks within the team and proactively develop your automation skills; ● Develop, maintain, and execute detailed manual test cases based on product requirements and specifications; ● Perform thorough cross-browser UI testing to ensure consistent behavior across Chrome, Firefox, Safari, Edge, etc.; ● Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs to ensure accuracy, completeness, and alignment with project requirements; ● Design and execute test scenarios to verify AI behavior, including prompt—response consistency, contextual relevance, and deterministic vs. non-deterministic outputs; ● Evaluate AI model quality using defined acceptance metrics (precision, recall, hallucination rate, duplication rate); ● Conduct manual API testing to validate request/response accuracy, error handling, and integration; ● Use TestRail to create, manage, and track test plans, test cases, and test execution results; ● Use Jenkins CI/CD pipelines; ● Report and document defects clearly using the bug tracking system (JIRA); ● Participate in daily stand-ups, sprint planning, and other Agile team meetings; ● Review user stories and acceptance criteria to understand the scope of testing; ● Estimate own tasks, report on risks to the team lead; ● Collaborate with developers and other QA team members to reproduce issues and verify fixes; ● Maintain and update test documentation as needed.
Must haves ● 4+ years of experience in software testing; ● Strong manual & automation knowledge; ● Basic understanding of SDLC and STLC (Software Development and Testing Life Cycle) models; ● Knowledge of Testing and Quality Assurance processes; ● Strong experience with Test Design activities to cover a set of product requirements; ● Knowledge of Test Design techniques; ● Knowledge of testing methodologies; ● Defect analysis skills and the ability to write detailed and useful defect reports with expected defect-fixing outcomes; ● Experience with Postman (API testing) and Browser developer tools; ● Experience with UI testing; ● Experience with TestRail as a Test Management system (as a user, test execution is required); ● Basic understanding of CI/CD process; ● English — an Intermediate, higher knowledge level is an advantage.
Nice to haves ● Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles; ● Familiarity with AI/ML concepts, APIs, and frameworks (e.g., OpenAI API, Azure OpenAI, LangChain); ● Knowledge of data protection and compliance standards for AI systems (GDPR, zero-data retention, PII anonymization); ● Experience in SSO testing is a plus; ● Experience in Performance or Load testing is a plus.
Perks and benefits ● Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps ● Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities ● A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands ● Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office — whatever makes you the happiest and most productive.
Meet Our Recruitment Process Asynchronous stage — An automated, self-paced track that helps us move faster and give you quicker feedback: ● Short online form to confirm basic requirements ● 30–60 minute skills assessment ● 5-minute introduction video Synchronous stage — Live interviews ● Technical interview with our engineering team (scheduled at your convenience) ● Final interview with your future teammates If it’s a match—you’ll get an offer!
