Senior Quality Engineer (Forward Deployed)

We’re a boutique agency—small yet impactful—gaining real momentum helping B2B eCommerce industry giants transform digitally with our specialized products and services.

Today, our wins are clear in the multi-million dollar weekly sales achieved online by our clients based in the USA and around the world. And for tomorrow, we’re strengthening our solutions through innovation, customization, and smart AI integration.

When you join Villvay, you don't just join a 'friendly neighborhood offshore development team'—you join a team of solutionizers—critical thinkers who see the whole picture, tackle problems end-to-end, and bring more to the table than anyone expects.

YOUR ROLE:

• Design, implement, and execute evals using agreed benchmarks, samples, and baselines, and analyse results statistically.
• Define measurement criteria when acceptance criteria are non-deterministic or similar, including criteria for agentic components.
• Present evaluation results to clients and internal stakeholders, highlighting risks and mitigation plans.
• Research evaluation techniques relevant to each engagement, document your findings, and implement improvements that make evaluation more efficient.
• Maintain locked baselines and approved sources of truth, and update benchmarks regularly based on stakeholder feedback.
• Report clear, reproducible defects with evidence and test artifacts, and take ownership of end-to-end testing process through the AI testing life cycle (AITLC) and the defect lifecycle.
• Execute functional, regression, and exploratory testing against defined acceptance criteria
• Perform automated, manual, and exploratory testing of web and mobile applications across multiple devices and platforms.
• Design, implement, and execute deterministic and non-deterministic test cases using agentic AI testing tools, then evaluate and document the results.
• Use AI tools with clear, structured prompts to generate test cases and draft evaluation steps.
• Collaborate with Engineering, Product, and DevOps to build a shift-left quality engineering discipline and culture aligned with industry standards and our core values.
• Design, execute, and document frontend and backend performance tests as required, analyse the results, and identify bottlenecks.
• Become an ambassador of Villvay’s values when representing the company to its clients and stakeholders.

REQUIREMENTS:

• 4 to 6 years of experience in software quality assurance or test engineering.
• Experience contributing to quality measurement approaches, evaluation and automation frameworks.
• Ability to design, implement, and execute deterministic and non-deterministic testing.
• Web, mobile, and API functional testing experience (automation and manual) across multiple devices and platforms.
• Experience presenting to clients or business stakeholders, with clear, formal written and verbal communication.
• Strong multitasking and context-switching ability across concurrent workstreams.
• Practical use of AI-assisted tools (Cursor, Claude, Rovo, or equivalent), with structured prompts and careful review of AI output before use.

ADDED ADVANTAGES:

• Ability to turn an unclear quality concern into a defined sample, baseline, test plan, and decision.
• Experience designing, implementing, and executing e-commerce search evaluations using production benchmarks, and analysing results statistically.
• In-depth understanding of search evaluation principles and how search-quality changes affect overall results.
• Prior client-facing or embedded delivery experience.
• Hands-on experience with Elasticsearch, e-commerce platforms, web and API automation tools, performance testing tools, and server interactions.

Villvay is an equal opportunity employer. But here are some job perks we are best known for:

• Competitive salaries for those who meet our high bar.
• Opportunities to network and connect with a global team, working with international clients and aided by world class consultants.
• A hybrid work model to bring out the best kind of productivity in our teams.
• Paid parental leave to genuinely encourage a healthy family life style.
• Paid menstrual leave because we believe in equity.
• A flat structure supported by a boutique concept of quality and transparency.
• OPD and Insurance because life happens, even during your probation period.

If you’re not just looking for a job—but looking to make things better, apply now.

You at Villvay

To become a VVayfinder means you are driven by purpose, curiosity and the need to leave things better than how you found them. You might be an ideal fit if you:

• Care deeply about how you can add value by doing purposeful work.
• Learn fast and adapt faster, seeking professional and personal growth.
• Bring critical thought and opinions to the table.
• Are fundamentally detailed and thorough, whatever the contribution may be.
• Using AI as a natural extension of your work, not an add-on.
• Work well with a competitive team, welcome what is different and makes good use of our feedback culture.
• Treat change as an opportunity—not a blocker, thriving in complexity and challenge.

For more on who we hire, check out our blog post → The Ideal VVayfinder

To make a real impact working with an unrelenting team, apply now.