Sr. QA Engineer - Portugal

About the role

As a Sr. QA Engineer at HerculesAI, you’ll lead the charge in embedding quality throughout our entire software development lifecycle. You’ll own end-to-end product quality—from defining success criteria and automation strategies to managing releases and ensuring post-deployment stability. You’ll work closely with Engineering, Product, AI, and DevOps teams to ensure every release meets our standards for reliability, performance, and user impact—making quality a shared responsibility across the organization.

What you'll do

• Lead end-to-end product quality and integrate QA across the SDLC (shift-left, CI/CD quality gates, test evidence as part of “definition of done”).
• Own Release Management: plan releases, cut release candidates, manage freeze windows, lead go/no-go, coordinate phased rollouts (flags/canary), and execute rollback plans.
• Design and maintain automation for UI, API, component, and E2E tests in partnership with all Engineering teams; establish non-functional baselines (performance, security, accessibility, resilience).
• Design, own, and evolve performance and stress testing (load, capacity, scalability): define SLIs/SLOs, create repeatable perf/stress suites, profile bottlenecks, and gate releases via CI/CD and PRV.
• Translate business initiatives into clear acceptance criteria and measurable Success Criteria Docs (KPIs, telemetry, rollout/rollback triggers).
• Partner with Engineering, Product, AI, and DevOps to ensure observability, PRV (post-release verification), incident retrospectives, and continuous improvement of quality KPIs.

Qualifications

• 6–10+ years in QA/Software Engineering, including ownership of release management and large-scale test automation.
• Required: Proficiency in Python (test harnesses, AI/LLM eval tooling, CI utilities) and ReactJS (TypeScript preferred) for UI testability reviews and building test fixtures/mocks.
• Hands-on with modern delivery stacks (microservices, containers/K8s, CI/CD) and test frameworks (Playwright/Cypress, PyTest/JUnit/TestNG, k6/JMeter for performance).
• Demonstrated experience validating AI/LLM features (prompt testing, guardrails/red-teaming, offline/online eval alignment).
• Strong systems thinking and risk-based testing; fluency in telemetry/observability (logs, metrics, traces) and security/a11y/performance gates, including capacity planning and perf profiling.
• Excellent communication skills for turning ambiguous requirements into unambiguous, testable criteria and leading cross-functional quality reviews.

Success Metric Examples
(The Sr. QA Engineer will owns definition, targets, and reporting of these KPIs.)

• Shift-Left & SDLC Health: % stories with testable criteria at grooming ↑; pre-merge test pass rate ↑; escaped defects from unit/component layers ↓; lead time for changes ↓.
• Release Quality: change failure rate ↓; rollback/hotfix frequency ↓; PRV (post-release verification) pass rate ↑; incident MTTR ↓.
• Automation & Coverage: automated regression coverage ↑; flaky test rate ↓; time to fix flaky tests ↓; security/a11y/perf gate pass rate ↑.
• Performance & Resilience: p95/p99 latency ↓; throughput ↑; saturation/error budget burn ↓; successful load/stress/soak test pass rate ↑; resiliency checks (retry/backoff, timeouts) pass rate ↑.
• Observability & Evidence: % launches with telemetry + dashboards + alert thresholds = 100%; quality evidence attached to releases ↑; data-driven retros with action closure rate ↑.