Senior Automation Tester in Python

About the project

We are looking for a Senior Automation Tester in Python to join Production Pods within the program. This is not a manual/functional QA role. You will build and maintain automated test frameworks for AI/GenAI-powered clinical applications, validate non-deterministic AI outputs, and ensure GxP compliance through automated IQ/OQ/PQ qualification. You will work across multiple use cases in Safety, Regulatory, Clinical Operations, and SAGE platform tooling.

This is how we organize our work

This is how we work

• you focus on a single project at a time
• you focus on product development
• agile

Team members

• backend developer
• data scientist
• automated test programmer

Your responsibilities

• Build automated test frameworks from scratch for AI-powered clinical applications using Playwright, Selenium, and pytest
• Design and execute test strategies for non-deterministic AI/GenAI outputs, including accuracy, RAG relevance, hallucination checks, and prompt regression
• Integrate automated tests into CI/CD pipelines with quality gates such as block-on-failure and coverage thresholds
• Execute GxP-compliant IQ/OQ/PQ scripts and generate qualification evidence packages
• Manage AI-specific test data, including edge cases, adversarial inputs, and golden datasets
• Monitor model drift, regression, and performance degradation over time
• Produce test strategy documents, test plans, and compliance-ready test evidence
• Collaborate with Data Scientists and Backend Developers to define testable acceptance criteria for AI features

Our requirements

• 5+ years of hands-on QA/test automation experience
• Expertise in building and maintaining automated test suites from scratch using Playwright, Selenium, Cypress, and pytest
• Skills in API and backend test automation, including REST APIs, microservices, and serverless architectures with Postman/Newman and pytest + requests
• Proficiency in Python as the primary automation language, with the ability to write harnesses, fixtures, and utilities independently
• Knowledge of CI/CD test integration, embedding automated tests into pipelines such as Jenkins and GitHub Actions with quality gates
• Background in testing AI/GenAI applications, validating non-deterministic AI/LLM outputs for accuracy, RAG relevance, hallucination, and prompt regression
• Familiarity with AI/ML model testing, including accuracy, bias, and performance validation using RAGAS, LLM-as-judge, and golden datasets
• English proficiency at B2 level or higher

Optional

• Background in pharmaceutical / life sciences domain, with GxP awareness and IQ/OQ/PQ qualification
• Experience with Jira/Xray, TestRail, or Zephyr test management
• Skills in performance/load testing using JMeter and Locust
• Understanding of security testing basics such as OWASP
• Familiarity with AWS services including Lambda, API Gateway, S3, and CloudWatch

What we offer

We gather like-minded people:

- Top tech minds driving innovation in AI, cloud and digital platform modernization

- Supportive team and agile, startup-like culture

- Hybrid by design mode and opportunity to work remotely within Poland

- Chance to work abroad for up to 60 days annually

- Business-driven relocation opportunities

We provide growth opportunities:

- Career development programs

- Thought leadership, mentoring, soft skills and well-being programs

- Certification (Anthropic, Gemini, GCP, Azure, AWS)

- English classes

We cover it all:

- Stable pay

- Participation in the Employee Stock Purchase Plan with a 15% discount

- Benefits package (health insurance, multisport, shopping vouchers)

- Referral bonuses up to $2,000

- Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more

- Corporate, social and well-being events

Please, note:

- Benefits listed above are available to employees only

- We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually

- We will reach out to selected candidates exclusively

This is how we work on a project

• Continuous Integration
• integration tests
• performance tests
• test automation

Development opportunities we offer

• conferences in Poland
• development budget
• external training
• industry-specific e-learning platforms
• intracompany training
• mentoring
• soft skills training
• space for experimenting
• substantive support from technological leaders
• support of IT events
• technical knowledge exchange within the company

Benefits

• private medical care
• life insurance
• flexible working time
• fruits
• integration events
• retirement pension plan
• no dress code
• video games at work
• coffee / tea
• parking space for employees
• leisure zone
• employee referral program
• charity initiatives

EPAM Systems (Poland) sp. z o.o.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.