Senior Data Engineer / Tech Lead (Spark)

Technologies we use

Expected

• Python
• SQL
• Spark
• Airflow
• AWS
• Iceberg

Optional

• Kubernetes
• Kafka
• NiFi
• Trino
• Hudi
• Java
• Scala
• Docker
• Databricks
• dbt

About the project

As a Senior Data Engineer (Spark), you will join a complex data engineering project focused on building and developing scalable data pipelines and modern data processing solutions.

The role combines approximately 70% hands-on data engineering with 30% technical leadership. Alongside designing and developing data solutions, you will take technical ownership of selected areas, contribute to architectural decisions, and support the team in delivering reliable and scalable solutions.

This is how we organize our work

This is how we work

• in house
• at the client's site
• you can change the project
• you have influence on the choice of tools and technologies
• you have influence on the technological solutions applied
• you have influence on the product
• you develop the code "from scratch"
• you focus on product development
• agile
• scrum

Team members

• technical leader
• big data developer
• data scientist
• project manager

Your responsibilities

• Design, develop, and maintain scalable data pipelines using Python, SQL, Spark, and Airflow
• Build and improve data processing solutions based on Apache Iceberg and modern lakehouse architectures
• Work with technologies such as PyArrow, PyIceberg, PostgreSQL, and Trino
• Take technical ownership of selected areas and contribute to technical and architectural decisions
• Combine hands-on development with technical leadership and knowledge sharing within the team
• Build and maintain containerized workloads using Docker and Kubernetes
• Ensure data pipelines are scalable, reliable, maintainable, and production-ready
• Collaborate with engineers and other stakeholders to understand business and technical requirements and translate them into effective solutions
• Contribute to engineering best practices, including code quality, testing, CI/CD, and automation
• Monitor, troubleshoot, and continuously improve existing data pipelines and platform components
• Use modern development practices, including AI-assisted development, to improve engineering efficiency

Our requirements

• At least 5 years of commercial experience in Data Engineering or Big Data projects
• Strong programming skills in Python
• Very good knowledge of SQL
• Strong hands-on experience with Apache Spark and distributed data processing
• Practical experience with workflow orchestration tools, preferably Apache Airflow
• Experience with Apache Iceberg and/or modern lakehouse architectures
• Experience working with Docker and Kubernetes
• Good understanding of data modelling, data pipelines, and data processing architectures
• Experience working with relational databases and/or query engines such as PostgreSQL or Trino
• General understanding of cloud environments and cloud-based data solutions
• Ability to take ownership of technical solutions and contribute to architectural and technical decisions
• Experience supporting other engineers through technical guidance, knowledge sharing, or mentoring
• Ability to work independently and take ownership of project deliverables
• Strong communication and collaboration skills
• Fluent English (at least C1 level).
• Bachelor’s degree in technical or mathematical studies.

Optional

• Experience with an MLOps framework such as Kubeflow or MLFlow.
• Familiarity with Databricks and/or dbt.

Division of working time

What we offer

• Work in a supportive team of passionate enthusiasts of AI & Big Data.
• Engage with top-tier global enterprises and cutting-edge startups on international projects.
• Enjoy flexible work arrangements, allowing you to work remotely or from modern offices and coworking spaces.
• Accelerate your professional growth through career paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences, including a partnership with Databricks, which offers industry-leading training materials and certifications.
• Choose your preferred form of cooperation: B2B or a contract of mandate, and enjoy 20 fully paid days off.
• Participate in team-building events and utilize the integration budget.
• Celebrate work anniversaries, birthdays, and milestones.
• Access medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
• Get full work equipment for optimal productivity, including a laptop and other necessary devices.
• With our backing, you can boost your personal brand by speaking at conferences, writing for our blog, or participating in meetups.
• Experience a smooth onboarding with a dedicated buddy, and start your journey in our friendly, supportive, and autonomous culture.

Development opportunities we offer

• assistance in preparation to public speeches
• conferences abroad
• conferences in Poland
• development budget
• external training
• industry-specific e-learning platforms
• intracompany training
• soft skills training
• substantive support from technological leaders
• support of IT events
• technical knowledge exchange within the company
• time for development of your ideas

Benefits

• Reimbursement for prescription glasses or contact lenses
• sharing the costs of sports activities
• private medical care
• sharing the costs of foreign language classes
• sharing the costs of professional training & courses
• remote work opportunities
• flexible working time
• fruits
• integration events
• no dress code
• video games at work
• birthday celebration
• employee referral program

Recruitment stages

• Let's get to know each other (45 min)
• Technical Interview (1 h)
• Management Interview (1 h)

Important:

Addepto may publish job opportunities on external job boards, but all applications are collected and processed through our official recruitment system: Recruitee

https://addepto.recruitee.com/

Please make sure that any further recruitment communication you receive comes from an official Addepto recruitment channel. If you receive a job offer, interview invitation, or recruitment email from an unofficial or unverified source, please be cautious, as it may not come from Addepto.

Addepto

Addepto is a leading AI consulting (https://addepto.com/ai-consulting/) and data engineering (https://addepto.com/data-engineering-services/) company that builds scalable, ROI-focused AI solutions for some of the world's largest enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. With an exclusive focus on Artificial Intelligence and Big Data, Addepto helps organizations unlock the full potential of their data through systems designed for measurable business impact and long-term growth.

The company's work extends beyond client engagements. Drawing from real-world challenges and insights, Addepto has developed its own product - ContextClue - and actively contributes open-source solutions to the AI community. This commitment to transforming practical experience into scalable innovation has earned Addepto recognition by Forbes as one of the top 10 AI consulting companies worldwide.

As part of KMS Technology, a US-based global technology group, Addepto combines deep AI specialization with enterprise-scale delivery capabilities—enabling the partnership to move clients from AI experimentation to production impact, securely and at scale.