Senior Data Engineer / Tech Lead (Spark)

Opis oferty

Addepto is a leading AI consulting (addepto.com/ai-consulting/) and data engineering (addepto.com/data-engineering-services/) company that builds scalable, ROI-focused AI solutions for some of the world's largest enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. With an exclusive focus on Artificial Intelligence and Big Data, Addepto helps organizations unlock the full potential of their data through systems designed for measurable business impact and long-term growth.

The company's work extends beyond client engagements. Drawing from real-world challenges and insights, Addepto has developed its own product - ContextClue - and actively contributes open-source solutions to the AI community. This commitment to transforming practical experience into scalable innovation has earned Addepto recognition by Forbes as one of the top 10 AI consulting companies worldwide.

As part of KMS Technology, a US-based global technology group, Addepto combines deep AI specialization with enterprise-scale delivery capabilities—enabling the partnership to move clients from AI experimentation to production impact, securely and at scale.

As a Senior Data Engineer (Spark), you will join a complex data engineering project focused on building and developing scalable data pipelines and modern data processing solutions.

The role combines approximately 70% hands-on data engineering with 30% technical leadership. Alongside designing and developing data solutions, you will take technical ownership of selected areas, contribute to architectural decisions, and support the team in delivering reliable and scalable solutions.

Discover our perks and benefits:

• Work in a supportive team of passionate enthusiasts of AI & Big Data.
• Engage with top-tier global enterprises and cutting-edge startups on international projects.
• Enjoy flexible work arrangements, allowing you to work remotely or from modern offices and coworking spaces.
• Accelerate your professional growth through career paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences, including a partnership with Databricks and Anthropic, which offers industry-leading training materials and certifications.
• Participate in team-building events and utilize the integration budget.
• Celebrate work anniversaries, birthdays, and milestones.
• Access medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
• Get full work equipment for optimal productivity, including a laptop and other necessary devices.
• With our backing, you can boost your personal brand by speaking at conferences, writing for our blog, or participating in meetups.
• Experience a smooth onboarding with a dedicated buddy, and start your journey in our friendly, supportive, and autonomous culture.

Important: Addepto may publish job opportunities on external job boards, but all applications are collected and processed through our official recruitment system: Recruitee
addepto.recruitee.com

Please make sure that any further recruitment communication you receive comes from an official Addepto recruitment channel. If you receive a job offer, interview invitation, or recruitment email from an unofficial or unverified source, please be cautious, as it may not come from Addepto.

Wymagania

What you’ll need to succeed in this role:

• At least 5 years of commercial experience in Data Engineering or Big Data projects
• Strong programming skills in Python
• Very good knowledge of SQL
• Strong hands-on experience with Apache Spark and distributed data processing
• Practical experience with workflow orchestration tools, preferably Apache Airflow
• Experience with Apache Iceberg and/or modern lakehouse architectures
• Experience working with Docker and Kubernetes
• Good understanding of data modelling, data pipelines, and data processing architectures
• Experience working with relational databases and/or query engines such as PostgreSQL or Trino
• General understanding of cloud environments and cloud-based data solutions
• Ability to take ownership of technical solutions and contribute to architectural and technical decisions
• Experience supporting other engineers through technical guidance, knowledge sharing, or mentoring
• Ability to work independently and take ownership of project deliverables
• Strong communication and collaboration skills
• Fluent English (at least C1 level).
• Bachelor’s degree in technical or mathematical studies.

Zakres obowiązków

• Design, develop, and maintain scalable data pipelines using Python, SQL, Spark, and Airflow
• Build and improve data processing solutions based on Apache Iceberg and modern lakehouse architectures
• Work with technologies such as PyArrow, PyIceberg, PostgreSQL, and Trino
• Take technical ownership of selected areas and contribute to technical and architectural decisions
• Combine hands-on development with technical leadership and knowledge sharing within the team
• Build and maintain containerized workloads using Docker and Kubernetes
• Ensure data pipelines are scalable, reliable, maintainable, and production-ready
• Collaborate with engineers and other stakeholders to understand business and technical requirements and translate them into effective solutions
• Contribute to engineering best practices, including code quality, testing, CI/CD, and automation
• Monitor, troubleshoot, and continuously improve existing data pipelines and platform components
• Use modern development practices, including AI-assisted development, to improve engineering efficiency