Azure Databricks Engineer

Technologies we use

Expected

• Azure Databricks
• Delta Lake
• Python
• PySpark
• Spark
• SQL
• T-SQL
• Unix
• Hive

Optional

• Spark
• Kafka
• Flink
• HBase

Operating system

• Windows

About the project

We are currently looking for an Azure Databricks Engineer to join the team of one of the world's leading global investment banks. The project is delivered together with a large international IT services company. For this role, Optiveum is the recruitment and contracting party, which means you will sign the cooperation agreement directly with us.

You will design and build big data solutions in Azure Databricks for batch and real-time analytics, working with Delta Lake, PySpark, Kafka streaming and the Lakehouse / Medallion architecture.

Key Details:

Rate: 135–160 PLN/h (B2B), depending on experience

Work Model: Hybrid, 3 days per week from the client's office (non-negotiable), full-time

Location: Kraków (candidates must live in or near Kraków)

This is how we organize our work

This is how we work

• at the client's site
• you have influence on the technological solutions applied
• you focus on product development
• agile
• scrum

Your responsibilities

• Design and implement big data solutions in Azure Databricks for batch and real-time analytics use cases
• Build Spark pipelines that populate downstream data marts for BI and generate feeds for Data Science
• Integrate real-time services that process business events from Kafka and persist them in ADLS Gen2 using Databricks
• Develop near real-time stream processing to derive features for ML model inference
• Optimise Databricks performance (liquid clustering, vacuum, Z-ordering etc.)
• Work with DevOps teams to drive operationally excellent infrastructure

Our requirements

• Bachelor's degree or higher in Computer Science, Engineering or Science
• 8–10 years of experience, including 5+ years with Azure cloud-native services
• Strong hands-on experience with Azure Databricks, Delta Lake, Python / PySpark and Spark
• Knowledge of Lakehouse, Data Mesh, Data Virtualisation, dimensional modelling and data aggregation techniques
• Experience building distributed data platforms (e.g. Spark, Kafka, Flink, HBase) at large scale
• Hadoop-based implementations, SQL / T-SQL, Unix and Hive
• Azure Data Factory, ADLS Gen2 and Azure SQL Database
• Good understanding of RDBMS, Data Lake, Data Warehouse, Lakehouse and Medallion Architecture
• At least one workflow orchestration tool (Apache Airflow, Luigi, Oozie, AWS Glue or similar)
• Knowledge of Microsoft Fabric and basic understanding of data quality dimensions
• Fluent English
• Polish or EU citizenship, or a permanent residence / work permit valid for at least 12 months

This is how we work on a project

• documentation
• issue tracking tools

Development opportunities we offer

• external training
• mentoring

Recruitment stages

• IT Partner & Bank Interviews
• Technical Interview & Live Coding
• Background Check
• Identity Verification
• Eligibility Review

• One interview with the IT services partner, followed by 1–3 interviews with the bank
• Technical tasks are done live: please have your programming environment set up and tested beforehand. The use of AI tools, coding assistants or code generators during interviews is not allowed
• The selected candidate will go through a background check by an external company (HireRight), covering identity, right to work, education and employment history, so the dates and employers in your CV must be accurate
• Your name in the application and CV must match your official ID document exactly (Polish citizens: please include all first names)
• Candidates who took part in a recruitment process for this IT services partner or this bank in the last 12 months cannot be considered

Interested? Apply now and help a leading Nordic bank stay ahead of security risks!

OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ

Optiveum is acting as a sourcing partner for this role. You will be signing a cooperation agreement directly with our partner—a renowned IT contracting company with nearly 20 years of presence on the market.

The end client operates in the international public, defense, or government administration sector, leading large-scale, multi-million euro digital transformation projects.

This is how we work