Senior Databricks Engineer - Governance

Technologies we use

Expected

• Databricks
• Unity 3D

Optional

• Terraform
• Databricks
• Python
• SQL
• Apache Spark
• Azure DevOps
• AWS

About the project

Your role

You will be a senior Databricks engineer who owns the governance layer of the platforms we build. On Databricks engagements you work as any senior engineer does, on pipelines, data model and platform setup, and you take charge of Unity Catalog, access and evidence. On governance engagements, our data and AI governance consultants decide what good looks like for a client: who owns which data, who may see what, which quality rules apply, what an auditor needs to see. You make it true in Unity Catalog and in the pipelines around it: metastore and catalog design, access models that hold (groups, row filters, column masks, tag-based policies), lineage and audit evidence pulled from system tables, data quality checks inside pipelines, models and AI assets governed through Unity Catalog and MLflow. You will work daily with our data governance and AI governance consultants and with the legal and sovereignty specialists, and you will learn the regulatory side from them. The Databricks depth you bring with you.

Your responsibilities

• Design and implement Unity Catalog for clients: metastore and catalog structure, workspace binding, external locations and storage credentials, identity federation (SCIM, service principals), ownership model.
• Build access control that reflects the governance design: privileges and groups, row filters and column masks, attribute-based access control with tags, data classification; test it and document it so that auditors can read it.
• Produce governance evidence from the platform: lineage, audit logs and billing from system tables, access reviews, dashboards for data owners and compliance teams; integrate with enterprise catalogs (Microsoft Purview, Collibra) where the client has them.
• Implement data quality in pipelines: expectations in Lakeflow pipelines, Lakehouse Monitoring, quality metrics published to the catalog and to the governance team.
• Govern AI assets on the platform with the AI governance team: models and functions registered in Unity Catalog, MLflow approval workflows, AI Gateway policies for LLM endpoints, logging for traceability.
• Implement sovereignty and residency controls with the sovereignty team: region placement, customer-managed keys, private connectivity, Delta Sharing and Lakehouse Federation configured so that data stays where it must.
• Migrate clients from Hive metastore and legacy permission models to Unity Catalog, and clean up what governance cannot see.
• Build reusable assets: Unity Catalog reference setups as code (Terraform, Databricks Asset Bundles), access policy patterns, evidence dashboards; contribute to the governance-by-design reference architecture.

Our requirements

• 5+ years of hands-on Databricks engineering in production, including at least two years designing and administering Unity Catalog at enterprise scale (several workspaces, many teams).
• Deep knowledge of Unity Catalog: privilege model, row and column security, tags and attribute-based policies, system tables, lineage, Delta Sharing, Lakehouse Federation; migration from Hive metastore.
• Strong Spark, SQL and Python; Delta Lake; Lakeflow pipelines and jobs; infrastructure as code with Terraform and Databricks Asset Bundles; CI/CD.
• Working knowledge of cloud security on Azure or AWS as it applies to Databricks: identity, storage access, private networking, key management.
• Databricks Certified Data Engineer Professional or Associate; Platform Administrator accreditation is a plus.
• Interest in data governance and regulation; experience with a governance or catalog tool (Purview, Collibra) or with GDPR, EU AI Act or sector-regulated data is a plus.
• Ability to explain an access model or a lineage graph to a non-technical data owner; fluent English.
• Experience in financial services, energy, pharma or public sector is a plus.

What we offer

• Practical benefits: yearly financial bonus, private medical care with Medicover with additional packages (e.g., dental, senior care, oncology) available on preferential terms, life insurance and access to NAIS benefit platform.
• Access to over 70 training tracks with certification opportunities (e.g., GenAI, Excel, Business Analysis, Project Management) on our NEXT platform. Dive into a world of knowledge with free access to Education First languages platform, TED Talks and Udemy Business materials and trainings.
• Cutting-Edge Technology: Position yourself at the forefront of IT innovation, working with the latest technologies and platforms. Capgemini partners with top global enterprises, including 145 Fortune 500 companies.
• Enjoy hybrid working model that fits your life - after completing onboarding, connect work from a modern office with ergonomic work from home, thanks to home office package (including laptop, monitor, and chair). Ask your recruiter about the details.

Benefits

• access to courses e.g. Excel, VBA, RPA, Customer Care
• unlimited access to Udemy Business
• free chat/call with a therapist
• sharing the costs of sports activities
• private medical care
• life insurance
• no dress code
• parking space for employees
• extra social benefits
• redeployment package
• employee referral program
• charity initiatives

Recruitment stages

• Interview with the recruiter
• Competency Tests/Language Verification
• Video call with a manager
• Final Decision

Capgemini Polska

Capgemini is the business transformation partner for enterprises in the age of AI. We help organizations imagine and build an intelligent, sustainable future, combining AI, technology and human ingenuity to transform how they operate, innovate and grow. With unique end-to-end capabilities spanning strategy, technology, engineering and intelligent operations, we bring together deep industry expertise and market-leading capabilities in AI, cloud and data to turn ambition into measurable business outcomes at scale. Supported by a robust ecosystem of partners and nearly 60 years of expertise, Capgemini is a responsible and diverse global organization of over 410,000 team members in more than 50 countries. The Group reported 2025 revenues of €22.5 billion.

This is how we work