Senior Data Engineer

Opis oferty

We are looking for a highly skilled Senior Data Engineer to join our innovative data team. The ideal candidate will have extensive experience in data extraction, transformation, and loading (ETL) processes, as well as a strong background in cloud data tools. You will be responsible for building and optimizing data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver actionable insights.

Performance Excellence:

The ideal candidate will demonstrate high performance, consistently exceeding targets and expectations related to the data team. Contributions should lead to impactful projects, such as the Global Data Mart utilized across regions.

Subject Matter Expertise:

To deliver the company’s strategic objectives, the candidate must demonstrate significant expertise in the Financial Services domains, concentrated on a captive finance company like Volvo Financial Services. In-depth knowledge of local, regional, and global financial processes, and data related to our customers, dealers, and suppliers, including personal data is essential. Expertise in the Volvo Group’s infrastructure and security policies is crucial for success in this role, as this enables us to keep our data safe and ensure smooth daily operations. These qualifications will drive our company’s strategic objectives forward, helping us achieve our goal of becoming a fully data driven company

Wymagania

Qualifications:

• Education: Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
• Experience: 5+ years of experience in data engineering or a related field, with a strong focus on ETL processes and cloud data tools.
• Certifications required: Certified Azure Data Engineer, Databricks Certified Data Engineer

Technical skills:

• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with MS Cloud SQL Server, Azure Data Factory, and Azure Databricks, PySpark, Spark SQL, ADLS, SQL, PowerBI, SSIS, SSRS, Agile and Scrum
• Strong knowledge of SQL and experience with relational databases.
• Familiarity with big data technologies and data processing frameworks

Soft Skills: 

• Excellent problem-solving skills, strong communication abilities, and a collaborative mindset.

Zakres obowiązków

• Data Management: Extract, combine, transform, load, manipulate, and cleanse data from various sources to ensure high-quality datasets for analysis. This also includes data migration from legacy systems into next generation systems
• Collaboration: Work closely with data modelers and business analysts to understand the data model and source-to-staging data mapping.
• Data Structures: Create and integrate data structures suitable for analysis, including developing DDLs to create various MS Cloud SQL Server tables and views in Pond (a subset of the Data Lake) DB.
• Databricks Development: Design and develop Databricks notebooks for extracting data from various input files, joining related entities to create Shared File Formats (SFFs).
• Ingestion Framework: Configure the Ingestion Framework (IF) tables to utilize the existing framework for loading staging and Data Lake Core (DLC) tables.
• Algorithm Development: Develop and deploy algorithms in performance-efficient code across analytical and data platforms.
• Data Pipelines: Build large-scale batch and real-time data pipelines using data processing frameworks in cloud platforms or on-premises environments.
• Continuous Integration: Leverage best practices in continuous integration and delivery to enhance data workflows.
• Data Governance: Track data consumption patterns, collaborating with data governance and information stewards, and participate in vetting and promoting content for the curated data catalog.
• Automation: Recommend and implement automation in existing and future integration flows to improve efficiency.
• Data Quality Improvement: Drive advancements in data acquisition, testing, and tooling to enhance data quality.
• Agile Collaboration: Work in cross-functional agile teams to continuously experiment, iterate, and deliver on new product objectives.
• Requirement Analysis: Play a key role in requirement analysis and project estimates to ensure alignment with business goals.
• Performance Monitoring: Monitor performance and optimize SQL queries for maximum efficiency.
• Documentation: Prepare detailed design documents with prototypes and screenshots based on Business Requirement Documents (BRD) and System Requirement Documents.
• Cloud Tools Utilization: Work with various cloud data tools such as Azure SQL, Azure Data Factory, and Azure Databricks to ingest and prepare data for analysis as per business requirements.