W skrócie
zdania wybrane z oferty- Design, develop, and maintain scalable, robust data pipelines and workflows to support data ingestion, transformation, and storage.
- Architect and optimize data models, ELT processes, and data architecture to ensure efficiency, quality, and security.
- The main purpose of the Data Engineer role is to contribute to the Core Engineering team by developing scalable and maintainable data platforms that enhance the decision-making process and support technological advancements within the organization.
Technologie
Pełny opis
z justjoin.itSummary
The main purpose of the Data Engineer role is to contribute to the Core Engineering team by developing scalable and maintainable data platforms that enhance the decision-making process and support technological advancements within the organization.
Main Responsibilities:
- Collaborate with analysts, architects, and stakeholders to understand requirements and produce maintainable solutions.
- Analyse raw data against requirements and provide early input for JIRA tickets, along with design and development activities.
- Design, develop, and maintain scalable, robust data pipelines and workflows to support data ingestion, transformation, and storage.
- Architect and optimize data models, ELT processes, and data architecture to ensure efficiency, quality, and security.
- Lead initiatives to improve existing data infrastructure, including migration, automation, and performance tuning.
- Monitor data quality and pipeline performance, troubleshoot issues, and minimize downtime.
- Ensure compliance with data governance, security policies, and privacy standards.
- Collaborate within a multidisciplinary engineering squad and provide technical guidance where necessary.
- Write and maintain documentation and operational runbooks.
- Support and improve the deployment processes.
Key Requirements:
- Strong analytical skills in working with both structured & unstructured datasets.
- In-depth knowledge of data modelling, data architecture, and ETL/ELT design principles.
- Expert-level proficiency in SQL.
- Proven track record developing data platforms on Cloud databases, ideally Snowflake.
- Hands-on experience working with AWS, particularly data services such as RDS, S3, Glue, etc.
- Proficiency in Python programming, including writing tests and maintaining code quality.
Nice to Have:
- Experience with DBT tooling.
- Experience processing raw data at scale (e.g., scripting, web scraping, API calls).
- Advocate of Continuous Integration/Continuous Deployment (CI/CD) and Test-Driven Development (TDD).
Other Details:
Team Structure: Multidisciplinary engineering squad. Technology Stack: AWS technologies and other third-party systems.
Treść ogłoszenia pochodzi z justjoin.it i należy do jego autora.
Historia oferty
bez zmian od publikacji- Opublikowana na justjoin.it