
B2Bnetwork · Warszawa, Gdańsk, Łódź, Gdynia
IngestOps Engineer Data Engineer (f/m)
W skrócie
zdania wybrane z oferty- Build and maintain data ingestion processes for external systems.
- Very good knowledge of SQL, including joins, aggregations, stored procedures, and performance optimisation.
- Analyse logs and perform daily operations in a Linux environment.
Technologie
Pełny opis
z justjoin.itWe are looking for an experienced professional to join a team responsible for data ingestion, processing, and loading. This position combines data engineering responsibilities with the operational maintenance of data-processing environments.
Responsibilities
Build and maintain data ingestion processes for external systems.
Validate, transform, and prepare data for downstream processing.
Monitor data pipelines and troubleshoot operational issues.
Analyse logs and perform daily operations in a Linux environment.
Create, analyse, and optimise SQL queries.
Identify performance bottlenecks and improve data-processing efficiency.
Ensure the quality, integrity, and consistency of data loaded into target systems.
Cooperate with technical and business teams.
Participate in deployments and the maintenance of data-processing solutions.
Requirements
Very good knowledge of Linux, including log analysis, file operations, monitoring, and command-line tools.
Very good knowledge of SQL, including joins, aggregations, stored procedures, and performance optimisation.
Experience in building or maintaining data ingestion and integration processes.
Practical knowledge of data validation, cleansing, and transformation
Ability to analyse complex data flows and troubleshoot technical issues
Experience working with large data volumes
English proficiency sufficient for effective communication in an international project environment
Independence, accountability, and strong problem-solving skills.
Nice to have
Knowledge of Python.
Experience with Apache Spark, PySpark, or the Hadoop ecosystem.
Knowledge of Apache Airflow or other workflow orchestration tools.
Experience with Apache Kafka and stream processing.
Knowledge of Data Lake environments and ETL ELT processes.
Experience with cloud solutions, particularly AWS.
Familiarity with Docker, CI CD, Databricks, or Snowflake.
Treść ogłoszenia pochodzi z justjoin.it i należy do jego autora.
Historia oferty
bez zmian od publikacji- Podbita na górę listy
Pracodawca odświeżył ogłoszenie na portalu.
- Opublikowana na justjoin.it