Azure Data Engineer

As a Data Engineer you are responsible for the data pipelines, all imaging data, metadata, and tools for interacting with such data and have experience in processing of big files (CSV, XML over 1 GB). You have hands-on experience with Azure, Databricks, Spark, Azure Synapse, Python, PySpark, and SQL. You are used to working in international organizations and are willing to learn new skills. Besides the technical experience we are looking for data engineers that can cooperate, have strong analytical skills, are result-driven & customer focused, while still following & expanding the data architecture.

— Technical Skills (must have)
• Azure Databricks
• Azure Active Directory
• Azure Data Lake Storage
• Delta format
• Spark, Structure Streaming
• Python, PySpark, SQL
• Parquet, Avro

• Understanding of Data Lake and Data Lakehouse concepts
• Knowledge about processing of big files (CSV, XML over 1 GB)
• Processing of unstructured log data
• Productionzation of Spark Structure Streaming solution
• Deployment and Release (CI/CD) experience with data solutions
• Performance of querying over Big Data
• Orchestration of Spark processing

— Other Skills (nice to have)
• Understanding of Data Mesh concept
Microservice architecture

Contract Duration: 12 Months

Required level of English: Fluent
Required level of other language: German good to have

450 - 550 per day EUR/hr

File name:

File size: