Required for a global market leader in Aireports a Data Engineer.
- saraaraby
- Aug 7, 2021
- 2 min read
Your tasks and responsibilities:
1. You will be designing, developing, testing, documenting, deploying and maintaining the data collection framework. The data collection consists of (complex) data pipelines from (IoT) sensors and low/high level control components to our Data Science platform.
2. You will develop scalable data pipelines to transform and aggregate data for business use, following software engineering best practices. For these data pipelines you will make use of the best frameworks available for data processing like Spark and Splunk. We are continuously improving on the solutions we use and encourage you to keep challenging the status quo.
3. You develop our data services for customer sites towards a product, using (test & deployment) automation, componentization, templates and standardization in order to reduce delivery time of our projects for customers. The product provides insights in the performance of our material handling systems at customers all around the globe.
4. You design and build a CI/CD pipeline, including (integration) test automation for data pipelines. In this process you strive for an ever-increasing degree of automation.
5. You work with the IT department to develop the underlying cloud services.
6. You will coach and train the junior data engineer with the state of art big data technologies.
Your qualifications and skills
Hard skills:
• Hands-on experience in
o Programming
Python 3
Scala/Java (or other JVM based languages)
Nice to have c#
o Building data pipelines and managing them in a production environment
Processing frameworks (e.g. Spark, Dask, Flink)
o DevOps tooling like test automation and CI/CD
Docker and Kubernetes
o Streaming and/or batch solutions (e.g. Kafka, Cassandra)
o Data storage and evolution
• Proficient experience with
o Cloud solutions such as Microsoft Azure, AWS
o Functional Programming
o Event processing tools like Splunk or the ELK stack
o Message queues (e.g. Kafka, RabbitMQ, ActiveMQ, Pulsar)
• Good knowledge on front-end development languages
o HTML, CSS
o Javascipt
Location: Veghel
For applying, fill the application https://bit.ly/TalenRecIT2021
Follow other vacancies on:
https://saraaraby.wixsite.com/talenrec/blog
Comments