Data Operations Engineer

BA Markets wants to professionalise and streamline its activities with regards to data streaming. Therefore, a new agile team has been created to build up a technical platform for BA Markets users to get a hub for all required streaming services. ​

The team will develop the service suite largely internally but will also rely on managed vendor services, such as Databricks. The streaming technology used is Kafka. ​

Many BA Markets users are technically quite advanced so the central data streaming team must constantly keep up to date to deliver state of the art services.So, there will be constant development opportunities for the candidate, and the candidate must be comfortable to work in a fast-changing, international environment. The team consists of four members in Hamburg, one member in Stockholm and should also have three colleagues in Poland. 

What will you do? 

This is a hybrid role to support the Data Engineers and Software developers with automatization and simplification for the users. ​

Vision for this role should be to “Simplify the user experience”. ​

​Your tasks and responsibilities: 

• Stream deployment and stream architecture, developments and deployments ​
• Automate workflows and orchestrate data pipelines. ​
• Implement CI/CD routines​
• Implement and monitor “system health” with observability tools and data quality checks​
• Support the development of Client Libraries so other applications can integrate streams in own application and services ​
• Perform Python development ​
• Perform “glue code” development that 95% of use cases can apply. ​

Location  
Katowice or in Poland remotely

Who are you? 

As our Data Operations Engineer you have:

• Interest in understanding what the user needs with several years of hands-on experience as software developer with an interest in the responsibilities of a data engineer or vice versa. ​
• A proactive, communicative team player. Fluent in English.

Hard skills (the more the better) ​

• Deep understanding of Kafka architecture (brokers, topics, partitions, replication), and experience with Kafka Streams, Kafka Connect, and schema registry (e.g., Confluent). ​
• Proficiency in designing and managing Kafka clusters (including monitoring and scaling).​
• Hands-on Experience building and maintaining real-time ETL pipelines. Familiarity with stream processing frameworks like: Apache Flink or Apache Spark Streaming​
• Strong skills in: Python and Java and at least basic Scala and at least solid SQL experience ​
• Has build several CI/CD pipelines (e.g., Jenkins, GitLab CI, GitHub Actions). And used infrastructure as Code (IaC) tools like Terraform or Ansible. ​
• Hands-on performed containerization (Docker) and orchestration (Kubernetes) tasks. ​
• Knows monitoring tools like Grafana, ELK stack, Datadog. And ideally also Kafka-specific monitoring tools such as (e.g., Burrow, Confluent Control Center).​
• Experience with Databricks components is a merit. ​
• Experience with schema management best practices.

Our offer 
Good remuneration, a challenging and international work environment, and the possibility to work with some of the best in the field. You will be working in interdisciplinary teams and you can always count on support from committed colleagues. We offer attractive employment conditions and opportunities for personal and professional development. 

More Information 
We welcome your application in English. We kindly request that you do not send applications by any means other than via our website as we cannot guarantee that we will be able to process applications that are not made via our website. 
 
For more information about the recruitment process you are welcome to contact our recruiter Ewa Krajewska via ewa.krajewska@vattenfall.com

At Vattenfall we are convinced that diversity contributes to build a more profitable and attractive company and we strive to be good role model regarding diversity. Vattenfall works actively for all employees to have the same opportunities and rights regardless of gender, ethnicity, age, transgender identity or expression, religion or other belief, disability or sexual orientation. 

The security of Vattenfall and its employees is essential. For that reason, a pre-employment screening will be part of your recruitment process. The screening is based on the role you will fulfill within Vattenfall. The screening will be performed by a third party, DISA Global Solutions AB.
 
We look forward to receiving your application! 

Vattenfall is a European energy company with approximately 21 000 employees. For more than 100 years we have electrified industries, supplied energy to people’s homes and modernized our way of living through innovation and cooperation. We are looking for talented individuals who, in addition to their passion for their own role, also have strong team spirit and want to contribute to supporting a meaningful corporate mission.