Lead Data Engineer

Join the EPEX SPOT Team: Innovate, Collaborate, Thrive

Job Summary:

As a Lead Data Engineer, you will play a pivotal role in designing, developing, and evolving EPEX SPOT's cloud-native Data Platform on Google Cloud Platform (GCP). You will combine hands‑on engineering expertise with technical leadership, helping deliver trusted, scalable, and AI‑ready data products that support business operations, analytics, and innovation initiatives.

Acting as a technical leader within the Data Platform team, you will coach and mentor Data Engineers, drive engineering excellence, and contribute to the adoption of modern data platform capabilities such as BigQuery, Dataform, Dataplex, Dataflow, BigLake, and Vertex AI.

Working in a highly collaborative Agile environment, you will partner closely with Product Owners, Architects, Data Analysts, Data Scientists, Security teams, and other platform teams to deliver business value while ensuring platform reliability, governance, and scalability.

Your scope will include:

• Enterprise Data Platform development and operations
• Data ingestion, processing, and analytics enablement
• Data Governance and Data Product implementation
• AI and Machine Learning enablement
• Coaching and technical leadership within the Data Engineering team
• Cross-team collaboration and alignment in an Agile environment

Key Responsibilities:

• Drive the design, development, deployment, and continuous improvement of enterprise‑scale data solutions by:
• Designing and implementing scalable data pipelines and data products
• Building robust batch and streaming data processing solutions
• Ensuring platform reliability, performance, security, and cost efficiency
• Supporting architecture decisions and technical roadmap execution
• Acting as the technical escalation point for complex engineering challenges

2. Coach and Develop Data Engineers

• Promote engineering excellence across the Data Platform team by:
• Coaching and mentoring Data Engineers in technical and professional growth
• Conducting code reviews and design reviews
• Encouraging knowledge sharing and continuous learning
• Supporting onboarding and development of new team members
• Driving adoption of engineering standards and best practices

3. Enable Data Governance and Self‑Service Data Products

• Support the delivery of trusted and governed data assets by:
• Implementing governance capabilities using Google Dataplex
• Supporting metadata management, lineage, cataloguing, and data quality initiatives
• Ensuring compliance with security, privacy, and regulatory requirements
• Enabling self‑service analytics through reusable and governed data products

4. Foster Agile Collaboration and Platform Alignment

• Collaborate effectively within Agile product teams and across the technology landscape:
• Promoting DevOps and DataOps practices
• Supporting a culture of transparency, ownership, and continuous improvement

5. Contribute to EPEX SPOT's AI and Data innovation ambitions by:

• Building data foundations for analytics and AI use cases
• Supporting the integration of Vertex AI and related AI services
• Collaborating with Data Scientists to operationalize Machine Learning solutions

Key Requirements

Hard Skills

• Strong hands‑on experience with:
• BigQuery
• Dataform
• Data lake and Lakehouse implementation on GCP: Cloud Storage, external tables, BigLake, and the trade‑offs between external and materialized data
• IAM and Security Services
• Cloud Monitoring and observability
• Dataflow
• ELT/ETL architecture and implementation
• API‑based data integration
• Data Contract implementation in the pipeline (assertions, quality checks, validation at build time)
• Real‑time and streaming architecture
• Metadata management and data lineage
• Event‑driven architectures
• AI‑assisted engineering: proficient with GCP AI assistance (Gemini in BigQuery, Gemini Code Assist) and able to coach the data engineers on using it productively and safely
• Python
• SQL
• Infrastructure as Code (Terraform preferred)
• Git and version control
• DevOps and DataOps practices
• Platform monitoring and observability
• Performance optimization and cost management

Agile Delivery

• Proven experience working in Agile environments

Soft Skills

• Strong technical leader who drives results through expertise, influence, and a passion for developing engineering talent.
• Excellent communicator and collaborator, ab