Data Engineer

Climate Talent Specialist | Helping Climate Founders Build Exceptional Teams | Angel Investor in Female-Founded Climate Start-Ups | Podcast Host Location: Paris or Toulouse Our mission: Only 1% of freshwater on earth is usable for humanity's needs and climate change is exacerbating both water scarcity and water hazards – such as floods, droughts, and water pollution. BWI is an impact startup on a mission to use data and machine learning to provide governments and companies with river flow forecast services in order to help better adapt to the impact of climate change on water management and availability. BWI subscribers benefit from near real-time hydrological forecasts based on dozens of data sources – such as space sensors, weather radars, and other ground sensors. The role: After recently securing a major new contract, we’re now looking for 3 x Big Data Engineers to join our engineering team in either our Paris or Toulouse offices. Working as part of either our satellite or water simulation team, you will be responsible for building and operating data ingestion and processing pipelines for terabytes of weather and hydrological data. You’ll be focused on ensuring scalable, reproducible and production-grade delivery of model inputs and outputs. Responsibilities: Implement ingestion (batch/streaming), ELT/ETL steps, and data publishing workflows. Handle scientific formats (netCDF, GRIB2) and columnar storage (Parquet); optimize I/O and algorithms. Design storage with eventual‑consistency patterns (atomic publishes, manifests, versioned paths) and a metadata catalog. Partition and parallelize workloads for distributed compute; compact small files and tune for cost/performance. Build and run containerized services and orchestrated workflows; ensure observability, retries, idempotency, and runbooks. Collaborate with scientists to define data models and validation rules. Top paradigms & architectural patterns required: ELT-first with ETL where needed; streaming/micro-batch for low-latency sources Idempotent, restartable workflows with orchestration Versioned datasets, atomic publish patterns, and catalogue as source of truth Observability-driven ops and infra-as-code Python (xarray, netCDF4, pyarrow), PySpark or Dask AWS (S3, EKS, EC2) or equivalent cloud; Terraform for IaC Must-haves: 3+ years relevant experience building/operating large data pipelines Strong software engineering in Python with testing and CI/CD Practical experience with partitioning, and parallelism Computer science education Understanding of architectural patterns in storing & processing large volumes of data Humility & eagerness to accept both: full hands-on coding position or technical leadership depending on circumstances Clear communication, teamwork, and analytical problem-solving Nice-to-haves: Hydrology, meteorology, remote-sensing or space-ground-segment experience Understanding of low-level subject (memory management, HTTP, S3 implementation) Willingness to develop managerial skills Our culture: Dynamic, fast paced start-up, working on an important problem for humanity. Multicultural Team - With team members from various countries like France, Poland, India, Lebanon and Morocco, you will be exposed to a multicultural work environment. Offices in Central Paris and Toulouse Diverse Domains - work across software engineering, hydrology, climate and satellite. Hybrid working w/ 4 days in office and 1 day remote. Seniority level Mid-Senior level Employment type Full-time Job function Engineering and Information Technology Industries: Climate Data and Analytics and Software Development #J-18808-Ljbffr