KAFKA/Confluent Expert

# KAFKA/Confluent Expert
* Home
* KAFKA/Confluent Expert
* October 28, 2026
* 0 Comments
* By Roberte OkoyaContext: Experienced Profile (profile with 3 to 7 years of experience in the same service category). As part of a project to implement a data exchange platform based on the open-source Apache Kafka technology, using the Confluent distribution, we are looking for a Kafka expert to strengthen the project team.
MISSIONS To strengthen the project team, we are seeking Kafka technical expertise for the implementation of a new Kafka Confluent platform. The objective is to establish an enterprise Kafka platform to support various use cases for data exchange, replication, and integration. The responsibilities will cover the different phases of the project that have been identified: Phase 1 – Implementation of the pilot platform: This first step consists of setting up a pilot platform to feed an initial use case of the Data Platform type. The data made available to this platform will be produced by the various technical platforms of the information system, namely: Management Information System (PostgreSQL), Hadoop (Cloudera), SID-RC (Datastage) + Teradata. Among the identified tasks to be performed: – Installation and configuration of the Kafka/Confluent cluster. – Definition and implementation of operating standards. – Implementation of administration and operational tools. – Implementation of security and access management mechanisms, integrated into the IT ecosystem; – Optimization of platform performance; – Construction and industrialization of deliverable manufacturing and integration processes; – Implementation of integration and deployment workflows. – Installation and configuration of the \"standard\" and \"Confluent-specific\" Kafka producer and consumer connectors required for the initial use cases: PostgreSQL, Hadoop (HDFS), DataStage, Teradata, and other connectors necessary for the pilot scope. Particular attention will be paid to the industrialization of the connectors, including: – Declaration of new producers and consumers, – Configuration management, – Initialization and refresh of consumer data, – Automated deployment of subscriptions, – Consideration of data schema changes by Kafka Topics, – Automation of recurring operations. It will also be necessary to implement all the operating procedures for the platform: – Shutdown and restart procedures; – Management of technical constraints; – Supervision and monitoring; – Implementation of dashboards; – Operational documentation. – Connectors to be implementedPhase 2: Platform Extension: Following validation and delivery of the pilot platform, the project will enter an extension phase to build a more robust target platform capable of supporting new use cases. The main use cases considered are: – Use Case 1: Data Platform Extension + Extension of the initial pilot to all databases and uses of the Data Platform. – Use Case 2: Replacement of the IBM CDC Tool + Migration of flows currently managed by IBM Change Data Capture (CDC) to Kafka. – Use Case 3: Replacement of Batch Extractors + Gradual replacement of batch extraction mechanisms feeding the business intelligence platforms with real-time flows based on Kafka. – Use Case 4: PostgreSQL Database Replication + Implementation of PostgreSQL database replication and synchronization mechanisms via Kafka. Use Case #5: Migration of the WebMethods platform + Study and implementation of the migration of exchanges currently managed by the WebMethods platform to Kafka. Desired Profile: We are looking for a Kafka/Confluent expert capable of participating in all phases of the project and supporting the team on the following topics: – Kafka and Confluent architecture; – Installation and administration of Kafka platforms; – Securing exchanges; – Performance and tuning; – Implementation of Kafka Connect connectors; – Industrialization and automation of deployments; – Monitoring and operation; – Supporting project teams and transferring skills; – Writing architecture documents, standards and best practices, operational documentation, etc. The expert will join the project team to contribute to the design, implementation, and industrialization of the company's future data exchange platform. Expected Deliverables » – Weekly Reporting – Monthly Activity Monitoring Dashboard – Technical Documentation: Standards, principles, architecture document, administration document, operations document, operating procedures – Knowledge Transfer: organizing knowledge transfer sessions and writing training materials. – Request Management: Duly completed incident/request/change tickets – Contributing to service continuity within the team » » –> Technical and Technological Environment « Kafka Core Environment: – Kafka Confluent and Strimzy Distribution – Scripting and development languages: Shell, Python, JSON – Operating sys