ksqlDB Terraform Provider, API, and SDK
Embedded Files
Client: Confluent
Client: Confluent
Year: 2022
Role: Software Engineer
Duration: 4 Months
Relevant links: Github, API documentation, SDK
Summary: I built the first public ksqlDB API, SDK, and Terraform provider. These were all open-source which is why I can share this here.
Problem: ksqlDB is underperforming in growth, user conversion, and market penetration.
In 2022, we noticed stagnation in growth and user conversion for ksqlDB. Although numerous issues with the product were a significant factor, we also suspected that the customer journey was contributing to the problem.ย
To illustrate this, let's examine the typical user journey for someone at Confluent:
As we can see, we can use a public API, Terraform, and SDK until we reach ksqlDB. At this step, we then have to switch to the UI or CLI for all operations. Yuck!ย
Neither operators nor developers want to use this kind of manual operation for several reasons: it causes operational disruptions, can't be integrated with existing DevOps tools and pipelines, and imposes security risks, among other issues.
After talking to many users and conducting some analysis (which I unfortunately cannot share here), we decided to build all these features, achieving great success in our internal growth metrics.
Page updated
Google Sites
Report abuse