Connecting Pod in Minikube to Kafka or any Services Running in Docker Desktop
I'm working on a demo where I need to subscribe my application to Kafka locally in Docker Desktop. I have 3 use cases:
From the Host (my computer) Outside of Docker Network
Next is my application that runs outside of Docker desktop, since it won't resolve the kafka host, it has to use the 2nd entry of the advertised listener. In my case, I had to change the port from 9092 to prevent conflict with other Kafka instance, but for it to work, I had to change the port mapping, so the Kafka container configuration looks like the following:
From Pod Inside Minikube
- Connecting from a different container in Docker desktop, so in the same network as the Kafka container.
- Connecting from the application running on the host, so outside of Docker desktop for debugging purposes.
- Connecting from a pod inside Minikube running in Docker Desktop.
Same Docker Network
On the first case, I actually need to connect AKHQ container to Kafka, my Kafka container env variable for advertised listeners looks like the following:
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092, ... (removed for brevity)
Since AKHQ running in Docker desktop as well, it can use kafka:29092.
ports:
- 19092:19092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:19092, ...
To prevent conflict, the port mapping on the host and container has to be modified, so my application connects using localhost:19092
This one is confusing me, but I found out that kube-dns is installed by default and Minikube provides a convenient hostname. For more details: https://minikube.sigs.k8s.io/docs/handbook/host-access/
host.minikube.internal
So updating my configuration a little, I can connect from the pod using host.minikube.internal:19094
ports:
- 19092:19092
- 19094:19094
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: ...,PLAINTEXT_POD://host.minikube.internal:19094
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: ...,PLAINTEXT_POD:PLAINTEXT
Comments
Post a Comment