Kafka + Zookeeper: Connection to node -1 could not be established. Broker may not be available
UPD: if you are running in single-node mode:
I have seen this message in spark console log while trying to deploy application. Solved by changing this parameter in server.properties:
listeners=PLAINTEXT://myhostname:9092
to
listeners=PLAINTEXT://localhost:9092
make sure that you have java process listening on 9092 with netstat -lptu
Change:
#listeners=PLAINTEXT://:9092`
in server.properties to:
listeners=PLAINTEXT://localhost:9092
Note: You also need to uncomment this statement aka remove the # symbol.
I found the error. Observing zookeeper logs when the server started I noticed:
server.1=mylocal-0.:2888:3888
with a dot (.) after the name of the host.
The script that produces the zookeeper's config is from https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zkGenConfig.sh
Looking inside I see that DOMAIN is not filled:
HOST=`hostname -s`
DOMAIN=`hostname -d`
function print_servers() {
for (( i=1; i<=$ZK_REPLICAS; i++ ))
do
echo "server.$i=$NAME-$((i-1)).$DOMAIN:$ZK_SERVER_PORT:$ZK_ELECTION_PORT"
done
}
For my case (localhost) I don't need domain, so I removed that variable.
Now zookeeper and kafka communicate with no errors.