Using Consul with Azure App Services

it looks like we will not be able to use consul with Azure App Service (aka Web Apps) at all.

Here is what I've tried.

1. Naive approach - consul as WebJob

Due to networking restrictions attempt to connect to ANY localhost ports what did not spawn with processes that belong to App Service (Web App) itself will end up with the following exception.

An attempt was made to access a socket in a way forbidden by its access permissions 127.0.0.1:8500.

Reference from documentation:

https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox#networking-restrictionsconsiderations

The only way an application can be accessed via the internet is through the already-exposed HTTP (80) and HTTPS (443) TCP ports; applications may not listen on other ports for packets arriving from the internet. However, applications may create a socket which can listen for connections from within the sandbox. For example, two processes within the same app may communicate with one another via TCP sockets; connection attempts incoming from outside the sandbox, albeit they be on the same machine, will fail. See the next topic for additional detail.

Here is an interesting piece:

Connection attempts to local addresses (e.g. localhost, 127.0.0.1) and the machine's own IP will fail, except if another process in the same sandbox has created a listening socket on the destination port.

2. Consul spawned from App Service itself

I've copied consul to Web App (as build output) and added the following lines to application startup code:

var consul = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "bin/consul/consul.exe");

Process.Start(consul, "agent --data-dir=../../data");
Process.Start(consul, "join my-cluster-dns.name");

... and it joined the cluster and I even was able to connect to consul via 127.0.0.1:8500 from the App Service (Web App) itself.

However, it is still useless setup as Consul agent MUST be reachable from server, so all I was able to see from cluster standpoint is a dead node with failing "serf" health-check. Again, according to the documentation there is no work around this: "The only way an application can be accessed via the internet is through the already-exposed HTTP (80) and HTTPS (443) TCP ports".

https://www.consul.io/docs/agent/basics.html

Not all Consul agents in a cluster have to use the same port, but this address MUST be reachable by all other nodes.

Summary

All-in-all, probably there is no way to properly host/use Consul with Azure App Services.