How can I determine the IP address of an Azure hosted WebApp

Well, that would be one way, but PowerShell would be better.

Get-AzureRmWebApp -Name $WebAppName$
$webapp.outboundipaddresses

for me it returns a bunch of IP addresses

The quickest way would be to login to the Azure portal and select your web app from the resources menu. Once you have the blade open for your web application there are two types of IP addresses. Inbound and outbound. For outbound IP, click properties from the resources menu. This will display a list of all possible outbound IP addresses.

For inbound IP address, click on Custom Domains from the properties menu and your external IP will appear there.