Azure: Application logs are switched off. You can turn them on using the 'Diagnostic logs' settings

I figured it out. Within portal.azure.com:

Microsoft Azure > (My App) > Settings > Logs

There are all the settings for enabling the logs.

Its 3 years later (July 2018) and things seem to be rearranged.

From https://docs.microsoft.com/en-us/azure/app-service/web-sites-enable-diagnostic-log.

Azure > (My App) > Settings / Monitoring (down LH Side) > Diagnostic Logs to turn on.