Why is "It works!" showing up at my http://127.0.0.1 (localhost)?

This is default behavior in OSX when you enable the web server in the properties. You can find it in this control pane:

Newer versions of macOS might not have the option displayed in the Sharing panel, as is shown in Bart De Vos's answer. In that case, running this command from Terminal should stop it:

sudo apachectl stop

Based on my own experience, it shouldn't start up automatically after that, even after a system restart.

Or, if you want to start it manually:

sudo apachectl start