Created user can access all databases in PostgreSQL without any grants

At the SQL level, every user can indeed connect to a newly created database, until the following SQL command is issued:

REVOKE connect ON DATABASE database_name FROM PUBLIC;

Once done, each user or role that should be able to connect has to be granted explicitly the connect privilege:

GRANT connect ON DATABASE database_name TO rolename;

Edit: In a multi-tenant scenario, more than just the connect privilege would be removed. For multi-tenancy tips and best practices, you may want to read on the postgresql public wiki: Shared Database Hosting and Managing rights in PostgreSQL.

PUBLIC has access to the database by default, but it can't access the data. You can REVOKE the PUBLIC:

REVOKE CONNECT ON DATABASE your_database FROM PUBLIC;

If you want this setting for all future databases, revoke CONNECT on the template1 database (default template database for creating a new database):

REVOKE CONNECT ON DATABASE template1 FROM PUBLIC;

Besides revoking connection privileges from PUBLIC by default, and granting them as specifically desired, the other level at which you can control access is through the pg_hba.conf file.

You can find where the file is stored with:

SHOW hba_file;

If you choose to use this mechanism, there are embedded comments which may be enough to get you started. The docs are here:

http://www.postgresql.org/docs/current/interactive/auth-pg-hba-conf.html