How valuable is it to build next to a river?

A river in the capital makes a very big difference in the early economy. If you like to have a maritime city state ally very early for growth, or something similarily expensive, a river start is extremely beneficial.

Gold is very useful in Civilization 5, and a river gives you one gold/turn for each hex adjacent to the river. That is quite a lot of gold for a city that can work those tiles.

If you build near rivers you can also accelerate your growth significantly by slingshooting to Civil service. Then farms near fresh water produce 1 food more, which is a huge bonus. The technology which gives more food for farms not near a river is Fertilizer, which comes significantly later.

Building near rivers is very powerful, I think you're doing it just right. Of coure, sometimes you just need that strategic or luxury resource that is not near a river, so you have to build there first. But prioritising cities near rivers is a good strategy.

I want to stress out a difference between building a city near a river and close to a river. River tiles are great, as they give you additional gold and production. That is especially useful when a golden age kicks in, as even a tile with one production/gold yield will give you an additional one, thus increasing income from that tile two times.

The thing is your city doesn't necessarily have to be next to a river to work those river tiles. It can be a tile or even two away and still have access to them.

The only reason you would want your city to be next to a river are then available buildings, which are: water mill, garden and hydro plant. Water mill (+2 food) is a very low priority building, especially with maritime city states and taking you would still have an option of granary. Garden (+25% Great people) is useless unless you specialize that city specifically for great people generation. Hydro plant (+25% production) is not that necessary, as you would still have options of other type power plants.

So basically when settling try to get access to river tiles, but not necessarily settle right next to a river if otherwise you could get access to some resources or other valuable tiles.

Rivers give increased commerce to every square they're touching. It basically ammounts to free money -- you're already going to be working the hills / grassland / plains for food or production, but this way you get commerce without having to build trading posts everywhere.

Additionally, watermill and hydroelectric plants can only be built in cities with rivers (and maybe other buildings as well? Gardens?).

If you value those buildings (which depend solely on city position) than shoot for the river every time.