Where is the first available house in Skyrim?
Following the main quest a bit further, I stumbled upon an opportunity to buy a house in Whiterun. You have to complete the Dragonstone quest (Bleak Falls Barrow), then you'll be granted the right to buy a house in the city.
To buy the house you have to speak to Proventus Avenicci, he's near the Jarl in the keep. Unfortunately the house is pretty expensive for a starting character, it costs 5000 gold.
The house itself is called Breezehome, it is the second house on the right when you enter Whiterun.
The first house available in the game is in White run. It's a great house to have, cheapest and actually probably the most useful.
White run is by far the best town for selling, crafting, etc, in other words a place to quickly pop over to in between dungeons, quests, or anytime you get a full load of loot.
The house is literally like 15 steps from the fast travel location. And next door on either side are the blacksmith and general goods store, as well as the potion shop.
This blacksmith (war maidens) is fully stocked with smelter, tanning rack, workbench, grindstone, and of course the forge. Shopkeeper restocks with more than a thousand gold each day.
It's perfect, you travel in, stop at shops very quickly to offload loot and your house for anything you wish to keep. Then whenever you need to you can do all your crafting very conveniently.
Also travelling to dragons reach keep allows you to conveniently use an enchanter, and the wizard there is stocked with all sorts of spells and soul gems as well.
I'm pretty far in the game, having completed many major quest lines and visited all the cities, and I can say that this house is by far the most useful.
That being said, it's nothing to look at and for aesthetic reasons it's probably the worst! But as a storage location it's the best.
Although I do wish there were more chests to store things in, to keep them sorted better.
If you're referring to the house in Whiterun, it's called Breezehome, and it's right next to the smithy when you enter town. (Zoning in from the main Skyrim map, it's the second building on the right.)