Is there any use of taking a pokemon out of his pokeball?

The most obvious benefit is that Pokemon that follow you can find items hidden in bushes. Your Pokemon will have an exclamation point above its head and go to the bush (or other plant). When you talk to it, it will either give you a useless textbox or give you a more or less valuable item.

There are some special cases of Pokemon "following" you:

• You can ride some Pokemon (e.g. Onix). This mainly increases your walk speed (and also hitbox!)

• You can ride some Pokemon in the water (e.g. Gyarados). It will not follow you on land, but as soon as you use Sea Skim you will ride that Pokemon (instead of the surf board with Pikachu). Not sure about speed compared to regular Sea Skim.

• Some Pokemon fly and let you ride on them (Charizard, Dragonite, Aerodactyl). That means, as soon as you are in an open area you will ascend and be able to fly over trainers, wild Pokemon etc. without triggering an encounter. This will only work on Pokemon Road until you beat the Elite Four, at which point you can fly anywhere. You have to dismount by putting the Pokemon back into its Pokeball. Thanks to Kruga for this info.

Taking Pokemon out of their Pokeballs makes them like you, which makes them more effective in battle. Pokemon that like you are more likely to resist or recover from status effects, and to survive an attack (at 1 HP) which otherwise would have made them faint. When this happens in battle, you'll see a heart overlay on the Pokemon, and text such as "[Name] toughs it out, because it didn't want to make you sad."

Your starter (Pikachu/Eevee) can't be "taken out" of its Pokeball because it's already out. Instead, you can build up the "it likes you" factor by playing with it or feeding it (select "play with [nickname]" or "play with [Pikachu/Eevee]" on the pause screen).

(Also, all of the benefits that Ian mentioned in their answer.)