Other than appearance, what is the difference between regular Pokemon and "shiny" Pokemon?

Bulbapedia has an extensive article on it, but it boils down to, in Generation III and on, there is no difference in stats between normal Pokemon and shiny ones.

Generation II (Gold, Silver, and Crystal) is the exception because shininess is determined based on the stats of the Pokemon. Namely:

If a Pokémon's Speed, Defense, and Special IVs are 10, and its Attack IV is 2, 3, 6, 7, 10, 11, 14 or 15, it will be Shiny

But, from Generation III and onward, it is determined based on a secret ID and a trainer ID, so it has nothing at all to do with the stats and will be just like every other Pokemon.

Something fun to note is that if the Male/Female ratio is 7:1, in Generation II, you cannot have a female shiny Pokemon because of the way it is calculated.

The only difference between a shiny and non-shiny Pokemon in Generation III and later is their appearance. It merely exists as a way to give players something more to collect in the game, and has no affect on game play [source: bulbapedia]

In Generation II whether or not a Pokemon is shiny is determined by it's Individual Values (IVs). Specifically, the following must be true:

If a Pokémon's Speed, Defense, and Special IVs are 10, and its Attack IV is 2, 3, 6, 7, 10, 11, 14 or 15, it will be Shiny.

Due to this, any shiny Pokemon in Generation II will have slightly above average speed/defense/special stats compared to non-shiny Pokemeon. There's no other non-visual differences between shiny and non-shiny pokemon in Generation II.