Is it possible to color the fontawesome icon colors?

All you need is to set color: yellow. Because the icons are a font, they will take whatever colour you'd set to any other font (text) in the same way.

if you want to fill the whole star as yellow, try icon-star instead of icon-star-empty

you can try this

-webkit-text-stroke-width: 1px;
-webkit-text-stroke-color: orange;

to add stroke (outline) to the font itself. I hope thats what you are looking for.

and to fill it just use the normal

color: yellow;

Font-awesome comes with a number of both outlined and filled icons. The star is one of them.

There are four different star icon classes that you can use:

class="icon-star"
class="icon-star-empty"
class="icon-star-half"
class="icon-star-half-empty"

If you're a glass-half-full type of person, you can also use the alias for 'icon-star-half-empty':

class="icon-star-half-full"

You can colour the font-awesome icons and use different effects to achieve what you're looking for:

<i class="icon-star-empty icon-large icon-a"></i><br><br>
<i class="icon-star-empty icon-large icon-b"></i><br><br>
<i class="icon-star icon-large icon-c"></i> or <i class="icon-star icon-large icon-d"></i>

Where the following CSS is used (rather than using inline styles):

.icon-a {
    color: #888;
} 
.icon-b {
    color: orange;
}
.icon-c {
    color: yellow;
}
.icon-d {
    color: yellow;
    -webkit-text-stroke-width: 1px;
    -webkit-text-stroke-color: orange;
}

You can also substitute/set the size too, if you don't want to use icon-large.

The above code outputs the following:

but I've put the above code and a few more options in a JSFiddle, which you can look at here.

It's also possible to use css-transitions that provides a way to animate changes to CSS properties instead of having the changes take effect instantly and or in combination with javascript.