How can I get rid of or purify the Crimson?

The same methods you use to break through Corruption should work just as well in the Crimson. Bear in mind, however, that the Crimson's tunnels are oriented in diagonals, like long veins, so when getting deep into it, you should take this into consideration.

You could start your project by blocking it off with a wall. Make it 6 tiles long, since I've seen varying reports on how far Crimson can reach, and make it deep enough to block any Crimson grass that might reach beneath the earth. 10 tiles down will probably be plenty.

Purifcation powder should work well for breaking through Crimson blocks, but Dynamite and other explosions will also work (not grenades) if you haven't got the Dryad yet.

Also keep in mind that if you're going to be breaking the special 'orbs' (Crimson Hearts now I think) in the Crimson, you'll be dealing with Brain of Cthulhu instead of the Eater of Worlds. So be prepared for a new boss.

And, as 3ventic says, a pickaxe of 65% power or greater (nightmare/deathbringer pickaxe or better) can cut through Crimson.

Read more here.

Although this is not the most accessible way to do it, the easiest way is to use the Clentaminator with Green Solution.

The more accessible ways are to use the Purification Powder from a Dryad, bomb the Crimson with anything that breaks blocks when it explodes or mine it with a pickaxe (power 65% or higher, or in terms of tools: nightmare/deathbringer pickaxe or better).