Removing a default constraint after adding a new column in code first

Personally I can see nothing wrong with the first approach. Yes, you have to create a default constraint to add a non-nullable column to a non-empty dataset. And yes, you have to delete it afterwards if you need to make sure the new column is always added explicitly in the future, as per your requirement.

And even if you still have issues with this approach, the problem is, there's most likely no other alternative apart from your second approach. And the cost of updating a possibly large table with a default value would seem to me greater than the cost of creating and immediate dropping a default constraint.

I might consider a slight modification, though: I might create the constraint in SQL to avoid the fuss of looking up the default name assigned by the engine, something like this:

Sql("ALTER TABLE tablename
  ADD columnname type NOT NULL
  CONSTRAINT DF_tablename_columnname DEFAULT defaultvalue");

(Could be rewritten to use a helper function.)

Dropping a constraint would then be as trivial as executing a single ALTER TABLE statement:

Sql("ALTER TABLE tablename
  DROP CONSTRAINT DF_tablename_columnname");

(Again, a helper function could easily be used here.)

But that all might be a T-SQL developer in me speaking louder than a C# one. So you might very well go with the code like the example you've posted.