Change the inner color of markers

In Version 10 you can use the PlotTheme "OpenMarkersThick":

data = Table[{x, x^k}, {k, 1, 4}, {x, 0, 1, 0.1}]

ListLinePlot[data, PlotTheme -> {"OpenMarkersThick", "LargeLabels"}, 
 PlotLegends -> {x, x^2, x^3, x^4}] 

\[FilledSquare] is a font glyph and you cannot color parts of it.

I believe you need to draw your markers with Graphics primitives. For example:

square[in_, out_: Black, size_: 12] :=
 Graphics[{in, EdgeForm[{AbsoluteThickness[2], out}], Rectangle[]},
   PlotRangePadding -> 0,
   ImageSize -> size]

data = {{1, 2, 3, 5, 8}, {2, 3, 6, 9, 10}, {4, 5, 7, 10, 12}};

ListLinePlot[data, 
 PlotMarkers -> square /@ {Red, Green, Blue}
]

ListLinePlot[data,
  PlotMarkers -> square @@@ {{Yellow, Red}, {White, Blue}, {Black, Pink}}
]

---

Update: proof that this method can easily be used for markers of arbitrary shape.

Generic marker function:

marker[prim_, opts___][in_: White, out_: Black, size_: 13] :=
 Graphics[{in, EdgeForm[{AbsoluteThickness[2], Opacity[1], out}], prim},
  opts, ImageSize -> size]

Shapes:

square  = marker @ Rectangle[];
circle  = marker @ Disk[];
diamond = marker @ Polygon[{{0, 1}, {1, 2}, {2, 1}, {1, 0}}];
triangle =
 marker[
   Polygon[{{1, 0}, {0, Sqrt[3]}, {-1, 0}}], 
   AlignmentPoint -> {0, 1/Sqrt[3]}
 ];

Plot:

SeedRandom[12]

ListLinePlot[
  Accumulate /@ RandomReal[3, {4, 10}] {1, 2, 3, 4}, 
  PlotMarkers ->
    {square[], circle[Yellow], diamond[Brown, Pink], triangle[Magenta, Purple]},
  PlotLegends -> Automatic
]

You could build your own PlotMarkers

ngon[p_, q_] := 
 Polygon[Table[{Cos[2 Pi k q/p], Sin[2 Pi k q/p]}, {k, p}]]

g1 = Graphics[{EdgeForm[Black], White, Disk[{0, 0}, 1]}];
g2 = Graphics[{EdgeForm[Black], White, Rectangle[{1, 1}]}];
g3 = Graphics[{EdgeForm[Black], White, ngon[4, 1]}];
g4 = Graphics[{EdgeForm[Black], White, Polygon[{{1, 0}, {0, Sqrt[3]}, {-1, 0}}]}];

ListLinePlot[Table[n^(1/p), {p, 4}, {n, 10}],
Filling -> Axis,
PlotLegends -> Automatic,
PlotMarkers -> Table[{s, 0.05}, {s, {g1, g2, g3, g4}}]]