Rounding problems inside InputField

This seems to work for me, is it what you need?

InputField[Dynamic[h2, If[NumericQ[#], h2 = Round[#, 0.001], h2 = h2] &]]

Since the type Number won't allow formatted numbers to be displayed, set the accuracy (or precision) of the displayed value to control the number of digits:

InputField[Dynamic[If[h2 == 0, 0., SetAccuracy[h2, 4]], (h2 = Round[#, 0.001]) &], Number]

FullForm[h2]
(*  0.468`  *)

It does have the disadvantage that numbers like 0.12 will be displayed as 0.120. A more complicated program could be written to handle that, if desired.