Is there a name for categories whose objects are sets?

Essentially, what you have in mind is a concrete category - a category which can be mapped to Set by a faithful functor.

Often categories whose objects "having underlying sets", and so that the morphisms have "underlying set-maps", are called "concrete categories".

Category theory doesn't really care what its objects are per se. So it's not common for terminology in the field to distinguish categories based on what the objects are. You can ask about categories who behave somewhat like the category of sets (or a subcategory thereof), but that won't tell you what the objects truly are.