Finding a center and radius that minimizes total distance

You can use BoundingRegion with "MinBall" as the form spec:

SeedRandom[1];
pts = RandomReal[10, {100, 10}];

BoundingRegion[pts, "MinBall"]

Ball[{5.03195, 4.49308, 5.1499, 4.36746, 5.62561, 5.27036, 5.95754, 4.78651, 4.46375, 5.04961}, 10.8419]

kglr has provided an answer if you want the points to be enclosed within the sphere. However, that was not specified in your question, so I propose a direct approach that minimizes the distance between the points and the surface of a sphere, as you mentioned.

Note that this could result in a large sphere whose surface is locally "almost flat" where your points are, thereby minimizing the distance between that locally-almost-planar-surface and the points.

SeedRandom[20191231]
pts = RandomReal[{3, 5}, {20, 3}];

ClearAll[distfun]
distfun[centerCoords_?(VectorQ[#, NumericQ] &), radius_?NumericQ, points_] := 
 Module[{rdf},
   rdf = RegionDistance@Sphere[centerCoords, radius];
   Total[rdf@points]
 ]

min = NMinimize[{distfun[Array[x, 3], r, pts], r > 0}, Flatten@{Array[x, 3], r}]

Graphics3D[{
    Red, PointSize[0.02], Point[pts],
    White, Opacity[0.5], Sphere[Array[x, 3], r] /. Last@min
  }, Lighting -> "Neutral"
]

Here, as a supplement, my simple straightforward approach, which doesn't need the Region- functionality

SeedRandom[20191231]
pts = RandomReal[{3, 5}, {20, 3}];

opt = NMinimize[{Total@Map[ (Norm[(# - {x1, x2, x3})] - r )^2 &, pts], r > 0}, {r,x1,x2, x3}]
(*{25.4341, {r -> 1.91772, x1 -> 3.9304, x2 -> 3.94589, x3 -> 3.97011}}*)

Graphics3D[{Opacity[.1] , Sphere[{x1, x2, x3}, r], Opacity[1], Red,Point[{x1, x2, x3}], Blue, Point[pts]} /. opt[[2]], Lighting -> "Neutral"]