Interpolating Maps - Statistical Learning Techniques vs Spatial Statistics Techniques

In your case, where you have a multivariate problem, ordinary Kriging is quite inappropriate. I find your interpretation of this as an "interpolation" problem is a bit off base as well. This is an estimation problem and more suited for Machine Learning or spatial regression, not geostatistics. The grey area are Splines. This can be a univariate interpolation method but can also be used in a semiparametric form to fit a multivariate nonlinear model and estimate a surface.

I will just say now, GWR is off the table. There are considerable problems with this method and it is really only suitable for exploratory analysis of nonstationarity. There are several papers that demonstrate the instability of GWR via simulations.

Regression Kriging is a promising method but it is very important that you read the primary literature to understand what it doing. If you distill regression Kriging to its basic form your are, in essence, Kriging the residuals of a linear model. The entire point is to violate iid assumptions so you have structure in your residual error. With complex high dimensional multivariate space this estimate may be difficult to interpret and plausibly, nonsense. It is really most suited for nicely parsimonious models.

Spatial regression and mixed effects methods will require you to fit a model. Since your problem is not inferential in nature this seem like overkill and is a steep learning curve.

As far as machine learning approaches, be aware most methods do not explicitly account for spatial process without help. There are proposed methods such as the inclusion of naive spatial process via [X,Y] coordinates, including a fit Nth order polynomial of [X,Y] or the specification a distance matrix as covariates.

Because you want to include both continuous and categorical covariates you are limited in methods. It sounds like your use of machine learning methods are the best suited to your problem.

It is critical that you clearly specify your problem and then select a statistical method to fit the problem. The approach throwing methods against the wall to see what sticks is quite unsatisfactory. It would be beneficial for you to review the literature in order to understand methods, model assumptions and implications of utilizing a given method. There is considerable information online that can be discovered with a simple Google Scholar search.