Co/fibrant replacements via coend calculus

Addressing this sort of question was one of the main goals of math/0610194. The best answer I was able to give is that if $B$ has a suitable model structure with respect to which $F$ is objectwise fibrant (resp. cofibrant), then $\overline{F}$ (resp. $\underline{F}$) belongs to a "right (resp. left) deformation retract" (an abstraction of the notion of fibrant (resp. cofibrant) replacement) that is suitable for constructing derived functors of functors such as homotopy limits (resp. colimits) and also the homotopy category of the functor category (which therefore involves homotopy coherent transformations).

A closely related perspective that you may also be interested in can be found in this paper by Gambino: the colimit of $\underline{F}$ is equivalently the colimit of $F$ weighted by a projective-cofibrant replacement of the terminal weight.

This will not answer your question but is to mention that we only defined (and studied in depth) the functors $\underline{F}$ and $\overline{F}$ where the codomain $\mathbf{B}$ of the functor was a 'locally Kan' simplicially enriched category and thus fibrant in the usual model category structure on $\mathcal{S}$-cat. (This eliminates the awkwardness of the example that Dimitri mentions.) This means that the unit and counit maps $\eta_F$ etc, are levelwise homotopy equivalences not just weak equivalences. (N.B. Our philosophy was to show such things as directly as possible so there is no use of spectral sequences (at least explicitly in our contribution), for instance, by showing some map is a weak equivalence, followed by invocation of the Kan-ness of the objects involved so get it is a 'real' homotopy equivalence. This makes some of the arguments more constructive' but at the cost of being heavier on explicit, almost combinatorial or geometric, computation with induction up skeleta etc. as a tool.)

I suspect that some of the ideas explored in the article:

P. J. Ehlers and T. Porter, Ordinal subdivision and special pasting in quasicategories, Advances in Mathematics, 217 (2007), No 2. 489 - 518

may be useful, but do not, at the moment, see how to prove what you suggest to be the case. It is worth noting that Dan Dugger's construction, mentioned by Dimitri, is very similar to that given by `$F$ goes to $\overline{F}$', so adapting his proof may give the result.