Convexity of matrix trace functions $A\mapsto \operatorname{Tr} (Bf(A))$

After reading user1551's answer, I have subsequently realized that my question can be answered in the negative as follows.

For $n\times n$ matrices, we write $A\geq 0$ if $A$ is positive semidefinite, and $A\geq B$ if $A-B\geq 0$. We say a function $f:\mathbb{R}\rightarrow\mathbb{R}$ is matrix convex if, for all integers $n$, it holds that $$ f(tA+(1-t)B)\leq tf(A)+(1-t)f(B) $$ for all $n\times n$ hermitian matrices $A,B$ and all $t\in(0,1)$, where the inequality is respect to the ordering of positive semidefinite matrices.

Theorem Let $f:\mathbb{R}\rightarrow\mathbb{R}$ be a function. The following are equivalent:

1. $f$ is matrix convex
2. For all integers $n$ and all $n\times n$ matrices $B\geq 0$, the function $A\mapsto \operatorname{Tr}(Bf(A))$ is convex.

Proof. If $f$ is matrix convex then the result clearly follows. So suppose that $f$ is not matrix convex. Then there exists an $n$ and $n\times n$ hermitian matrices $A$ and $B$ and $t\in(0,1)$ such that $$ f(tA+(1-t)B)\not\leq tf(A)+(1-t)f(B). $$ Thus there is a vector $v\in\mathbb{C}^n$ such that $$ v^*\bigl(f(tA+(1-t)B)\bigr)v >tv^*f(A)v+(1-t)v^*f(B)v $$ which is equivalent to $$ \operatorname{Tr}\bigl(vv^*f(tA+(1-t)B)\bigr) >t \operatorname{Tr}\bigl(vv^*f(A)\bigr) + (1-t)\operatorname{Tr}\bigl(vv^*f(B)\bigr), $$ hence the function $A\mapsto \operatorname{Tr}(vv^*f(A))$ is not convex.

The answer is no. Since every positive semidefinite matrix is a nonnegatively weighted sum of orthogonal projections, we may assume that $B$ is an orthogonal projection. With this simplification, it is not hard to see that the inequality in question must hold if $f$ is operator convex.

So, to find a counterexample to the inequality, we should look for a function $f$ that is convex but not operator convex. Take $f(x)=x^4$. Let $n=2$ and $B=\operatorname{diag}(1,0)$, so that $\operatorname{tr}(Bf(A))$ is just the $(1,1)$-th entry of $f(A)$. We use the following Octave/Matlab script to see if the first entry of $f(A)$ is mid-point convex:

[U,S,V]=svd(rand(2,2)); % this generates two orthogonal matrices U,V
D1=diag(rand(2,1)); % random positive diagonal matrix
D2=diag(rand(2,1)); % random positive diagonal matrix
A1=U*D1*U';      % random positive definite matrix
A2=V*D2*V';      % random positive definite matrix
F1=A1^4;         % f(A1)
F2=A2^4;         % f(A2)
A3=(A1+A2)./2;   % mean of A1 and A2
F3=A3^4;         % f(A3)
% If the (1,1)-th entry of f(A) is convex, we will have d>=0:
d = (F1(1,1)+F2(1,1))/2 - F3(1,1)

The answer turns out to be "no":

d = -0.0029889

A1 =
   0.44099  -0.21308
  -0.21308   0.91124

A2 =
   0.39698  -0.27903
  -0.27903   0.50098

A3 =
   0.41898  -0.24606
  -0.24606   0.70611

F1 =
   0.14056  -0.32144
  -0.32144   0.84997

F2 =
   0.11822  -0.14139
  -0.14139   0.17092

F3 =
   0.13238  -0.22015
  -0.22015   0.38927