BarLegend ticks distorted in MatrixPlot

Update: The function Graphics`ArrayPlotDump`Private`HybridRankingAndNaturalScale performs the mysterious "scaling based on a mixture of relative value and ranking for each matrix element". We construct a piecewise re-scaling function (reScale) using mpReScale and use it to specify the option value for "Ticks":

ClearAll[reScale,  cfMinMax]
reScale[{min_, max_}, {cfmin_, cfmax_}]  := 
  If[# < 0, Rescale[#, {min, 0}, {cfmin, 1/2}], Rescale[#, {0, max}, {1/2, cfmax}]] &;

cfMinMax = MinMax @ Graphics`ArrayPlotDump`Private`HybridRankingAndNaturalScale[
   Union @ SparseArray[#]["NonzeroValues"], 0., {0, 1}, .5] &;

Examples:

data = {{1, 2, 1}, {2, 0, 1}, {0, -5, -1}};
ticks = {-4, -2, 3/2, 2};

cfminmax = cfMinMax[data];
minmax = MinMax @ data;
 
Row[{MatrixPlot[data, ImageSize -> 400, 
   PlotLegends -> BarLegend[Automatic], LabelStyle -> 16, 
   PlotLabel -> "default"], 
  MatrixPlot[data, ImageSize -> 400, 
   PlotLegends -> BarLegend[Automatic, 
     "Ticks" -> (Transpose[{reScale[minmax, cfminmax] /@ #, #}] & @ ticks)], 
   LabelStyle -> 16, 
   PlotLabel -> Row[{"Ticks : ", ticks}]]}, Spacer[10]]

An aside: We can use another undocumented option to specify tick labels

BarLegend[Automatic, "TickLabels" -> ticks, 
    "Ticks" -> (reScale[minmax, cfminmax] /@ ticks)]

to get the picture in the second plot above.

Change data and ticks to

data = Table[Sin[x] Cos[y], {x, 0, 2 Pi, 0.01}, {y, 0, 2 Pi, 0.01}];
ticks = {-0.5, 0, 0.5};

to get

Using the second example in OP:

data = Table[Sin[x] Cos[y] + 0.05 x y, {x, 0, 2 Pi, 0.01}, {y, 0, 2 Pi, 0.01}];
ticks = {-0.2, 0.5, 1.1};

we get

With

data = 1. + Table[Sin[x] Cos[y] + 0.05 x y, {x, 0, 2 Pi, 0.01}, {y, 0, 2 Pi,  0.01}] ;
ticks = {0.1, 0.5, 1.1, 2};

we get

Original answer:

This seems to be related to the special way scaling is done in MatrixPlot as mentioned in MatrixPlot >> Details and Options

Easiest fix is in OP's case is to change the tick specification to Transpose[{Rescale[#, {-1, 1}, {0, 1}], #} &@{-0.5, 0, 0.5}]:

MatrixPlot[data, 
 PlotLegends -> 
  BarLegend[Automatic, 
   "Ticks" -> Transpose[{Rescale[#, {-1, 1}, {0, 1}], #} & @ {-0.5, 0, 0.5}]], 
 LabelStyle -> Large]

As an alternative (more general) work-around use the default color function with re-scaled argument and the option ColorFunctionScaling -> False:

defaultCF = "DefaultColorFunction" /.
    (Method /. Charting`ResolvePlotTheme[Automatic, MatrixPlot])

 Blend[System`PlotThemeDump`$ThemeDefaultMatrix, #1] &

MatrixPlot[data, 
 ColorFunction -> (defaultCF[Rescale[#, {-1, 1}]] &), 
 ColorFunctionScaling -> False, 
 PlotLegends -> BarLegend[Automatic, "Ticks" -> {-0.5, 0, 0.5}], 
 LabelStyle -> Large]

Alternatively, specify the color function in BarLegend:

MatrixPlot[data, 
 PlotLegends -> BarLegend[{defaultCF[Rescale[#, {-1, 1}]] &, {-1, 1}}, 
   ColorFunctionScaling -> False, "Ticks" -> {-0.5, 0, 0.5}], 
 LabelStyle -> Large]

kglr has already resolved the problem, here's just some additional analysis and another possible work-around.

First of all, I don't think this is a bug, becauce by default MatrixPlot knows how to set proper Ticks for BarLegend:

plot = MatrixPlot[data, ImageSize -> 400, PlotLegends -> Automatic]

Looking into the plot, we find that:

plot[[2, 1]] // InputForm
(*
BarLegend[{Blend[System`PlotThemeDump`$ThemeDefaultMatrix, #1] & , 
  {0.2889327547713697, 1.}}, LabelStyle -> {}, 
 LegendLayout -> "Column", LegendMarkerSize -> 400, 
 Ticks -> {{0.39446607623278385, -0.5}, {0.5, 0.}, {0.3100389372190109, 
  -0.9}, {0.626985112913238, 0.5}, {0.753970225826476, 1.}, 
  {0.880955338739714, 1.5}, {0.9825434290703043, 1.9000000000000001}}, 
 "PinningPoint" -> 0.5, "SmoothRange" -> False, 
 Charting`TickSide -> Right, ColorFunctionScaling -> False]
 *)

As we can see, the ticks of BarLegend are set by the undocumented Ticks option, we plot the ticks:

autotick = Cases[plot[[2, 1]], (Ticks -> a_) :> a][[1]] // Sort

ListLinePlot[autotick]

Not hard to notice it's a piecewise line, splitting at {0.5, 0.}. We can further verify this with LinearModelFit:

LinearModelFit[autotick[[#]], {1, x}, x]["RSquared"] & /@ {;; 3, 3 ;;}
(* {1., 1.} *)

So, even if we're not aware of the Graphics`ArrayPlotDump`Private`HybridRankingAndNaturalScale, we can still rescale the ticks in the following manner:

ticks = {-0.2, 0.5, 1.1}

Clear[rescale]
rescale[tick_List, rest__] := rescale[#, rest] & /@ tick
rescale[tick_?Positive, mysteryminmax_, {min_, max_}] := 
 Rescale[tick, {0, max}, {1/2, mysteryminmax[[2]]}]
rescale[tick_, mysteryminmax_, {min_, max_}] := 
 Rescale[tick, {min, 0}, {mysteryminmax[[1]], 1/2}]

plot /. (Ticks -> _) :> (Ticks -> {rescale[ticks, 
       Sequence @@ (autotick[[{1, -1}]]\[Transpose])], ticks}\[Transpose])