Shortening line geometry using expression dialog in QGIS?

Your requirements seem to be similar to the requirements I had for a flow map a while ago. To shorten the lines at the start and at the end, I used geometric difference functions as shown here:

difference(
  difference(
    $geometry,
    buffer( start_point($geometry), 10000 )
  ),
  buffer( end_point( $geometry), 10000 )
)

What you need to do is find the ratio of the diagonal you need vs the one you have. You're essentially finding a similar right triangle to the one you already have so if the diagonal needs to be 78% shorter, you can multiply your x and y by 78%.

Note this diagram (sorry for hand drawn, on my phone)

So for x offset you need something like:

(sqrt("x offset"^2 + "y offset"^2) - "radius")
/sqrt("x offset"^2 + "y offset"^2) * x

You'll need to know the radius of the circle in meters. If it's set to 'map units' for its radius that's easy, if it's set to mm you'll need to multiply by your scale (convert mm to meters)

So if your circle is 5mm radius you could do something like 0.005 * @map_scale