Give Me a Ladder!

Python 2, 316 bytes

def f(m):
 r=[list(' '*61)for y in range(31)]
 (y,x),(b,a)=[[(109-p)/10*3,1+min(p%20,19-p%20)*6]for p in[max(m)-1,min(m)-1]]
 while y<b:c=cmp(a,x);r[y][x:x+4]=r"  /|\___/|\ "[c%3::3];x+=c;y+=1
 if x==a:r[y-1][x+2-(r[y-1][x]>' ')]=" "
 while x-a:c=cmp(a,x)+1;r[y-2][x+c:x+2+c]="__";r[y-1][x+c*3/2]="_";x+=c-1
 print r

Try it online! (footer paints over grid for pretty output)

Function that inputs a list of two numbers and outputs a list of lists of characters.

How it works

We find character coordinates of the higher point (e.g. 96), then translate it along diagonal, vertical, or horizontal movements to the coordinates of the lower point (e.g. 54), modifying the character matrix r as we go.

def f(m):
    r = [list('.'*61) for y in range(31)]
    m = [max(m)-1,min(m)-1]
    # (x,y) is the higher point
    # (a,b) is the lower point
    # x increases right; y increases down
    (y,x),(b,a) = [[(109-p)/10*3,1+min(p%20,19-p%20)*6] for p in m]
    # move (x,y) to (a,b) via 1-unit translations
    # \ diagonal
    while x < a and y < b:
        r[y][x+1:x+4] = "\\_\\"
        x += 1
        y += 1
    # / diagonal
    while x > a and y < b:
        r[y][x:x+3] = "/_/"
        x -= 1
        y += 1
    # straight down
    while x == a and y < b:
        r[y][x+1:x+4] = "|_|"
        y += 1
    # remove space at the bottom of the ladder
    # (r[y-1][x]!='.' deals with off-by-one error in the case of
    # \ diagonals with no straight segment)
    if x == a:
        r[y-1][x+2-(r[y-1][x]!='.')] = " "
    # left
    while x > a:
        r[y-2][x+1] = "_"
        r[y-2][x] = "_"
        r[y-1][x] = "_"
        x -= 1
    # right
    while x < a:
        r[y-2][x+2] = "_"
        r[y-2][x+3] = "_"
        r[y-1][x+3] = "_"
        x += 1
    return r

Charcoal, 130 bytes

ＦχＦχ⊞υ×⁶⎇﹪ι²⁻⁹κκ≔⁻ＬυＮθ≔⁻ＬυＮη≔⊕§υηζ≔׳÷ηχηＪ§υθ׳÷θχ Ｆ›ζⅈ«≦⊕ζ »Ｆ⁻ζ⁺ⅈ⁻ⅉηＧ←←↑→↘→²_Ｆ⁻ⅈ⁺ζ⁻ⅉηＧ↑←↙→²_Ｗ›ⅉη«≔⎇›⁻ⅉη↔⁻ζⅈ²∨›ζⅈ³ιＰ✳ι¹←←✳ι¹→¿›ⅉη_

Try it online! Link is to verbose version of full 198-byte program that includes drawing the entire background. Explanation:

ＦχＦχ⊞υ×⁶⎇﹪ι²⁻⁹κκ

Get the X-coordinates of all 100 squares (since each row reverses direction).

≔⁻ＬυＮθ≔⁻ＬυＮη

Input the starting and ending square, but subtract from 100.

≔⊕§υηζ≔׳÷ηχη

Calculate the destination character.

Ｊ§υθ׳÷θχ

Jump to the start character.

 Ｆ›ζⅈ«≦⊕ζ »

Erase the start character, and if this is a right-leaning ladder, then erase the next character too (this is because right-leaning ladders start 1 character further right than left-leaning ladders for some reason).

Ｆ⁻ζ⁺ⅈ⁻ⅉηＧ←←↑→↘→²_

Draw the horizontal part of a right-leaning ladder, if applicable. This is complicated slightly because the horizontal part does not start 1 character further right.

Ｆ⁻ⅈ⁺ζ⁻ⅉηＧ↑←↙→²_

Draw the horizontal part of a left-leaning ladder, if applicable.

Ｗ›ⅉη«

Repeat until the top of the ladder is reached.

≔⎇›⁻ⅉη↔⁻ζⅈ²∨›ζⅈ³ι

Work out which direction we're going in (3 = up left, 2 = up, 1 (default) = up right).

Ｐ✳ι¹←←✳ι¹→

Draw the next section of ladder.

¿›ⅉη_

Draw the next rung, unless we've got to the top of the ladder.