Changing the position of rows and columns in a matrix

Simply use Part and Set:

f[A_?SquareMatrixQ, i_Integer, j_Integer] := Module[{p, idx},
 idx = Range[i, j, Sign[j - i]];
 p = Range[1, Length[A]];
 p[[idx]] = RotateLeft[idx];
 A[[p, p]]
 ]

A = Outer[Plus, Range[5], Range[0, 4]];
A // MatrixForm

$\left( \begin{array}{ccccc} 1 & 2 & 3 & 4 & 5 \\ 2 & 3 & 4 & 5 & 6 \\ 3 & 4 & 5 & 6 & 7 \\ 4 & 5 & 6 & 7 & 8 \\ 5 & 6 & 7 & 8 & 9 \\ \end{array} \right)$

B = f[A,2,4];
B // MatrixForm

$\left( \begin{array}{ccccc} 1 & 3 & 4 & 2 & 5 \\ 3 & 5 & 6 & 4 & 7 \\ 4 & 6 & 7 & 5 & 8 \\ 2 & 4 & 5 & 3 & 6 \\ 5 & 7 & 8 & 6 & 9 \\ \end{array} \right)$

to move a row from $i$ to $j$:

rowmove[A_?MatrixQ, i_Integer, j_Integer] := Insert[Delete[A, i], A[[i]], j]

to move a column from $i$ to $j$:

colmove[A_?MatrixQ, i_Integer, j_Integer] := Transpose@rowmove[Transpose[A], i, j]

both at the same time:

move[A_?MatrixQ, i_Integer, j_Integer] := colmove[rowmove[A, i, j], i, j]

Define the indices that you want to interchange in a list $ind$ and then you can index into the array $a$ directly.

a = {{1, 2, 3, 4, 5}, {2, 3, 4, 5, 6}, {3, 4, 5, 6, 7},
     {4, 5, 6, 7, 8}, {5, 6, 7, 8, 9}}; 
ind = {1, 3, 4, 2, 5};
a[[ind, ind]]

If you don't want to specify the index array each time, Roman points out that you can build a simple function to do it:

indexlist[n_, i_, j_] := Insert[Delete[Range[n], i], i, j]

So to get the above you would specify

ind = indexlist[5,2,4]