Create a vortex!

MATL, 60 bytes

3<G+Eqt&OlG8M+t&(1G:8*X^Q2/kG+G:Gq+&vG2=+Z{(XJ3:"J@X!+]g35*c

Try it online! Or verify test cases: 0, 1, 2, 3, 4, 10.

This turned out to be funnier than I expected. Explaining is going to be harder, though...

Jelly, 48 bytes

Rȯ1ĖŒṙ‘Ė’ḣµ;NµN0¦€U;
Ç+;ẋ2W$+⁸<3¤µFṀR,þ$e€€ị⁾# Y

Try it online!

(Make it all more square by replacing the final Y with G, adding a space between each column).

How?

Builds a list of # coordinates of an arm relative to the centre. Transforms that to the coordinates of the four arms from the top-left corner and adds the centre coordinate. Builds a table of all the coordinates in the space and sets the arms to # and the space to and joins up the rows with newlines.

Rȯ1ĖŒṙ‘Ė’ḣµ;NµN0¦€U; - Link 1, arm coordinates relative to centre: arm-length a
R                    - range: [1,2,...,a]
 ȯ1                  - or 1 (stops Œṙ erroring with arm-length 0; builds no redundant coordinates in the end)
   Ė                 - enumerate: [[1,1],[2,2],...[a,a]]  (or [[1,1]] if a=0)
    Œṙ               - run-length decode: [1,2,2,...,a,a,...,a] (or [1] if a=0)
      ‘              - increment: [2,3,3,...,a+1,a+1,...,a+1] (or [2] if a=0)
       Ė             - enumerate: [[1,2],[2,3],...,[T(a)-a,a+1],[T(a)-a+1,a+1],...,[T(a),a+1]] where T(a)=(a+1)*a/2 (or [[1,2]] if a=0)
        ’            - decrement: [[0,1],[1,2],...,[T(a)-a-1,a],[T(a)-a,a],...a[T(a)-1),a]] (or [[0,1]] if a=0)
         ḣ           - head to a (first a of those) - these are an arm's relative coordinates from the central `#` at [0,0])
          µ          - monadic chain separation (call that list r)
           ;         - r concatenated with
            N        - negate r (gets the opposite arm)
             µ       - monadic chain separation (call that list s)
                 €   - for €ach coordinate pair in s:
               0¦    -     apply to index 0 (the right of the two values):
              N      -         negate
                  U  - upend (reverse each pair of that, gives the other two arms)
                   ; - concatenate that list with s (gives all four arms)

Ç+;ẋ2W$+⁸<3¤µFṀR,þ$e€€ị⁾# Y - Main link: arm-length a
Ç                           - call the last link(1) as a monad (get centre-relative coordinates)
 +                          - add a (make the coordinates relative to the top-left)
      $                     - last two links as a monad:
   ẋ2                       -     repeat a twice: [a,a]
     W                      -     wrap in a list: [[a,a]] (the centre coordinate)
  ;                         - concatenate (add the centre coordinate)
           ¤                - nilad followed by link(s) as a nilad:
        ⁸                   -     link's left argument, a
         <3                 -     less than three?
       +                    - add (a in 0,1,2 are special cases requiring a grid one-by-one more than all the rest)
            µ               - monadic separation (call that c)
             F              - flatten c into one list
              Ṁ             - get the maximum (the side-length of the space)
                  $         - last two links as a monad:
               R            -     range: [1,2,...,side-length]
                ,þ          -     pair table: [[[1,1],[1,2],...,[1,side-length]],[[2,1],[2,2],...,[2,side-length]],...,[[side-length,1],[side-length,2],[side-length, side-length]]]
                   e€€      - exists in c? for €ach for €ach (1 if a # coordinate, 0 otherwise)
                       ⁾#   - literal ['#',' ']
                      ị     - index into
                          Y - join with new line characters
                            - implicit print