All your base palindromic belong to us

Pyth, 13 bytes

mlf_ITjLdSdSQ

The brevity of this is mostly due to the Invaluable "Invariant" command.

msf_ITjLdSdSQ       implicit: Q=input
m         d         map lambda d over
           SQ       Inclusive range 1 to Q
      jLdSd         Convert d to all the bases between 1 and d
  f                  filter lambda T:
   _IT                 is invariant under reverse
 l                  number that are invariant under reverse

If True is an acceptable output for 1, msm_IjdkSdSQ (12 bytes) works.

Try it here.

Jelly, 14 bytes

bR‘$µ=UP€S
RÇ€

Try it online!

Non-competing version

The Jelly interpreter had a bug that made converting to unary impossible. This has been fixed now, so the following code (12 bytes) also accomplishes the task at hand.

bRµ=UP€S
RÇ€

Try it online!

How it works

bR‘$µ=UP€S  Helper link. Argument: z

 R‘$        Apply range and increment, i.e., map z to [2, ..., z + 1].
            In the non-competing version R simply maps z to [1, ... z].
b           Convert z to each of the bases to the right.
    µ       Begin a new, monadic chain. Argument: base conversions
     =U     Compare the digits of each base with the reversed digits.
            = has depth 0, so [1,2,3]=[1,3,3] yields [1,0,1].
       P€   Take the product of the innermost arrays.
         S  Sum all resulting Booleans.


RÇ€         Main link. Argument: n

R           Yield [1, ..., n].
 ǀ         Apply the helper link to each.

MATL, 19 20 bytes

:"0@XK:Q"K@:YAtP=A+

Uses current release (10.1.0), which is earlier than this challenge.

Try it online!

Explanation

:            % vector [1,2,...,N], where "N" is implicit input
"            % for each number in that vector
  0          % push 0
  @          % push number 1,2,...N corresponding to current iteration, say "n" 
  XK         % copy n to clipboard
  :Q         % vector [2,3,...,n+1]
  "          % for each number "m" in that vector
    K        % push n
    @:       % vector [1,2,...,m]
    YA       % express n in base m with symbols 1,2,...,m
    tP       % duplicate and permute
    =A       % 1 if all entries are equal (palindrome), 0 otherwise
    +        % add that number
             % implicitly close the two loops and display stack contents