Old style number, bold small caps, and Palatino

You could try using the recently-released newpxtext and newpxmath packages. In the MWE below, issuing the command \useosf after the math-related packages are loaded enables the use of regular- and bold-weight oldstyle numerals in text mode, while lining numerals will be used in math mode. Small caps are available in bold (as well as regular-weight, of course).

Update April 2018: The newpxtext and newpxmath packages have evolved quite a bit over the past five years. One can still enable oldstyle numerals and small-caps letters by running \useosf. According to the packages' user guide, though, the preferred way is to specify the option osf while loading the newpxtext package.

\documentclass{article}
\usepackage{newpxtext,newpxmath}
\useosf % old-style figures in text, not in math
\linespread{1.05} % Palatino needs a bit more leading than CM
\begin{document}
0123456789, {\bfseries 0123456789}, {\itshape 0123456789}, {\bfseries\itshape 0123456789}

\textsc{Once upon a time, \dots}
{\bfseries \scshape Once upon a time, \dots}

Some math expressions: $1+1=2$,  $\int_0^1 \mathrm{d}x = 1$
\end{document}

This is more thought as chronicler’s duty after first writing an erroneous answer and searching for cure than as useful addition in regard to the question.

Beside Mico’s answer:

If you load first mathpazo and tgpagella second, then tgpagella does only overwrite text related definitions of mathpazo, but not the maths part.

This at least unexpected behaviour, if not bug, of tgpagella to produce oldstyle figures in small caps mode, but lining figures in other modes, can be changed with the help of newenviron and xstring (notes below):

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{mathpazo}
\usepackage{tgpagella}

\usepackage{textcomp} % for "\text...oldstyle" commands

\usepackage{newenviron,xstring}
  \newenviron{osf}{ % automatic definition of "\osfbody"
    \expandarg\StrSubstitute{\osfbody}{1}{\textoneoldstyle}[\osfI]
    \StrSubstitute{\osfI}{2}{\texttwooldstyle}[\osfII]
    \StrSubstitute{\osfII}{3}{\textthreeoldstyle}[\osfIII]
    \StrSubstitute{\osfIII}{4}{\textfouroldstyle}[\osfIV]
    \StrSubstitute{\osfIV}{5}{\textfiveoldstyle}[\osfV]
    \StrSubstitute{\osfV}{6}{\textsixoldstyle}[\osfVI]
    \StrSubstitute{\osfVI}{7}{\textsevenoldstyle}[\osfVII]
    \StrSubstitute{\osfVII}{8}{\texteightoldstyle}[\osfVIII]
    \StrSubstitute{\osfVIII}{9}{\textnineoldstyle}[\osfIX]
  }{
    \expandarg\StrSubstitute{\osfIX}{0}{\textzerooldstyle} %could be set into start code as well!
   }

\begin{document}
\textsc{\TeX\ Gyre Pagella} -- a \textsc{Palatino} clone.

1234567890 glyphs in \TeX\ Gyre Pagella.

\textoneoldstyle\texttwooldstyle\textthreeoldstyle\textfouroldstyle
\textfiveoldstyle\textsixoldstyle\textsevenoldstyle\texteightoldstyle
\textnineoldstyle\textzerooldstyle\ glyphs in \TeX\ Gyre Pagella.

\begin{osf}
1234567890 glyphs in \TeX\ Gyre Pagella (1234567890 times).

$(1 + 2) * 34 - 5 - 6 + 7 - 8 = 90$ %produces warnings
\end{osf}

\scshape
1234567890 Small Caps in \TeX\ Gyre Pagella.

\bfseries
1234567890 Small Caps in \TeX\ Gyre Pagella.

$(1 + 2) * 34 - 5 - 6 + 7 - 8 = 90$
\end{document}

• All TeX Gyre font examples use themselves the \textcomp commands for oldstyle figures.

• I used newenviron because it can be nested, for the minimal working example probably environ would have been enough (cf. What is the problem with nested environments using \BODY (environ package)?).

• Note that math mode inside the osf environment produces warnings.

• I did not test this, but applying the environment to the whole document should significantly increase the compile time.

In LuaLaTeX or XeLaTeX, fontspec lets you load any font with the Numbers=OldStyle option, and to use the \oldstylenums and \liningnums commands. TeX Gyre Pagella should just work out of the box with \setmainfont[Ligatures={Common,Rare,TeX}, Numbers=OldStyle]{TeX Gyre Pagella}. So will any recent OpenType version of Palatino (although some old versions of Palatino Linotype have the bug that small-caps i is dotted.)

Then, you can load either Asana Math or TeX Gyre Pagella Math in unicode-math.