How do I draw a rectangle across subplots?

Define a coordinate in each plot to save eg. the upper right and the lower left corner of the rectangle. Then you can draw the rectangle after the second group plot.

Code:

\documentclass[border=10mm]{standalone}
\usepackage[utf8]{inputenc}
\usepackage[dvipsnames]{xcolor}
\usepackage{mathtools}
\usepackage{pgfplots}
\usepgfplotslibrary{groupplots}
\pgfplotsset{compat=1.8}

\pgfplotsset{
axis line style={line width=0.3pt, BrickRed},
axis x line = middle,
% axis y line = center,
y axis line style = {draw=none}
every axis label/.append style ={NavyBlue},
every tick label/.append style={Emerald},
major grid style={line width=.2pt,draw=gray!30},  
grid style = {line width=.1pt, draw=gray!10},
xlabel style={font=\color{white!15!black}},
ylabel style={font=\color{white!15!black}},
axis background/.style={fill=white},
xmajorgrids={true},
xminorgrids={true},
ymajorgrids={true},
yminorgrids={true},  
title style={font=\bfseries},  
}

\begin{document}
\begin{tikzpicture}
 \begin{groupplot} [
        group style={group size=1 by 2, vertical sep = 1cm},
        domain=-8:8,
        %enlarge x limits=0.1,
        xlabel={k},
        %ymin=-1,
        %ymax=4,
        %ylabel={x[n]},
        %title={Exp},
        height=5cm,
        width=17cm,
        %minor tick num=1,
        xtick distance=1,
        ytick distance=1,
        ticks=major,
    ]
    \nextgroupplot [ylabel={$x[k]=u[k]$}, title={$x[k]=u[k]$},]
    \addplot[ycomb,color=black,solid, mark=*, style={mark size=3pt}, mark options={solid,fill=white}, thick, domain=0:7, samples=8, ylabel={x[n]=u[n]}] 
    {1};

    \node[above] at (axis cs:7.5,0.3) {$\boldsymbol{\cdots}$};

    \addplot[ycomb,color=black,solid, mark=*, style={mark size=3pt}, mark options={solid,fill=white}, thick, domain=-8:-1, samples=8] 
    {0};    
    \node[] at (axis cs:-9,0) {$\boldsymbol{\cdots}$};  

    \addplot[ycomb, smooth] {0};
    \coordinate(h1)at(axis cs:5,1);% <- added

    \nextgroupplot [
        ylabel={}, 
        xticklabels={}, extra x ticks={4,5}, 
        extra x tick labels={$n-1$, $n$}, 
        extra x tick style={tick label style={rotate=45}}
        ]
    \addplot[ycomb,color=black,solid, mark=*, style={mark size=2pt}, mark options={solid,fill=white}, thick, domain=-8:5, samples=14] 
    {0.85^(-x)};
    \node[] at (axis cs:-3,1.5) {$h[n-k]=(\frac{3}{4})^{n-k} u[n-k]$};  

    \addplot[ycomb,color=black,solid, mark=*, style={mark size=2pt}, mark options={solid,fill=white}, thick, domain=6:8, samples=3] 
    {0};
    \addplot[ycomb, smooth] {0};
    \coordinate(h2)at(axis cs:0,0);% <- added
\end{groupplot}
\draw[blue]([shift={(-.2,-.2)}]h2)rectangle([shift={(.2,.2)}]h1);% <- added
\end{tikzpicture}
\end{document}

Additional remark: Do you really want compat=1.8? Current version is 1.15 and since version 1.11 axis cs is the default coordinate system inside environment axis.

Another way: once you have (or defined) two nodes, such as esdd did by

\coordinate (h1) at (axis cs:5,1);% in one group plot
\coordinate (h2) at (axis cs:0,0);% in the other group plot

you can draw the desired rectangle using the fit library:

\usetikzlibrary{fit}% in the preamble
...
\node[fit = (h1) (h2), inner sep = 6pt, draw] {};% in your `tikzpicture`

Advantage: it fits ;-) and it's easy to add some spacing (inner sep) but most valuable: you don't have only a rectangle, but you got a node that you can refer to, such as putting a label next to it or using it as anchor for drawing an arrow.