Lagrangian and Hamiltonian EOM with dissipative force
Problem: Given Newton's second law
$$\begin{align} m\ddot{q}^j~=~&-\beta\dot{q}^j-\frac{\partial V(q,t)}{\partial q^j}, \cr j~\in~&\{1,\ldots, n\}, \end{align}\tag{1} $$
for a non-relativistic point particle in $n$ dimensions, subjected to a friction force, and also subjected to various forces that have a total potential $V(q,t)$, which may depend explicitly on time.
I) Conventional approach: There is a non-variational formulation of Lagrange equations
$$\begin{align} \frac{d}{dt} \left(\frac{\partial L}{\partial \dot{q}^j}\right)-\frac{\partial L}{\partial q^j}~=~&Q_j, \cr j~\in~&\{1,\ldots, n\},\end{align}\tag{2} $$
where $Q_j$ are the generalized forces that do not have generalized potentials. In our case (1), the Lagrangian in eq. (2) is $L=T-V$, with $T=\frac{1}{2}m\dot{q}^2$; and the force
$$ Q_j~=~-\beta\dot{q}^j\tag{3} $$
is the friction force. It is shown in e.g. this Phys.SE post that the friction force (3) does not have a potential. As OP mentions, one may introduce the Rayleigh dissipative function, but this is not a genuine potential.
Conventionally, we additionally demand that the Lagrangian is of the form $L=T-U$, where $T=\frac{1}{2}m\dot{q}^2$ is related to the LHS of EOMs (1) (i.e. the kinematic side), while the potential $U$ is related to the RHS of EOMs (1) (i.e. the dynamical side).
With these additional requirements, the EOM (1) does not have a variational formulation of Lagrange equations
$$\begin{align} \frac{d}{dt} \left(\frac{\partial L}{\partial \dot{q}^j}\right)-\frac{\partial L}{\partial q^j}~=~&0,\cr j~\in~&\{1,\ldots, n\},\end{align}\tag{4} $$
i.e. Euler-Lagrange equations. The Legendre transformation to the Hamiltonian formulation is traditionally only defined for a variational formulation (4). So there is no conventional Hamiltonian formulation of the EOM (1).
II) Unconventional approaches:
Trick with exponential factor$^1$: Define for later convenience the function $$ e(t)~:=~\exp(\frac{\beta t}{m}). \tag{5}$$ A possible variational formulation (4) of Lagrange equations is then given by the Lagrangian $$\begin{align} L(q,\dot{q},t)~:=~&e(t)L_0(q,\dot{q},t), \cr L_0(q,\dot{q},t)~:=~&\frac{m}{2}\dot{q}^2-V(q,t).\end{align}\tag{6}$$ The corresponding Hamiltonian is $$ H(q,p,t)~:=~\frac{p^2}{2me(t)}+e(t)V(q,t).\tag{7}$$ One caveat is that the Hamiltonian (7) does not represent the traditional notion of total energy. Another caveat is that this unconventional approach cannot be generalized to the case where two coupled sectors of the theory require different factors (5), e.g. where each coordinate $q^j$ has individual friction-over-mass-ratios $\frac{\beta_j}{m_j}$, $j\in\{1, \ldots, n\}$. For this unconventional approach to work, it is crucial that the factor (5) is an overall common multiplicative factor for the Lagrangian (6). This is an unnatural requirement from a physics perspective.
Imposing EOMs via Lagrange multipliers $\lambda^j$: A variational principle for the EOMs (1) is $$\begin{align}L ~=~& m\sum_{j=1}^n\dot{q}^j\dot{\lambda}^j\cr &-\sum_{j=1}^n\left(\beta\dot{q}^j+\frac{\partial V(q,t)}{\partial q^j}\right)\lambda^j.\end{align}\tag{8}$$ (Here we have for convenience "integrated the kinetic term by parts" to avoid higher time derivatives.)
Doubling trick: See e.g. eq. (20) in C.R. Galley, arXiv:1210.2745. The doubled Lagrangian is $$\begin{align} \widetilde{L}(q_{\pm},\dot{q}_{\pm},t) ~=~&\left. L(q_1,\dot{q}_1,t)\right|_{q_1=q_+ + q_-/2}\cr ~-~&\left. L(q_2,\dot{q}_2,t)\right|_{q_2=q_+ - q_-/2}\cr ~+~&Q_j(q_+,\dot{q}_+,t)q^j_-\end{align}\tag{9}. $$ The initial conditions are $$\left\{\begin{array}{rcl} q^j_+(t_i)&=&q^j_i,\cr\dot{q}^j_+(t_i)&=&\dot{q}^j_i,\cr q^j_-(t_i)&=&0.\end{array}\right.\tag{10} $$ The final conditions are $$\begin{align}\left\{\begin{array}{rcl} q^j_-(t_f)&=&0\cr \dot{q}^j_-(t_f)&=&0 \end{array}\right. & \cr\cr\qquad\Downarrow&\qquad\cr\cr \left.\frac{\partial \widetilde{L}}{\partial \dot{q}^j_+}\right|_{t=t_f}~=~&0 .\end{align}\tag{11} $$ The $5n$ boundary conditions (10) & (11) do not overconstrain the system. One still gets the Lagrange equations (2) [now posed as an initial value problem!], and the physical limit solution $q_-^j= 0$. The doubling trick (9) is effectively the same as introducing Lagrange multipliers (8).
Gurtin-Tonti bi-local method: See e.g. this Phys.SE post.
--
$^1$ Hat tip: Valter Moretti.
Problem: solve the EOM
$$\ddot x + \beta \dot x + \omega^2 x = f(t)$$
As an approach we shall use, additionally to $x(t), \dot x(t)$, two new parameters $y(t), \dot y(t)$.
Let us, magically, introduce a Lagrangian for this auxiliary system
$$L(x, y, \dot x, \dot y, t) = \dot x \dot y - \beta \dot x y - \omega^2 x y - (x + y) f(t)$$
The important thing to notice is that the equations of motion for this system are
$$ \frac{d}{dt}\left( \frac{\partial L}{\partial \dot x} \right) - \frac{\partial L}{\partial x} = \ddot y - \beta \dot y + w^2 y - f(t) = 0\\ \frac{d}{dt}\left( \frac{\partial L}{\partial \dot y} \right) - \frac{\partial L}{\partial y} = \ddot x + \beta \dot x + w^2 x - f(t) = 0 $$
As one can see, we recover the equations of motion for our original system along with an auxiliary EOM.
From here on out, everything goes according to theory for Hamiltonian mechanics. We can find the generalized momenta:
$$ p_x = \frac{\partial L}{\partial \dot x} = \dot y - \beta y\\ p_y = \frac{\partial L}{\partial \dot y} = \dot x$$
And rewriting the Langrangian as a Hamiltonian
$$H(x, y, p_x, p_y, t) = p_x p_y + \omega^2 x y + \beta y p_y + (x + y) f(t)$$
The method is a bit more general, see Conservative perturbation theory for nonconservative systems which introduced me to the idea of auxiliary parameters by the example of the Van der Pol oscillator.
As far as I can see, this method should play nicely even when $x \in \mathbb R^n$ in which case you would also choose $y \in \mathbb R^n$.