What does closed form solution usually mean?

I would say it very much depends on the context, and what tools are at your disposal. For instance, telling a student who's just mastered the usual tricks of integrating elementary functions that

$$\int\frac{\exp{u}-1}{u}\mathrm{d}u$$

and

$$\int\sqrt{(u+1)(u^2+1)}\mathrm{d}u$$

have no closed form solutions is just the fancy way of saying "no, you can't do these integrals yet; you don't have the tools". To a working scientist who uses exponential and elliptic integrals, however, they do have closed forms.

In a similar vein, when we say that nonlinear equations, whether algebraic ones like $x^5-x+1=0$ or transcendental ones like $\frac{\pi}{4}=v-\frac{\sin\;v}{2}$ have no closed form solutions, what we're really saying is that we can't represent solutions to these in terms of functions that we know (and love?). (For the first one, though, if you know hypergeometric or theta functions, then yes, it has a closed form.)

I believe it is fair to say that for as long as we haven't seen the solution to an integral, sum, product, continued fraction, differential equation, or nonlinear equation frequently enough in applications to give it a standard name and notation, we just cop out and say "nope, it doesn't have a closed form".

To better understand closed forms, you may want to familiarize yourself with what's called Differential Algebra. Just as number theory relies on abstract structures such as rings, fields, ideals, etc. to express roots of algebraic equations using elementary numbers, similarly there is a parallel apparatus for expressing functions (i.e. solutions of differential equations) using differential rings, fields, ideals called Differential Algebra. It is this underlying mechanism that defines which functions can be expressed as "closed forms".

Parallels:

1. Similar to splitting fields for algebraic equations, there is a parallel Galois theory with Picard-Vessiot extensions and what not.
2. Similar to correspondence between subfields of number fields and Galois subgroups, on the differential side, there is a correspondence between differential subfields and subgroups of algebraic groups.
3. Just as algebraic equations can be determined to be solvable by radicals, similarly linear differential equations can be determined to be solvable by exponentials, Liouvillian functions, etc. There is an ascending tower of differential fields which can be built.

There is more... I am no expert in this differential algebra field but if you want some freely available references, see

1. Seiler Computer Algebra and differential equations
2. Van der Put Galois theory of differential equations, algebraic groups and Lie algebras
3. Papers by Michael F. Singer are good. See for example "Galois theory of linear differential equations".
4. Check the Kolchin seminar in Differential Algebra

See the links below for Timothy Chow's article.

a)
[Borwein/Crandall 2013] tries to give an answer.
$\ $

b)
Let us assume, $f(x)=0$ is to be solved for $x$.
If the equation $f(x)=0$ has no closed-form solution, the equation has no solution which can be expressed as a closed-form expression.
A mathematical expression is a closed-form expression iff it contains only finite numbers of only constants, explicit functions, operations and/or variables.
Sensefully, all the constants, functions and operations in a given closed-form expression should be from given sets.

c)
Let us say, a (partial) closed-form inverse ($f^{-1}$) is a (partial) inverse (= inverse function) whose function term can be expressed as closed-form expression.
Because of $f(x)=0$ and the definition of a (partial) inverse $f^{-1}(f(x))=x$, the following holds: $f^{-1}(f(x))=f^{-1}(0)$, $x=f^{-1}(0)$. And therefore: If an equation $f(x)=0$ has no closed-form solution, the function $f$ has no partial closed-form inverse, or a partial closed-form inverse exists but is not defined for the argument $0$ of the right side of the equation. This means, $x$ cannot be isolated on only one side of the equation
- by applying a partial closed-form inverse of $f$,
- by only applying the partial closed-form inverses and inverse operations of the closed-form functions respective operations which are contained in the expression $f(x)$.

The existence of a partial closed-form inverse is a sufficient but not a necessary criterion for the existence of a closed-form solution.

d)
Lin and Chow ask for closed-form numbers. A closed-form number is a number that can be generated from a rational number by only closed-form functions.

e)
The elementary functions are a special kind of closed-form functions. The elementary numbers of Lin and the exponential-logarithmic numbers of Chow are special kinds of closed-form numbers.
If $f$ is an elementary function, the following statements are equivalent:
- $f$ is generated from its only argument variable in a finite number of steps by performing only arithmetic operations, power functions with integer exponents, root functions, exponential functions, logarithm functions, trigonometric functions, inverse trigonometric functions, hyperbolic functions and/or inverse hyperbolic functions.
- $f$ is generated from its only argument variable in a finite number of steps by performing only arithmetic operations, exponentials and/or logarithms.
- $f$ is generated from its only argument variable in a finite number of steps by performing only explicit algebraic functions, exponentials and/or logarithms.
Whereas Ritt and Lin allow explicit and implicit algebraic functions, Chow restricts the approved algebraic operations to the explicit algebraic functions, that are the arithmetic operations.
$\ $

[Borwein/Crandall 2013] Borwein, J. M.; Crandall, R. E.: Closed Forms: What They Are And Why We Care. Notices AMS 60 (2013) (1) 50-65

[Chow 1999] Chow, T. Y.: What is a Closed-Form Number? Amer. Math. Monthly 106 (1999) (5) 440-448 or https://arxiv.org/abs/math/9805045

[Lin 1983] Ferng-Ching Lin: Schanuel's Conjecture Implies Ritt's Conjectures. Chin. J. Math. 11 (1983) (1) 41-50

[Risch 1979] Risch, R. H.: Algebraic Properties of the Elementary Functions of Analysis. Amer. J. Math 101 (1979) (4) 743-759

[Ritt 1925] Ritt, J. F.: Elementary functions and their inverses. Trans. Amer. Math. Soc. 27 (1925) (1) 68-90