What is the name of the vertical bar in $(x^2+1)\vert_{x = 4}$ or $\left.\left(\frac{x^3}{3}+x+c\right) \right\vert_0^4$?

This may be called Evaluation bar. See, in particular, here (Evaluation Bar Notation:).

Jeff Miller calls it "bar notation" in his Earliest Uses of Symbols of Calculus (see below). The bar denotes an evaluation functional, a concept whose importance comes to the fore when one studies duality of vector spaces (e.g. such duality plays a key role in the Umbral Calculus).

The bar notation to indicate evaluation of an antiderivative at the two limits of integration was first used by Pierre Frederic Sarrus (1798-1861) in 1823 in Gergonne’s Annales, Vol. XIV. The notation was used later by Moigno and Cauchy (Cajori vol. 2, page 250).

Below is the cited passage from Cajori

In my calculus book, the vertical bar is called the "evaluation symbol", and this phrase is bolded when first mentioned. It makes sense, I suppose.

Copy paste from wikipedia: Division is often shown in algebra and science by placing the dividend over the divisor with a horizontal line, also called a vinculum or fraction bar, between them.