Newtonian mechanics: massless particles

The other answers are right, but I'd like to give some practical advice as well.

You might encounter, for example, a block and tackle system where you are asked to treat the blocks as massless. What this means in practice is that you should enforce the constraint that the sum of forces on the blocks is $0$.

More generally, you might think of a massless Newtonian particle as a formal (not strictly physical) object which enforces the constraint $\sum F = 0$.

As you already pointed out, massless particles have no meaning in Newton mechanics. This comes back to the way in which mass is defined

\begin{equation} m = \frac{F}{a} \ . \end{equation}

Implying that a particle with $m=0$ would have no momentum and no kinetic energy and, hence, would be nothing at all.

It's sometimes meaningful to describe a particle as having "negligible mass", i.e. to consider the limit where the particle's mass tends to zero. Sometimes, colloquially, such particles are described as "massless", although they should not be confused with massless particles like photons in special (or general) relativity, which have a zero rest mass but a non-zero momentum, and thus behave qualitatively differently from the "negligible mass" limit of Newtonian physics (where the particle's momentum also tends to zero).

One common example of such a situation occurs with orbital dynamics under Newtonian gravity. For example, consider a man-made satellite orbiting the Earth. Since the satellite's mass is vanishingly small compared to the mass of the Earth (or of the Sun or the Moon or any other planets or moons in the solar system) it has essentially no measurable effect on the movement of the Earth or any other bodies in the solar system. But it still follows a well defined trajectory that mathematically tends to a well defined limit as the satellite's mass decreases towards zero.

So, for calculations, it's often useful to treat the satellite's mass as if it were exactly zero, both so that we don't actually need to assign any specific mass to the satellite when performing the calculation and so that its influence on the trajectories of other bodies does not need to be calculated.

Of course, a zero mass means that any non-zero force applied to the satellite would result in an infinite acceleration, which is clearly unphysical. But as long as the satellite is only affected by gravity, this turns out to cause no real problems, since the gravitational forces exerted on the satellite by other bodies are also proportional to its mass, and these effects cancel each other out so that even a particle of negligible mass still experiences a gravitational acceleration that tends to a finite limit as the particle's mass tends to zero.

Fundamentally, this convenient cancellation reflects the fact that gravity is actually a fictitious force in general relativity — and thus also behaves like one under Newtonian physics, which is a low-mass, low-speed limit of general relativity. (See also this earlier answer I wrote to a related question.)