Is there a distinct size limit between planet and star?

The basic physics here is that the virial theorem tells us that as a proto-planet/star radiates away energy, it will contract and it's interior will become hotter. It will eventually either become hot enough to begin nuclear fusion or dense enough that electron degeneracy supports the star and it can cool from there without getting any hotter in the middle.

There are at least four reasons why this boundary between star and planet is a bit blurred

  1. Nuclear fusion for different species is initiated at quite different temperatures. Deuterium and lithium undergo fusion at lower temperatures than hydrogen. This means that objects of lower mass (about 13 and 60 Jupiter masses respectively) will fuse these species. However, by convention these are not referred to as stars because the abundance of these species is too low to provide sufficient power to halt the contraction (see below).

  2. (And I think this answers the main points of your question) Nuclear fusion does not turn on like a light switch. It is the protons in the high energy tail of the Maxwell-Boltzmann distribution that participate, and this is a continuous distribution. At any particular energy, the population of "eligible" protons is a function of temperature. However, it is a steep function of temperature (like $T^4$ for the pp chain), so in some respects, the initiation of fusion is sudden, but it is not instantaneous. The point at which a star becomes a star is not that well defined. In stars like the Sun, you can use the point at which the radius stops getting smaller, when nuclear fusion supplies all the energy radiated by the star. But sometimes a criterion is used that say 99% of the star's luminosity must come from fusion. Either way, there are objects with mass just below this boundary that have fusion reactions going on, but only at a rate that slows the contraction and does not totally supply the luminosity of the object. Such objects will continue to contract, become more dense and ultimately will become supported by electron degeneracy pressure. From there, they can cool without further significant contraction and fade into obscurity as cool L-dwarfs, then T-dwarfs, then Y-dwarfs.

  3. The threshold mass for the transition to an object that can supply all its luminosity by fusion depends on the composition of the object. Low metallicity objects have a higher threshold mass. The difference is small, but not negligible. The threshold is probably about 80 Jupiter masses for low metallicity objects.

  4. Many would not see falling below some mass threshold as a good definition for a "planet". Indeed whilst many astronomers would refer to objects between the deuterium- and hydrogen-burning thresholds as "brown dwarfs", rather than planets, others argue that the formation process (around a star and perhaps with a rocky or icy core) should be the defining planetary features. This really is a blurring of the planet/brown dwarf boundary, though means that there probably isn't a planet/star boundary at all!