In which experiment did protons seem to consist of infinite amount of quarks?

Thanks for finding this amazing historical video.

He's talking about the deep inelastic scattering electron proton experiment at SLAC. This showed evidence that high energy electrons scattered off pointlike charged particles within the proton, which Feynman named 'partons'. It took some time to establish that these partons are the same as quarks, which had been postulated to make sense of the patterns of mesons and baryons. We now understand that they are the same, but that the proton consists of three 'valence' quarks (up up down) plus a 'sea' of quarks and antiquarks which the electrons will scatter off (as well as gluons). So in a sense there are three quarks in a proton and in a sense there are an infinite number.

The SLAC measurements were confirmed by later experiments, particularly the HERA electron proton ring at DESY, with much more detail. In particular the early evidence for 'scaling': that scattering depended only on $x$, the fraction of the proton momentum embodied in the stuck Parton, and not on $Q^2$, the mass of the exchanged virtual photon, turned out to be wrong. The experiment just happened to look at a region where it was approximately true, and maybe that misled us for a while. But apart from that the results hold, and we now understand that the contradiction that was puzzling Feynman in the video is not a contradiction after all.

The answer by RogerBarlow is correct, here I want to elaborate between the present experimental knowledge of the contents of a proton and the knowledge at the time the video was taken, (before 1970? he looks young enough).

This is the instantaneous picture of the proton as we know it now.

The three valence quarks are there, within the sea of gluons and quark antiquark pairs continually, due to the quantum chromodynamic interaction appearing and disappearing.

The parton distribution functions of the constituents have been measured in many experiments , example, :

Parton distribution functions in the proton extracted from the H1 data. The gluon g and sea S distributions are scaled by a factor 0.01. The uncertainties include experimental (inner) and model (middle) uncertainties and the parametrization variations (outer).

and confirm the QCD structure, and yes, depending on the energy transfer there can be an infinity of quark antiquark and gluons in the proton (hadrons generally) . This is now.

When the video was taken, Feynman had proposed the parton model at the time, which had successes.

The parton model was proposed by Richard Feynman in 1969 as a way to analyze high-energy hadron collisions.2 Any hadron (for example, a proton) can be considered a composition of a number of point-like constituents, termed "partons". The parton model was immediately applied to electron-proton deep inelastic scattering by Bjorken and Paschos.

In the 1970's the parton model fitted scattering data except at high transverse momentum, which was the first experimental indication for the theory of QCD which predicted distributions with high transverse momenta.

Feynman was reluctant to accept the quark model, which depended on the symmetries discovered in the eightfold way. That is why he is arguing in the video about 3 particles etc, his parton model had infinity, according to the energy supplied, and he wanted to reject the quark model by reductio ad absurdum. By 1980 when QCD from fitting deep inelastic data , overthrew the parton model, he was forced to accept that the partons proposed in his model were all this soup of quarks and antiquarks and gluons.

Return to the question, you state:

Richard Feynman is telling that in some experiment it seems that the proton should consist of infinite amount of quarks.

He is talking of his parton model, not quarks, which quarks, three for each nucleon, were at that time a proposal for explaining the symmetries obtained in strong interaction resonances. They were not considered partons at that time, since they were only three, in the quark model of the proton. . They are now called valence quarks, to contrast with the sea ones.

What is this case he's mentioning? Is it solved now

So yes, it is solved by accepting that the partons are the quarks antiquarks and gluons in the QCD dominated soup within the bag that is the proton.