Can the existence of certain natural numbers or sets of natural numbers imply large cardinals?
No, you do not get large cardinals in the actual universe out of the existence of such small sets. The reason is that if $\kappa$ is least such that $V_\kappa$ is a model of set theory, then $V_\kappa$ also contains all these small sets, but has no large cardinals (by minimality of $\kappa$).
Instead, what we get is the consistency of extensions of set theory with large cardinals. In practice we actually tend to get more, namely, that the large cardinals exist in certain inner models.
For instance, $0^\sharp$ can be defined as a certain set of numbers. If it exists, then in $L$ there are many inaccessible cardinals, although you cannot prove that it gives you any large cardinals in $V$.
For another example, to see how flexible the format is, consider the statement that all projective sets of reals are Lebesgue measurable. This is a statement about the first order theory of the reals (and it can be stated as a schema of second order arithmetic, via appropriate coding). You cannot get large cardinals from it directly, but it implies that $\omega_1^V$ is an inaccessible cardinal in $L$.
As phrased, it's not entirely clear what you are asking. Several possible answers, that don't have all that much in common, come to mind:
Certain structures, which can be coded as subsets of $\omega$, do imply that there are inner models with large cardinals. E.g. $0^\#$ implies that $L$ has a proper class of inaccessibles, $0^\dagger$ (read "zero dagger") implies that there is an inner model with a measurable cardinal, "zero pistol" (whose symbol is apparently not supported by MathJaX) implies that there is an inner model with a strong cardinal, ... They don't, however, imply that these large cardinals exist in our background universe.
Since there is no formal definition of 'large cardinal', the following has to be taken with a grain of salt: Subsets of $\omega$ don't provably add large cardinals to our background universe.
Why? Well, suppose $A \subseteq \omega$ did add/witness a large cardinal in $V$. Let $\kappa$ be the least wordly cardinal of $V$ (which should exist because $A$ adds a large cardinal). Then $A \in V_{\omega +1} \subseteq V_{\kappa}$ and $(V_\kappa; \in) \models \mathrm{ZFC}$ but the large cardinal that $A$ witnesses doesn't exist in $V_\kappa$.
Anything can be added by a subset of $\omega$:
(Jensen.) There is always a class generic $G$ and some $x \subseteq \omega$ in $V[G]$ such that $$ V[G] = L[x] \supseteq V $$ So any large cardinal that exists in $V$ is, in a sense, added by the real $x$.
My hope is that this helps you to get to the essence of your intuitively justified question which would then allow us to think about a more precise answer.