Is there a Yoneda lemma for categories other than Set?
In the case of enriched categories, there is an enriched version of the Yoneda lemma.
There are two ways to interpret your question. Identifying the role of Set in the Yoneda lemma as the category where your categories are enriched in reveals that if you consider $\mathcal V$-enriched categories, than Set there will change to $\mathcal V$ and you get the enriched Yoneda lemma.
Another way of interpreting the question is whether for ordinary categories (or more generally for categories enriched in a fixed given $\mathcal V$, but let's just keep everything simple by considering just ordinary categories) one can replace Set by another category and still make things work nicely. Now, thinking of the Yoneda lemma as turning an abstract category into a category whose objects are sets and whose morphisms are functions, namely as representing an abstract category as a subcategory of something absed on Set (generalising Cayley's representation of an abstract group as a group of permutations) what we are asking is if there is a category $\mathscr C \ne \mathbf {Set}$ such that any abstract (small) category can be represented as a subcategory of something based on $\mathscr C$. (Yes, this is a bit vague.)
The answer is trivially yes, simply by taking any category that contains $\mathbf {Set}$ as a subcategory (so in particular $\mathbf {Top}$ will work). However, there is more to the Yoneda lemma than just the representation. It is really tightly related to representable functors. So, we are asking not just to represent an abstract category, but to represent it in a very particular way. Adding this condition into the question leads to asking: is the codomain of the Yoneda lemma $\mathscr D\to \mathbf{Set}^{\mathscr D^{\mathrm op}}$ unique in some way? The answer is that it is, namely it the cocompletion of $\mathscr D$. So, in this wider context, the base category being $\mathbf {Set}$ is forced.