How can I cite a previously given, unavailable talk without self-plagiarism

I think the right action hinges on this point:

It is outside the scope of the current paper paper to present those ideas as a new thing, but it would be something that would be appropriate to reference as existing.

It seems like you want to reference your past work because it is out of the scope of the new work and yet still relevant. If this is correct, then referencing the talk will not help the reader, who may (or may not) need to know about the past idea, concept, or whatever it was. Therefore your only option would be to publish the ideas in some form (paper, blog, etc.) or explain them briefly in the new paper.

If you simply want to show the history of your ideas, but it is not important, then I would leave out and simply acknowledge whoever helped you out.

If you are worried primarily about self-plagiarism, I do not see how not citing a non-published talk, that you yourself gave, is much different from not citing some thoughts you had in your head two years ago. It is a bit different, but not to the extent that it would constitute self-plagiarism, which is really about making it clear what is and what is not new in a paper as it is assessed for novelty during peer-review.

It is not a problem to deliver a talk where you outline some ideas, which later turns into a research paper. You don't need to cite the talk, especially if there is no existing record of it. On the contrary, writing up something for the sole purpose of citing it in another paper, could be viewed as unethical "citation padding".

Consider putting a small acknowledgements section at the end of the paper, where you write something like: "I wish to thank the organizers and participants of Reading Group X, which gave me the opportunity to present and discuss the ideas behind this study".

Can you put the part from the talk in an appendix? You could start along the line of

The author is not aware of any publication of idea of X used in this paper. This idea was presented by the author in the study group Y at University of Z in fall 20?? and in the adjacent report My cool idea X. As both are not publicly available, this appendix contains a sketch of the basic idea.

As a bonus, you can cite this appendix in future publications as well as others can do.