How to avoid plagiarism when working on an idea not developed from a student who quit the PhD program?

It is easy to discharge your ethical obligations here: you treat the unpublished paper like you would any other source, i.e., citing it carefully and making sure not to rewrite sentences or rehash ideas without explicit attribution (include direct quotes if necessary, and probably don't include too many of those without a very compelling reason).

As you point out, the more serious risk is that you are starting thesis work on a problem on which someone else did a substantial amount of work on (though you worked on it independently) and that you may come out with little which is essentially new. You're right that this is a risk: it's a risk for all thesis work everywhere. Again my advice is to not treat this any differently than work that has been published: clearly you need to go substantially beyond it, or you don't have a thesis. Whether the prior work of this other person shows that this is a fruitful area for you to continue in or whether it is evidence that all the good stuff has already been done is one of those tough calls that your thesis advisor should help you with.

Finally, of course it is a shame if two people have done good work on something and neither of them publishes it. If it turns out that you do your thesis work on something else, then as a separate project (or perhaps better: a later project), you may want to explore a publication which is a synthesis of your work and this other person's. Offering him coauthorship would be appropriate, but if he is really out of academia then it becomes a service to the profession for you to publish the work, while accurately conveying its provenance. It's just that that kind of publication would probably not meet the requirements for a PhD thesis (and even if it did, it would not be a good launching of your career).

You cite your source and give proper credit, even if it is an obscure unpublished manuscript. Your advisor shouldn't let you defend such a dissertation unless you really have made a substantial contribution of your own.