Could teaching at community college for a few years harm my career?
I'm worried that it would create a record of affiliation with a low-ranked university that would make people see you as "from" that university, and thus be biased against hiring you. Is there any truth to this?
I think it depends when you do it. If it is in between your master's and PhD, as you somewhat tentatively suggest: well, I know many people in my academic field (mathematics) who paused in between degrees to teach at the high school level, most commonly for two years. I have seen absolutely no stigma attached to this. On the contrary, when people have to make lightly educated guesses as to whether a certain candidate will be a good (read: not problematically bad) teacher, then holding a teaching job for more than one year pushes that guess in a positive direction. Teaching at a community college is generally viewed as a "higher level job" than teaching at a high school, so I think this should carry over. As a specific example, within fairly recent memory we took a PhD student who had been teaching at a community college. She got her PhD and now has a tenure-track job at a university. So: no stigma here.
(The above answer assumes that you have made it into a PhD program and have kept up your "higher academic skills" as much as other candidates. Whether that is the case is up to you.)
The part of your CV after you get a PhD will be viewed with much more scrutiny, both in terms of chronology and specific jobs held. If you want to get a tenure-track job at a research university, any post-PhD academic employment other than a postdoc or tenure track job at a research university will count against you. (Sorry. I don't make the rules -- I'm just reporting the situation as I see it.) Even taking an industrial job might not count against you as much as teaching at a community college: in my neck of the academic woods an "industrial job" is an enigma of blankness eliciting neither delight nor scorn -- our eyes just slide right off of it.
Let me say though, that while teaching at a community college between degrees is a possible trajectory to an academic career, it does not necessarily make sense to me for someone to plan on that in advance. That is to say: why are you contemplating taking a break of several years from your graduate studies? If the answer is because you feel a bit burned out or that you want the chance to figure out whether an academic career is for you, I think you need to know that the post-PhD academic job market is one step shy of airtight at the moment. It is not for the faint of heart or the uncertain of commitment. I suggest you think carefully about your long term goals and whether a PhD is really a necessary part of them.
(Speaking from the perspective of mathematics in the U.S., ...) I'd second @PeteL.Clark's comments... Specifically, it makes an enormous difference in the current academic job market whether "teaching a few years at a community college" occurs before or after a Ph.D. If before, then I'd estimate that people would imagine you're more focused and know what you want, so it might enhance a PhD application to all but the most elite places. If after your PhD, it would almost surely create difficulty "returning" to the "research track". It's not that the affiliation itself would harm you (and community colleges are not "universities", so are "not even" lower-tier universities), but that you would have most likely doing something other than what is expected for post-docs and tenure-track people, unless you manage to get lots of good papers written while doing that community college teaching. That's the bottom line. A secondary but still still very significant problem would be finding letter writers for your next job, back in the research-university world. In contrast, in a post-doc, hopefully there are senior faculty who interact with you and can write very helpful letters. Glowing letters from community college people will not have the same impact.
And, yes, the "affiliation" issue will impede you in some cases, if/when people filter post-doc or tenure-track applicants by most-recent-job. They might (reasonably or not) figure you've been out of the loop for a few years, and might not even look to see whether you've been publishing papers.
That is, in other words, in the U.S., a sort of "clock" starts upon getting your PhD, and one is in competition with other people who got their PhD at the same time. (In some cases, maternity/paternity leave is possible, but it is naive to believe that everyone respects this...) Taking oneself out of the game for a few years may in fact be a reasonable thing in several regards, but I don't think it is currently viewed as such.
This is a rock and a hard place situation for most PhD and MFA grads. If your ultimate goal is a prestigious teaching job but right now you need to feed your children (or just yourself) and have exhausted fellowship and other post-grad competitions and grants, for heaven's sake, do what you must do. Just know that it will take all you've got to do research while teaching so many sections. You must be self-motivating and super protective of any hours left in your day for continuing your research. But this is happening at universities, too.
I have a colleague with a PhD who is teaching 5 courses per semester at The New School, and she has written books and excellent student evals. It's shocking, given the history of that school, that they now have an all-business person running the place who thinks faculty are disposable. But it is a buyers market. The glory days of teaching a 2-2 load plus service, committees, etc. are mostly over in America. You are lucky to get a real office in some places!
A good friend who never completed her dissertation (her department was inflexible when she had a premature baby), moved back to rural Wisconsin to be near helpful parents and start a business with her husband. Business is good and she teaches at a community college. Even though she has not been affiliated with a university for 12 years she is still brilliant and the presents papers at international conferences every few years. She is an independent scholar. A lot of us are.
If you can get on as a reviewer at a journal, join with a former grad school colleague or professor to start a new online journal or non-profit institute focusing on a subject or distinguished scholar not yet receiving adequate attention, and/or continue to stay active enough and connected to the literature to submit abstracts to conferences (you as moderator and other scholars from a variety of stances, geographic areas and demographics to serve on the panel), then your chances of moving on to a prestigious position improve.
It's not a "mission" career move like teaching maths to kids in Appalachia or joining the Peace Corp, but you are preparing students for university life and that is important work.
Good luck with the job search!