How is life as a PhD research scientist/staff scientist in academia?
In my experience there are two types of staff scientists.
The first are people who go out and get their own grants. Basically, if an institution thinks you have a legitimate shot at acquiring funding, they will give you access to their grants office to allow you to submit a grant to cover your salary and lab space. If you are good enough to bring in your own funding, life is good. You can live/work wherever you want and more or less name your own salary (of course you need to bring in the grants to cover the salary). As your salary is tied to your grant(s), there is a lot of stress associated with going down this road. A six month funding gap can be disastrous.
The second are people who work on the grant of someone else. This puts you at the same risks of having a funding gap, but know you are not responsible for getting the money, but someone else it. It requires a good working relationship with the PI as they have to be able to justify why paying more for a staff scientist is needed. It terms of flexibility/security, if the PI loses funding, leaves, or retires, getting another job can be really difficult and you should not expect to really have choices in where you will end up. Often grant funding, especially NIH funding, needs to be planned 18+ months out. People just don't have unallocated chunks of funding large enough to cover a staff scientist.
Answering as an epidemiologist who knows a fair number of staff scientists, and took a good, hard look at the option myself.
When/why do people hire research scientists/staff scientists instead of postdocs, and how do the positions differ from being a postdoc?
Postdocs are supposed to grow into independent researchers and eventually leave the nest to become PIs in their own right (or do whatever else they want to do). A staff scientist is someone who will stick around presumably for a longer term. As such, they have the time to build very intimate expertise on what your lab does. For example, they might know the workings of a long running cohort study, or a complicated code base inside and out. A postdoc, on the other hand, I'd have to retrain every two years.
How is job stability? By this I don’t mean job security, so much as “How often do people end up having to find a new job?” “How easy/hard is it to find a new job when that happens?” and “Can you usually find a new job at the same university/in the same city?”
This will depend very much on the lab you're working with. I know some staff scientists who have worked in the same position for essentially their entire professional lives, as the lab itself is stable. I know some who have been "inherited" as one PI retires and another takes up their lab, long running studies, etc.
Good staff scientists are often extremely valuable, and may be able to find new jobs in the same university, but this is highly variable. For example, if you work in a somewhat specialized lab it might be somewhat harder than if you do something lots of faculty in the school work on or near.
How is geographic flexibility? Would I still have to be willing to move anywhere in the country like I would if I were looking for a tenure-track job, or would I be able to choose where I wanted to live and then look for jobs there?
Technically, you can pick where you like and look for a tenure-track job there, you just might not be successful at it. The same is pretty much true for a staff scientist position - they're not often open enough that you can just rely on bunches being available in your area, depending on where "your area" is and what the demand in your field is there.
Do people in these positions first-author papers?
Depends on the lab, but I have known a number of staff scientists with either first or last positions on papers.
You do get paid more than a postdoc... right?
Usually.