What do I do if I cannot give a good reference to my PhD student?

A student who did the minimum during their PhD has very little chance of getting a prestigious fellowship. In the sciences, you must have publications to get prestigious fellowships; a student who did the minimum will not. Your role as a mentor is to guide students to attempt things they might succeed in. Tell the student they will not succeed in this application. Tell them to compare their record to past successful applicants.

You should not write the letter. It will not help the student and it can hurt your reputation. But you should have told the student that they would not get a letter for prestigious fellowships when you discussed their goals for completing their PhD.

I'd think you should have a candid discussion with the student about what they think you should write.

Yes, (as in another reasonable answer here), university administrations will never tell you you'd done enough, nor will funding agencies, nor even will departments when it comes time for salary raise consideration. Right, so one should avoid being driven by external approval, in some regards.

At the same time, the/one ideal of academic function is self-direction, and taking lots of initiative, regardless of bureaucratic pushes. Not "sacrificing family life", necessarily. But, also, not necessarily forgetting about everything after a 40-hour work-week, either. Much more amorphous. (Fortunately, my own family is fairly indulgent of my endless distraction [sic] by math stuff... partly, because I do manage to pay attention to them and participate, in a complicated way of integrating family-and-math.)

As a sort of diagnostic, you could ask the student to "persuade you", on a professional level. If they can give cogent reasons that everyone benefits from their choices about life/work balance, then it's a winning situation. If their notion of "balance" is more of a negative about the work part, you can/should point out that they are failing to offer a good exchange for such fellowships... and much other funding.

Btw, for my own PhD students, I certainly do not try to micro-manage their schedules or time allotted. If they say that they don't have time to do something for a day or two, I believe them. If they say that a family vacation will take them away from work for some days, I believe them and it's fine. Fortunately (for me and for them) no one has ever said to me that they definitely wanted to aim to limit their interaction with the mathematics, somehow thinking that "thinking about math" is in conflict with "being a good parent/partner/friend". I myself honestly do not see this supposed distinction as genuine.

Perhaps you can provoke your student to think a little about a less naive conception of "work-life balance", and then ask them why they think you should write a (helpful/supportive) letter for them?

EDIT: In addition to myriad other complicating issues, we should definitely note that (at least) the NSF currently cares about "broader impacts" and such. It's not literally about work/life balance, but is about impacts of one's work outside one's office/classroom and so on.

My reading of your question is that you don't think that you can actually recommend the student to your colleagues. It is a separate discussion whether we think that the student was right or wrong prioritizing family, so I'm simply going to address the question of the recommendation, devoid of whatever reason you might have for not wanting to recommend them.

The problem ultimately comes down to where your allegiance lies or should lie. You're stuck between your professional ethics to only write letters that are truthful, and your personal allegiance to your former student. That is an uncomfortable position to be in, but one every faculty knows. My take is that your professional ethics provide the overriding objective for the same reason as we would expect a professional engineer to not sign off on a bridge design they know is faulty, even if their employer is generously paying them (or even if the CEO of the company is the spouse of the engineer). What respect would we, as a profession, command if we had no ethics?

So, then, how do you find a way to do the professionally right thing in a confidential way? The usual approach is to write a letter that is short and says nothing. We have all seen such letters and we know what they mean.

Now, that is ethically correct but personally not satisfying. The solution to that is probably to have a candid conversation with the student about the fact that you cannot unconditionally recommend them for the position and that they might be better off asking someone else for a letter. I've had to have these conversations, and they're not pleasant, but students in this situation generally know that they might not be the best qualified ones, and respect the ethical argument.