Doing a math PhD with a great deal of student debt

A few things to address. First, note that a Ph.D is not a good financial choice. There are plenty of articles you can Google for, I'll link one but there's plenty more. Another thing to note is that the academic job market isn't that great, there is a problem with many people getting stuck in postdoc and adjunct positions, though there is data to show it's not as bad for mathematicians. If you are choosing to do a Ph.D, make sure you understand that it will not help you increase your wealth, but you're choosing this because it's a career path you wish to pursue.

Keeping this in mind, you choose to do a Ph.D and it's hard to find money. Financial advice helps, but when your income is low, the best help is to find more income and spend less money. What can you do? One thing to note is that in many ways you have your hands tied. Many times fellowships and departmental funding will have a requirement that you cannot have other sources of income, so having a night job is usually not an option (speaking in terms of the US). Thus you have to make do finding small amounts of help every way possible (always check with the administration that it's okay!). Small savings add up. Here are some things I found:

• Find free food. Find the seminars which have food, and go. You can also use this to network at the same time, and/or use this as your downtime.
• Find extra teaching / tutoring jobs. If your funding is a fellowship, they may let you TA as well for a bit extra cash. If you are already a TA, you can tutor local K-12 students in math for quite good pay. Another good place to look is at on-campus workshops that could use your talent. You can tutor / help out with programming workshops and other on-campus initiatives. These normally will pay minimum wage, but it also adds to your CV and again, everything adds up.
• At my university, there are some paid positions in the graduate student council. It's an interesting experience where you will have to take some time lobbying state governments and other outreach activities. Again, builds the resume and brings in some cash.
• Live on-campus and don't have a car. Use the fact that you need rides as time to catch-up with people. You'll be doing math so often that it will be nice.
• Try to get a living wage by doing something applied. This depends on your interests, but one thing that most mathematicians don't know is that salaries in different departments can be vastly different. At my university, students in the School of Biological Sciences get almost double the pay of math students. This is due to differences in salaries set by NIH vs NSF. Could you do the same kind of research in a Mathematical/Computational Biology program? Or find an adviser that could pay for you at a salary closer to bio students? This can make a huge difference. Even if you want to do something "pure", take a good look into this. There are applications for any math if you look hard enough, and so it's a balance between choosing to do what you want to do and what the general public thinks is more valuable (whether they are right or wrong...). It's at least something to think about.
• Find award money. Many times there are monetary awards for best posters, best presentations, etc. Most of these are more "outreach" types of presentations, i.e. not for mathematicians, but some university-wide project partnered with industry. Look out for these and have a cool "my math for the general public" presentation ready.
• Eat cheap. Get a big rice cooker and have that be the big part of each meal. It'll help cut down on costs.
• Don't do it alone. Live with friends / family / significant other. Eat together. Buy in bulk from Costco with others and share the savings.
• Plan ahead for summers. Plan to do extra teaching, or look into things like GRIPS which can give a supplemental summer income if you have a summer fellowship. You'd have to check with your adviser and the administrators that this is okay with the terms of your fellowship.
• Understand your health benefits. Many universities give you health care as part of the funding. In addition to the standard health benefits, my university has extra goodies like an on-campus dental care where you can get a free dental cleaning every 6 months. Using these kinds of services will help you stay healthy without spending much.

In the US, if you have federal student loans, you can defer the loan during graduate school, meaning that you don't have to make payments as long as you're in graduate school. However, depending on the details of the loan, interest may still be accumulating, and you may end up making bigger payments when you eventually start paying off your loans.

This kind of deferment doesn't apply to private loans, and you should think carefully about what it will mean in terms of your payments after graduate school, but lots of students take advantage of deferment.