What are some good ways to get rare materials?

There's no good substitute for just playing the game, a lot.

Time is what you need to invest in.

Specifically, here's what I do in order go get the upgrade materials:

• Spinmetal: Patrol on earth, go to mothyards and run circles in the zone, kill all enemies, pick up spinmetal out in the open and visit every cave and spawn area to look for chests. Chests usually contain 1-2 of the appropriate upgrade material for the area.
• Relic Iron: Patrol on mars, go down to the left through the tunnels to the dome building beyond and start a loop there:
  • Enter the dome, kill everything inside (sometimes a yellow colossus or a couple of yellow psions so some money as well), then visit the following chest areas:
    • Inside spawn area back right
    • Out of the spawn area back right and hold right until you're back inside another spawn area just next to it
    • Just outside that spawn area, against the separating wall just outside, to the right, there can be a chest
    • Exit the room with the separating wall and go immediately right, behind a low wall there can be a chest and in the corner there can spawn a relic iron node
    • Out on the half-circular walkway, to the right when you're looking out of the building, against the wall behind some large mechanical equipment there can spawn a chest
    • Below the half-circular walkway, to the left when you're looking out of the building, on the sand dunes there can spawn both relic irons and chests in various positions
  • Go across to building five, out of the dome building and 90 degress to the left, run straight forward and there is a spawn area in the bottom of the building
  • Go out of that spawn area and run across to the other entrance to the dome, can be a relic iron outside to the right of the doorway, enter inside and kill the two enemies and check the spawn area behind them
  • Go out of the dome area and turn slightly left, run across to the next building to the left of the silos and enter the spawn area there, can be a yellow enemy or two outside as well
  • Exit the spawn area and turn 90 degrees left and go into building five behind the silos and kill everything in your path, straight ahead there is a spawn area
  • Up top in building five there is a spawn area, behind the yellow colossus
  • Exit building five topside through the broken wall and head back into the dome to start over
• Spirit bloom: Patrol on venus and take the path to the right. Stop at the building and run a loop checking the following areas:
  • Spawn area in the first buildings on the right
  • Run a circle around the very first building on the right, along the broken tiled ground and check for a couple of chests that can be located either on it, against the wall, or just below it, typically on the very outskirts of the map. There's also a few spirit blooms that can spawn in some of the vegetation here.
  • Check the corners of the first area, there can be spirit blooms here
  • In the first area, when you start running towards the next area, there is a stairway with a low wall on the left, there can be a chest behind this wall.
  • Below the stairway there is a spawn area on the left.
  • Out among the cars in the battlefield there can spawn a few spirit blooms, typically next to vegetation
  • Run to the next area, the area with the billboards, a few cars, and the double stairway up to the entrance to the path to the academy. There can be spirit blooms in the corners here.
  • Also next to this area there is a spawn area that can contain a chest
  • Run to the next building on the far side, the one where the servitor and a few fallen spawns. Check the spawn area and behind all the pillars in the room, and also the room on the top of the stairs here, chests can spawn a lot of places here.
  • Run back and start again, everything should have spawned by now.
• Helium filaments: Patrol the moon and go to the mushroom-like building (sorry, I forgot what its called). Start a loop going from this around the mountain to the refinery-like building and back, killing everything in your path and checking every corner and spawn area for nodes. Apologize for not having a clearer list of spawn areas here.

That should take care of the basic grind.

There's also a couple of other ways to get the basic upgrade materials:

• You can get them when turning in bounties, so do your bounties every day if you can
• You can sometimes get them as an extra reward when completing the daily heroic mission or the weekly heroic strike
• You can buy them from either the crucible quartermaster or the vanguard quartermaster for 10 marks of the respective type. You get 20 units of the upgrade material of your choice.

When it comes to the more hard to get upgrade materials, ascendant energy and ascendant shard, you need to:

• Do at least one public event every day, the first one will often give you 1-2 of these items
• Decrypting blue and legendary engrams with the cryptarch will sometimes decrypt into one of the ascendant items instead of a weapon or armor. You can also get strange coins or motes of light.
• The odd chest will give you this, but this is rare

The final material, exotic shards, can be obtained the following ways:

• Dismantle exotic gear
• Buy one from Xûr for 7 strange coins

Apart from what Lasse said:

Strange coins:
Doing Nightfall and Weekly Heroic Strike is a most efficient way to get those. Weekly on max difficulty is a guaranteed 9 coins at once. Nightrall is a bit more random, but has a chance to give coins, exotics and old ascendant materials (and a weekly exp bonus!). See more on wiki.

Ascendant materials (old ones):
There are plenty of those in Vault of Glass raid (and some good loot too). If you can find a team that knows what to do, then it doesn't take much time to run it. It has 4 loot drops and 3 chests (4 if you can beat Templar without teleporting). So you can end up with plenty of materials. See more on wiki. If you don't have a group, then try Destiny LFG, there was an official Bungie blog post about this site, it helped lots of people to find a team.

Also, if you're short on ascendants but have some spare legendaries or exotics, then don't forget to level them up completely (no need to purchase perks) - legendary guns have a chance to give up to 6 Energy on dismantle (up to 3 if vanilla), and any such exotics will give 2 shards (only 1 if vanilla).

Vanguard Marks
Vanguard marks are best farmed from strike playlist. The best one right now is Roc - it gives rep, marks and guaranteed engram. If you do those, not only you get your 100 weekly marks quick, but you also get lots of engrams (basically you increase your change of getting legendary gear through sheer number of engrams you get, and cryptarch packages when it levels up). If you don't have The Dark Below, then Tiger playlist is your best bet, it has less rewards though, but same amount of marks.

This is one of those places you should be choosing your battles. Unfortunately it isn't really an option to fully upgrade (using ascendant) every legendary and exotic you have or come across.

So make sure that the items you do fully upgrade are the important ones. Test them out and make sure they have modifiers that are worthy of the materials.

Once you have identified the worthy, here is some general strategy.

• Do the daily heroic every day. It will yield 2 ascendant materials and these add up quickly.
• Don't be a hoarder! Dismantling less than desirable legendaries yield ascendant materials. Weapons will yield 2-3 energies, and Armor will yield 2-3 shards. This excludes legendaries that are obtainable by glimmer alone. If you have time, level your weapons prior to dismantling them. Do not spend material on them, but gain experience on them so that all skills are unlocked. This will double the amount of materials yielded (5-6 energies for legendary, 2 exotic shards for exotic), and if you fully unlock (without actually purchasing) all the circles you will also receive a telemetry for that type of weapon.
• Do the weekly heroic because it will give you 9 coins. Do the weekly nightfall as well, because it essentially guarantees a legendary, exotic, or sometimes up to 11 ascendant materials.
• You can solo the first chest of the raid!! What? No. yes. Load into the raid, make it to the loading circle that spawns the bridge. Cross the bridge. Enter the darkness crazyness. When you load into the darkness you will see a path and a light beacon looking thing. While aimed at that, go in the direction of 10 o'clock (as in to the left a decent amount) and run straight until you get the wall. There will be a door that opens to a closet sized space that can have a chest in there. If it is not there, die and try again. It shouldn't take more than 5 tries. Only good once per week.
• So, you did all of this? Good job Guardian! :) Now make another character than can also farm the material by getting carried by a friend so you don't have to invest in them. This will double or triple your weekly income of materials.
• Motes should be easy to come by, and in the process of all of the above you should have leveled to generate them or found a few at the cryptarch. Really, motes are only good for gambling with Xur (and even then, not terribly useful after you are all gambled out).