Can you obtain a mammoth tusk without fighting a giant?

An easy place to find several mammoth tusks is in the Halted Stream Camp. You get a miscellaneous objective from Ahmren in Whiterun to retrieve his family sword, which may be located in the cave at the camp. In the same area as the sword, there is a shelf with 5-6 Mammoth Tusks just laying there, ripe for the taking. It's slightly North-Northwest from Whiterun's main gate. The symbol on the map looks like two sticks forming an X.

The probably easiest place to find the Mammoth Tusk is right outside Whiterun, at the Khajiit camp next to the stables or in Dawnstar.
Their caravan moves frequently, so you might have to wait for them to reappear, if they happen to be "on the move".

Ri'saad the merchant who sits in front of his tent has one.

Yes, you can purchase them from some vendors, and find them laying about in other places.