How to filter noise from ground?

You don't filter ground. "Ground" is the reference voltage all other voltages are measured to, so it has no noise by definition. (There are cases in distributed systems where what exactly "ground" is is not clear, but that's another issue)

USB power can certainly be noisy. You would definitely want to filter it before it powers audio circuitry, since audio generally requires high signal to noise ratio. Just a LDO to make 3V is probably not good enough. The LDO will reduce some of the noise, but some will be too high frequency for its active circuits to deal with properly. You need to put some low pass filtering before the LDO. This filter needs to eliminate the frequencies the LDO can't handle properly.

I would start with two ferrite chip inductors in series and a 20 µF cap to ground after each. Then connect the input of the LDO to that, and the output should be pretty clean. You can also just low pass filter the 5V and use it dircectly to power the audio circuit. A typical audio "line" out level is around 1 V, but you need headroom for higher peaks. Even a 5V supply doesn't leave much headroom.