How can I delete a folder with lots of subfolders fast?

It depends on your definition of fast. The answers already here give a good solution for actually removing the directories from the filesystem, but if what you really need is to free the directory name as fast as possible, a rename on the same filesystem is instantaneous:

{ mv directory directory.gone && rm -rf directory.gone; } &

Technically this is cheating since I haven't sped up the actual deletion, but practically it's very useful: I use this trick all the time so I don't have to wait for slow deletion operations.

If your version of "find" implements the -delete sub-command, then you can try

find directory -delete

In this case:

find ~/.local/share/Trash/ -delete

Some commands, like rm, perform most of their work in the kernel. In the file-system routines, to be exact. Time spent performing system calls are accounted for in that way, so whilst your "rm" command runs for a long time, it doesn't do much work in user-land - the system calls performs most of the work.