Are there other commands like scp but for deleting files and folders?

The ssh command will allow you to execute pretty much any command on the remote host, e.g.,

ssh yourlogin@remotehost rmdir somedir

where in this example somedir is relative to the home directory of yourlogin.

Rsync will copy an entire path/tree and check files already at the destination and not bother copying them if they're unchanged. If you use the --delete option, it will delete any files that are at the destination that are no longer at the source. It works over ssh

If your remote SSH server supports the SFTP subsystem (most do, unless it's been explicitly disabled), you can use the sftp shell (or any other SFTP client) to manipulate files and directories on the remote server.

Unfortunately, one thing the basic sftp shell apparently does not handle is recursive directory removal; in order to delete a directory, you have to delete all its contents first. Most other clients (especially graphical ones) typically do support it, though.

(You don't mention what your local OS is, but if it's Linux, you can use your native file manager as GUI SFTP client by pointing it at a URL like sftp://user@host/path/. For Windows, WinSCP is a decent client, and FileZilla is a good cross-platform solution.)