Is there a way to delete an Apex Class using the Salesforce DX CLI?

You can now use the force:source:delete option to delete metadata from non-scratch orgs. See the documentation for full usage, but here's an example:

sfdx force:source:delete -m ApexClass:SomeClassName -r

For scratch orgs, delete the local files, then use the normal push command:

sfdx force:source:push

Original Answer

If, and only if, you're using scratch orgs, deleting a class in your repo and then using force:source:push should result in a deletion of the components that have been physically removed from the repository. For non-scratch-org deployments, you still need to use the normal force:mdapi:deploy function with a destructiveChanges.xml file.

Unfortunately, per the CLI Reference Docs, as of now there is no command that allows you to delete a class without using the metadata API. You have already mentioned using the metadata API/destructive changes, which is a good way to accomplish this.