Can I trust a security hash implementation after testing it with random inputs against another implementation?

Assume the library author can be trusted. Given my random input test above: how likely is it, that the author accidentally introduces a bug with the effect that there are inputs for which a wrong hash is calculated?

Being trusted does not imply that the author of the software is a competent programmer which knows all possible pitfalls. This means if you rely only on trust you cannot know the likeliness of introducing a bug. And since such bug might only occur in rare cases like a race condition or some integer overflow there is no guarantee that you will trigger the bug with your random test cases.

Assuming the library author cannot be trusted. Given my random input test above: how likely is it, that the author has purposely introduced a backdoor of some kind?

If the backdoor gets only activated with a specific input you will never find out without thoroughly inspecting the code. And even code inspection might not help, see the examples from the Underhanded C Contest. This means it is possible to introduce such a stealth backdoor. But again, you cannot give a specific likeliness for this solely based on the information that the author is not trusted.

Can I trust a security hash implementation after testing it ...

Based on the previous observations this question can not be definitely answered. Apart from that it depends a lot on what you use the library for: if it is just for getting some checksums in order to detect accidentally corrupted data your tests might be enough. If is used instead for purposes where software failure might lead to death, leakage of top secrets or malware infection of critical infrastructures than such tests are probably not sufficient, especially if you don't trust the author.