Should I be concerned if the "FBI" has logged onto my Ubuntu VPS?

An IP address can be set up in DNS to resolve to any host name, by whoever is in control of that IP address.

For example, if I am in control of the netblock 203.0.113.128/28, then I can set up 203.0.113.130 to reverse-resolve to presidential-desktop.oval-office.whitehouse.gov. I don't need control of whitehouse.gov to do this, though it can help in some situations (particularly, with any software that checks to make sure reverse and forward resolution matches). That wouldn't mean that the president of the United States logged into your VPS.

If someone has access to your system, they can change the resolver configuration which will effectively enable them to resolve any name to any IP address, or any IP address to any name. (If they have that level of access, they can wreak all kinds of other havoc with your system as well.)

Unless and until you verify that the IP address that was used to log in actually is registered to the FBI, don't worry about the host name being one under fbi.gov. That name mapping may very well be faked. Worry instead that there has been a successful login to your account that you cannot explain, from an IP address that you don't recognize.

Chances are that if the FBI wanted the data on your VPS, they would use a somewhat less obvious approach to get it.

You should worry, but not about the fbi.gov hostname.

Go read How do I deal with a compromised server? on Server Fault, and How do you explain the necessity of “nuke it from orbit” to management and users? here on Information Security. Really, do it. Do it now; don't put it off.

I think you MUST be concerned if anyone has unauthorized access to your server. As others mentioned there isn't much work for faking reverse DNS host name. Maybe they want you to believe it's okay for a government agency to have access to your server so you won't investigate the incident anymore.

You should backup all your server logs for later analysis and preferably rebuild your server to eliminate any risks that a compromised server could cause. After that you (with help of an expert) should setup server with security best practices and precautions.

So should you be concerned if it was the FBI, or is it ok if it was just some casual hacker? From the logs, someone successfully logged onto a host you control. It should be assumed compromised regardless of who it was. Scrap it and rebuild.

Also keep in mind that a reverse DNS entry can be created by anyone who has control of a specific IP block. It doesn't need to resolve to something they control, ie, if I control an IP block I can create a reverse entry to whoever I choose. Reverse and forward entries don't have to match, and they are often maintained by different people.