Shell script attack on Apache server, via an cron of unknown origin
After OP had added logs, it become clearly, that problem is in Remote Code Execution exploit for Struts 2 (CVE-2017-5638).
Some additional links:
- New Struts2 Remote Code Execution exploit caught in the wild.
- CVE-2017-5638 - Apache Struts2 S2-045.
Solution is to upgrade your Struts to version 2.3.32 or 2.5.10.1.
I've faced similar issues before when I was sysadmin. I think you must distinct if it's your tomcat server or your Java app.
When you start tomcat without the "infected Java app" is the cron getting enabled? (I mean, deleting your application from Tomcat and starting it) If so then you have a bigger problem, you'll need to verify the startup scripts and every application deployed in the tomcat server.
Otherwise we are sure your app is the issue.
If that's the case go to:
$CATALINA_BASE/webapps/your_app
Verify the integrity of your application, is there additional files that you do not recognize?
Now go to the webapps directory of your tomcat installation:
$CATALINA_BASE/webapps/
In that directory perform:
grep -R '91.230.47.40' *
To find the possible file/line-of-code which causes the infection, it could be a file of your app or a new one.
Do you have your code in a CSV system?
Build the war file outside the infected server from your CSV repo and do:
md5sum your_app.war
Remove your application from the tomcat server and re-deploy, verify that you're uploading the correct war through md5, then check if the crontab is being invoked.
If you provide feedback on this steps I'll be glad to help.
We just had to fight this kind of attack off on a server, it kept restarting overwriting crontab for our tomcat user as described above. The IP address was identical. Grep of entire webapps dir for IP address did not reveal a culprit.
In our case we do not use struts, but we had the "host-manager" and "manager" apps in webapps, and we had JMX enabled/port open. Restarting without those seem to have solved, so my hunch is that the vulnerability might be in one of those. Specifically there was a JMX vulnerability fixed in 7.0.73 that might be our culprit (https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.73).
Another precaution we are now taking is to restrict access to wget and chmod to root only (just do chmod 770 on those binaries).