problem running xp_cmdshell when Application Name set in connection string

You could write an app to try and monitor what's happening on the device, but it would require elevated permissions for your app like running as System or requesting a large set of permissions, so that you could query the appropriate services and access system resources to guess at what other apps are doing. The packagemanager exposes api calls for determining the installed applications and even examining their permissions, At best though like mentioned in the earlier answer all apps run in their own instance of the dalvik vm. Access to resources on the system are also restricted by permissions both DAC and capabilities (in 4.3). Examining permissions or trying to see what apps are doing would be difficult, especially in the case of malicious apps that abuse poorly written legitimate apps

Even Google's measures for detecting malicious apps aren't foolproof though this quote is from a presentation at defcon

In another bit of disturbing news related to Young’s research is the ability of an explicitly malicious app to get into the Google Play store and hang around for a while. According to reports, Young created an app to test the vulnerability and uploaded it to the Google Play store where it was clearly marked as malicious and users were warned not to install it. During the month it was available, until a user reported it, Google’s Bouncer service did not flag it as malicious. It is not clear whether the app was ever scanned by Bouncer or whether it “passed” the scan, but neither option reflects well on Bouncer’s effectiveness. Young also reported that most antivirus products for Android devices failed to detect the app as being malware, though one unnamed privacy application flagged it as having account access.

Modifying the platform would be the best way to try and detect malicious apps. SE for Android is project aimed at adding security and MAC to the android platform some of the concepts that they are exploring might be of interest.

Restart you Mac and try resetting the PRAM:

Hold down the ⌘+⌥+R+P keys until you hear the start up chime two times.

You can also try to reset the 'System Management Controller (SMC)' as described here: http://support.apple.com/kb/HT3964

Take note of the following description:

"After performing normal troubleshooting, these symptoms may indicate that an SMC reset may be necessary: [...]

Video

A computer that supports target display mode does not switch into or out of target display mode as expected.

A computer that supports target display mode switches into or out of target display mode at unexpected times."