Apple - Extreme energy usage by Spotlight
It turns out my problem is related to a bug in versions 9.9 and 9.91 of JAMF (enterprise management software) as described here: https://jamfnation.jamfsoftware.com/discussion.html?id=19626
It's unclear as to whether it's directly related to El Capitan or if it affects all versions of OS X.
As far as I know, this bug in JAMF has not been resolved yet but I have found a workaround that alleviates the high CPU usage by spotlight:
For reference, the steps I did to resolve was:
-remove all Spotlight.plist files in /Library/Manged Preferences/
-kill the actual spotlight process (it restarts automatically)
sudo rm -f /Library/Managed\ Preferences/<username>/com.apple.Spotlight.plist
sudo rm -f /Library/Managed\ Preferences/com.apple.Spotlight.plist
sudo killall Spotlight
Note that there may be more Spotlight.plist files than I have explicitly pointed out - delete all of them.
This will not work for you if the problem does not stem from JAMF!
CPU and energy usage should go back to normal at this point. As far as I know, this is still a temporary fix and the problem would come back upon reboot, upon which you would have to reapply this fix.
Haven't figured out a fix for spotlight but found a solution to disable spotlight and alleviate the CPU burn.
$ launchctl unload -w /System/Library/LaunchAgents/com.apple.Spotlight.plist
(courtesy of a coworker)