Bash scripts extremely slow to start on OSX
I had the same issue on my work machine, running macOS Sierra 10.12.5 and 10.12.6, and I confirmed that @charles-duffy is right. My company's anti-malware is the problem. They are using a daemon called Confer by Carbon Black for endpoint security.
Run this command to see if the problem stops:
launchctl unload /Library/LaunchDaemons/com.confer.sensor.daemon.plist
I think Confer is taking a fingerprint of every new program it sees and tries to compare it to a blacklist before allowing it to run. It looks like it times out after a second (trying to register the fingerprint online?), and something is making it try over and over again, yielding longer delays of integer seconds. This makes newly modified programs launch slowly, although speeds return to normal after the fingerprint has been taken.
Example runs
Here's how the problem manifested on my work laptop. As time went on, sometimes the delay increased, but always by an integer number of seconds at a time. If Confer remembers the program from before, there will be no delay.
bash scripts
$ bash --norc --noprofile -l # make sure rc scripts don't interfere
bash-3.2$ echo exit > exit.sh && chmod +x exit.sh
bash-3.2$ time ./exit.sh
real 0m1.004s
user 0m0.001s
sys 0m0.002s
bash-3.2$ time ./exit.sh
real 0m0.002s
user 0m0.001s
sys 0m0.001s
There were no problems running commands provided to bash -c
:
bash-3.2$ time bash -c exit
real 0m0.008s
user 0m0.002s
sys 0m0.003s
awk scripts
But the problem did affect awk scripts:
bash-3.2$ printf '%s\n' '#!/usr/bin/env awk -f' 'BEGIN { exit 0; }' > test.awk && chmod +x test.awk
bash-3.2$ time ./test.awk
real 0m4.010s
user 0m0.002s
sys 0m0.001s
bash-3.2$ time ./test.awk
real 0m0.005s
user 0m0.002s
sys 0m0.001s
Compiled binaries
It even affected compiled C and Golang code:
bash-3.2$ printf '%s\n' '#include "stdio.h"' 'int main() {' 'printf("Hello, world!\n");' 'return 0;' '}' > hello.c && gcc -o hello hello.c
bash-3.2$ time ./hello
Hello, world!
real 0m4.006s
user 0m0.001s
sys 0m0.001s
bash-3.2$ time ./hello
Hello, world!
real 0m0.004s
user 0m0.001s
sys 0m0.001s
bash-3.2$ printf '%s\n' 'package main' 'import "fmt"' 'func main() {' 'fmt.Println("test")' '}' > test.go && go build -o test ./test.go
bash-3.2$ time ./test
test
real 0m4.018s
user 0m0.001s
sys 0m0.003s
bash-3.2$ time ./test
test
real 0m0.005s
user 0m0.001s
sys 0m0.002s
Starting with 10.15, macOS might actually be talking to Apple every time a new script is executed. I found this in a recent changelog for TextMate:
Fixed: Delay when re-executing (bundle) commands on macOS 10.15. Starting with macOS 10.15 Apple will “call home” each time a new script/binary is executed, this can have a delay of more than a second, depending on internet connectivity/location. The result seems to be cached per inode, previously TextMate would use temporary files when executing scripts or shell commands (giving them a new inode on each run), it now re-uses these files to avoid the delay on repeated executions.