Unable to boot device due to insufficient system resources using Xcode 9

Following command helped me out. Try pasting the below command in the terminal and it should solve your problem.

sudo launchctl limit maxproc 2000 2500

You should increase the allowed running processes for your system. You can modify the limits through Terminal. The command for that is the following:

sudo launchctl limit maxproc [limit for one user] [total limit]

Inside Simulator Help, Apple's example values are 2000 for one user and 2500 total.

After setting up the values, quit iOS Simulator, then restart Xcode and try launching a simulated device again.

However, keep in mind that this can slow down your system and make the OS unstable. Since these values are only permanent until the next restart, you should reboot your system and it will restore these values to default.

From Simulator Help:

WARNING: Setting the the maximum number of processes to a number that is too low can prevent your Mac from operating correctly. Restart your machine to restore the original limits.