Normal Priority Builds Will Not Build in TFS 2010
I had this same problem where only builds that were "Above Normal" or High priority would actually build.
I tried restarting the build service, controller, and agents with no effect. There were no errors in the Event Log.
It turned out that there were 2 rows in the tbl_BuildQueue table for build definitions that I had recently deleted. Changing the status of those rows from 2 (Queued) to 16 (Cancelled) fixed the problem for me.
Here are some links that helped me figure this out:
http://networkprogramming.wordpress.com/2010/10/27/default-controller-cannot-be-deleted-because-there-are-builds-in-progress/
http://blog.hinshelwood.com/ghost-team-foundation-build-controllers