How to remove only intent-to-add files from index in git?

No. Git doesn't keep track of which files were added with -N and which were simply empty.