File.delete() does not completely delete image blank image file left behind

This isn't a direct answer to your specific question, but I'd like to propose a different work flow that may avoid the problem entirely.

When you first take the picture, either keep it in memory (use BitmapFactory.decodeByteArray instead of BitmapFactory.decodeFile), or write the file to a temp file (see File.createTempFile). In either case, the idea is to not write the file to the gallery's directory.

Then, if and when the user chooses 'save', write/copy the file to the to the gallery's directory. If they choose 'delete', delete the temp file (or don't, and let the OS clean it up).

Once you write the file (save), update the gallery with the one specific file using

sendBroadcast(new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE, Uri.fromFile(myNewFile)));

as mentioned in How can I update the Android Gallery after a photo?