How to Determine and Auto-Rotate Images?
Sometimes you aren't working from images captured by a camera and don't have EXIF data to work with. In my case, I have a project to scan and import into our digital repository tens of thousands of vintage postcards most of which are landscape on the front, and only a small percentage of which are portrait (the backs remain landscape even on these). To minimize the time spent scanning, deskewing, and cropping these they are done three-per-scan and landscape (always).
I came to this question looking for an answer to automating the detection and rotation of these portrait orientation card scans.
There are many discussions about doing this based on camera metadata. There are some examples of how to use machine learning to automatically level a photo where the camera was not held plum to the ground/horizon. I haven't found any that would help in my situation (which is not to say there aren't any, but if there are they are hard to find because of the other situations)...
EDIT 3/22/2019: Here's one https://d4nst.github.io/2017/01/12/image-orientation/
..., so I did come up with an answer that I'm going to try:
For each card front (batch processing using ImageMagick and simple scripts):
- Make a smaller jpeg version of the image (because we don't want to 200MB images to work with)
- Make three more copies of that smaller jpeg, each with an additional 90 degrees of rotation applied to it
- Submit each of these four orientations to a cloud machine learning API (I've had good luck with Microsoft's in the past)
- Analyze the responses. Chose the most detailed and/or with the highest confidence as the correct orientation
- Rotate the original full-size scan the appropriate amount and delete the four smaller jpegs.
I've tested this with one scan and in my n=1 case the API had a much longer list of tags and a better (and longer) suggested caption for the correct orientation.
Potential problems:
- The cloud provider may discontinue the API or start charging more than we can afford (when I scripted a test of metadata creation using a batch of these cards the level of use stayed in the free category).
- I think Microsoft may already rotate the image you send it for OCR purposes (to catch words written in any orientation) if they start applying the more generalized metadata AI to all the orientations as well then this could stop working (although, one would hope they would add a key in the response for best orientation guess).
- (In my situation) postcards often have writing on them at orientations not true to the image (photography studio names, etc.). If they aren't doing the above as I suspect then better OCR at one rotation could fool the script. One might have to ignore OCR results if that proves to be a problem.
- Uses bandwidth.
I had some issues getting some of the switch cases to work. Even when there was no rotation to be done, the AffineTransform would create a new image with black space in the image and would chop off some of the dimensions. Piggy backing off of the accepted answer here, I used the metadata-extractor class to determine what the orientation should be. Then I used the Imgscalr library for scaling and rotation.
The complete solution that worked for me can be seen below. Thank you Tapas Bose for the original solution. I hope that this helps anyone!
BufferedImage originalImage = Utils.prepareBufferedImage(fileUpload.getFile_data(), fileUpload.getFile_type());
BufferedImage scaledImg = Scalr.resize(originalImage, 200);
// ---- Begin orientation handling ----
Metadata metadata = ImageMetadataReader.readMetadata(fileUpload.getFile_data());
ExifIFD0Directory exifIFD0Directory = metadata.getFirstDirectoryOfType(ExifIFD0Directory.class);
int orientation = Integer.parseInt(id);
try {
orientation = exifIFD0Directory.getInt(ExifIFD0Directory.TAG_ORIENTATION);
} catch (Exception ex) {
logger.debug("No EXIF information found for image: " + fileUpload.getFile_name());
}
switch (orientation) {
case 1:
break;
case 2: // Flip X
scaledImg = Scalr.rotate(scaledImg, Rotation.FLIP_HORZ);
break;
case 3: // PI rotation
scaledImg = Scalr.rotate(scaledImg, Rotation.CW_180);
break;
case 4: // Flip Y
scaledImg = Scalr.rotate(scaledImg, Rotation.FLIP_VERT);
break;
case 5: // - PI/2 and Flip X
scaledImg = Scalr.rotate(scaledImg, Rotation.CW_90);
scaledImg = Scalr.rotate(scaledImg, Rotation.FLIP_HORZ);
break;
case 6: // -PI/2 and -width
scaledImg = Scalr.rotate(scaledImg, Rotation.CW_90);
break;
case 7: // PI/2 and Flip
scaledImg = Scalr.rotate(scaledImg, Rotation.CW_90);
scaledImg = Scalr.rotate(scaledImg, Rotation.FLIP_VERT);
break;
case 8: // PI / 2
scaledImg = Scalr.rotate(scaledImg, Rotation.CW_270);
break;
default:
break;
}
// ---- End orientation handling ----
if(fileUpload.getFile_type().toLowerCase().contains("jpeg")){
ImageIO.write(scaledImg, "jpeg", fileUpload.getFile_data());
user.setProfile_picture_ext("jpg");
}
else{
Sanselan.writeImage(scaledImg, fileUpload.getFile_data(), ImageFormat.IMAGE_FORMAT_PNG, null);
user.setProfile_picture_ext("png");
}
The pointer of metadata-extractor which Roger Rowland has provided solved the problem. I am posting it here for future reference:
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
import com.drew.imaging.ImageMetadataReader;
import com.drew.metadata.Metadata;
import com.drew.metadata.exif.ExifIFD0Directory;
import com.drew.metadata.jpeg.JpegDirectory;
public class Main {
private static String inFilePath = "C:\\Users\\TapasB\\Desktop\\MHIS031522.jpg";
private static String outFilePath = "C:\\Users\\TapasB\\Desktop\\MHIS031522-rotated.jpg";
public static void main(String[] args) throws Exception {
File imageFile = new File(inFilePath);
BufferedImage originalImage = ImageIO.read(imageFile);
Metadata metadata = ImageMetadataReader.readMetadata(imageFile);
ExifIFD0Directory exifIFD0Directory = metadata.getDirectory(ExifIFD0Directory.class);
JpegDirectory jpegDirectory = (JpegDirectory) metadata.getDirectory(JpegDirectory.class);
int orientation = 1;
try {
orientation = exifIFD0Directory.getInt(ExifIFD0Directory.TAG_ORIENTATION);
} catch (Exception ex) {
ex.printStackTrace();
}
int width = jpegDirectory.getImageWidth();
int height = jpegDirectory.getImageHeight();
AffineTransform affineTransform = new AffineTransform();
switch (orientation) {
case 1:
break;
case 2: // Flip X
affineTransform.scale(-1.0, 1.0);
affineTransform.translate(-width, 0);
break;
case 3: // PI rotation
affineTransform.translate(width, height);
affineTransform.rotate(Math.PI);
break;
case 4: // Flip Y
affineTransform.scale(1.0, -1.0);
affineTransform.translate(0, -height);
break;
case 5: // - PI/2 and Flip X
affineTransform.rotate(-Math.PI / 2);
affineTransform.scale(-1.0, 1.0);
break;
case 6: // -PI/2 and -width
affineTransform.translate(height, 0);
affineTransform.rotate(Math.PI / 2);
break;
case 7: // PI/2 and Flip
affineTransform.scale(-1.0, 1.0);
affineTransform.translate(-height, 0);
affineTransform.translate(0, width);
affineTransform.rotate(3 * Math.PI / 2);
break;
case 8: // PI / 2
affineTransform.translate(0, width);
affineTransform.rotate(3 * Math.PI / 2);
break;
default:
break;
}
AffineTransformOp affineTransformOp = new AffineTransformOp(affineTransform, AffineTransformOp.TYPE_BILINEAR);
BufferedImage destinationImage = new BufferedImage(originalImage.getHeight(), originalImage.getWidth(), originalImage.getType());
destinationImage = affineTransformOp.filter(originalImage, destinationImage);
ImageIO.write(destinationImage, "jpg", new File(outFilePath));
}
}