Taking photo with custom camera iOS 11.0 Swift 4. Update error

In iOS 11, you should use like this :

@available(iOS 11.0, *)
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
        let imageData = photo.fileDataRepresentation()
}

Thanks @Vini App, I tried the code it worked for me, I posted my code for image capturing and processing, hope that will help people who needs similar function in their app.

First you need to setup your video capturing device, search it google here is an example https://gist.github.com/tad-iizuka/fc35bc7835920c0b8b84e316f83e3a40

Makes sure that you need to define photoSetting at the top

...
var photoSetting = AVCapturePhotoSettings()
...

Configure the photo setting either in viewDidLoad() or viewWillAppear()

            // Configure camera
        photoSetting = AVCapturePhotoSettings.init(format: [AVVideoCodecKey: AVVideoCodecType.jpeg])
        photoSetting.isAutoStillImageStabilizationEnabled = true
        photoSetting.flashMode = .off

Then use the following function to process buffered image data

     func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {

    // Check if there is any error in capturing
    guard error == nil else {
        print("Fail to capture photo: \(String(describing: error))")
        return
    }

    // Check if the pixel buffer could be converted to image data
    guard let imageData = photo.fileDataRepresentation() else {
        print("Fail to convert pixel buffer")
        return
    }

    // Check if UIImage could be initialized with image data
    guard let capturedImage = UIImage.init(data: imageData , scale: 1.0) else {
        print("Fail to convert image data to UIImage")
        return
    }

    // Get original image width/height
    let imgWidth = capturedImage.size.width
    let imgHeight = capturedImage.size.height
    // Get origin of cropped image
    let imgOrigin = CGPoint(x: (imgWidth - imgHeight)/2, y: (imgHeight - imgHeight)/2)
    // Get size of cropped iamge
    let imgSize = CGSize(width: imgHeight, height: imgHeight)

    // Check if image could be cropped successfully
    guard let imageRef = capturedImage.cgImage?.cropping(to: CGRect(origin: imgOrigin, size: imgSize)) else {
        print("Fail to crop image")
        return
    }

    // Convert cropped image ref to UIImage
    imageToSave = UIImage(cgImage: imageRef, scale: 1.0, orientation: .down)
    UIImageWriteToSavedPhotosAlbum(imageToSave, nil, nil, nil)

    // Stop video capturing session (Freeze preview)
    captureSession.stopRunning()
}

In this function, pixel buffer is converted to image data in the format specified by photoSetting and then cropped to the size you want.

You can create a button in IB to call the capturing function above

@IBAction func onTakePhoto(_ sender: UIButton) {
    if let videoConnection = videoOutput.connection(with: AVMediaType.video) {
        // Adjust the orientaion of captured image
        let capturePhotoSetting = AVCapturePhotoSettings.init(from: photoSetting)
        videoConnection.videoOrientation = (previewLayer.connection?.videoOrientation)!
        // Save captured photo to system album
        self.videoOutput.capturePhoto(with: capturePhotoSetting, delegate: self)
    }
}