Unable to set AudioSession inactive with AVPlayer

A quick way to lose the lock screen controls is to make your audio session mixable (without deactivating your audio session):

try! AVAudioSession.sharedInstance().setCategory(.playback, mode: .default, options: .mixWithOthers)

Remove the .mixWithOthers when you want them back.

Another, perhaps more palatable way is to remove your MPRemoteCommandCenter targets once you pause playback. e.g. if you set it up like this

var playTarget: Any? = nil
var pauseTarget: Any? = nil

func addRemoteTargets() {
    let commandCenter = MPRemoteCommandCenter.shared()

    playTarget = commandCenter.playCommand.addTarget { (event) -> MPRemoteCommandHandlerStatus in
        print("play \(event)")
        return .success
    }

    pauseTarget = commandCenter.pauseCommand.addTarget { (event) -> MPRemoteCommandHandlerStatus in
        print("pause \(event)")
        return .success
    }
}

Then remove the targets like so:

func removeRemoteTargets() {
    let commandCenter = MPRemoteCommandCenter.shared()
    commandCenter.playCommand.removeTarget(playTarget)
    playTarget = nil
    commandCenter.pauseCommand.removeTarget(pauseTarget)
    pauseTarget = nil
}

p.s. these are "seems to work!" answers. I don't know how you are supposed to determine when an AVPlayer has finished its audio I/O (apart from polling via setActive:false failing). My reasoning is that becoming the lock screen/"Now Playing App" has 3 pieces (this is sort of documented somewhere, this is the closest thing I can find to that right now, although the most explicit documentation of how this works is in the system logs):

1. a non mixable audio session category
2. MPRemoteCommandCenter integration
3. current or "recently" finished audio IO

so if you can remove any one of these pieces, the lock screen controls should go away.