iOS 9 play audio in silent mode but keep other app's music playback running

The Audio Session Category AVAudioSessionCategoryAmbient will obey the Ring/Silent switch as stated in the documentation -

AVAudioSessionCategoryAmbient The category for an app in which sound playback is nonprimary—that is, your app can be used successfully with the sound turned off...

... Your audio is silenced by screen locking and by the Silent switch (called the Ring/Silent switch on iPhone).

I suggest trying a better suited category with mixing options like -

[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback 
                                 withOptions:AVAudioSessionCategoryOptionMixWithOthers 
                                       error:nil];

AVAudioSessionCategoryPlayback category will ignore the silent switch as mentioned in documentation -

When using this category, your app audio continues with the Silent switch set to silent or when the screen locks.