Converting 4-band TIF to PNG while keeping all 4 bands?

This is a limitation of the PNG format. It only has 3 information channels (RGB), so one of your bands will be suppressed. If you really need to, you can save your NIR band as an alpha channel, but beware - you won't be able to access it easily. Neither QGIS nor ArcGIS allow allocating the alpha channel to one of its display channels. The information will still be there, but you'll only access it through image processing libraries.

Why PNG, though? This is quite a non-standard for geographic images (for starters, you don't retain georeference). If you need a small file size, consider ECW or MrSid. If you need to open it in non-GIS software, GeoTiff is a good alternative, as TIF is supported by pretty much every image software. If you really need it saved as PNG, though, what you can do is save one image with red, green and blue bands in the RGB channels, and another image with, say, red, green and NIR in the RGB channels. Then reconstruct them into one single multiband image later on. Even in this last approach, JPEG2000 is the better alternative, as you don't lose spatial information.

I will just answer my own question here, if anyone runs into the same problem with multiband images and TensorFlow.

I ended up using Keras on top of TensorFlow and instead of feeding the image files into the network, I converted my tiff-files to numpy arrays and appended all of them to the same array, which I saved as a .npy-file. In this way I avoided the trouble of multiband PNGs altogether.