Android - YouTube Offline Videos deleted/removed

The "offline feature" rather counts as cache (note that content downloaded for offline use will be available this way for 2 days only), so it's not considered "your data". You performed a factory-reset. From our factory-reset tag-wiki:

What a factory reset does is basically to "wipe" (delete all content) from the /cache (holding the application cache) and /data (user data and user-installed apps, dalvik cache) partitions. Other media, like internal-sd and external-sd cards, are usually left untouched – though some devices/ROMs offer options to include them with the wipe.

So that includes your "files cached for offline use".

Also, as correctly pointed out by GiantTree, your comparison with several apps does not apply: Google Drive stores your data in the cloud (not on the device), same for most other "Google Apps". Nor do photos made with a camera app get wiped usually, because they are not affected by a factory reset (see above quote for SD cards).

I experienced the same problem in August. I had to subscribe to Youtube RED to continue playing these videos offline. I thought it was free, but it's not.