How do I use Etags for Youtube v3 Data API?
I figured out how to cache the data a while ago, sorry!
You can make a call to /playlists to get both the total count of items in a playlist, plus the etag changes if and only if the playlist itself changed, which is what I want. I only want to make new requests if the base playlist changed.
A call to /playlistItems always generates a new etag, regardless of changes. I think this endpoint is meant for temporary querying to figure out metadata of a video as it relates to a playlist, not for static data lookup. Playlists are very flexible and I think YouTube decided against caching this data since calls to /playlistItems are often on a case-by-case basis. It's likely their backend automatically generates an etag, but doesn't actually save anything for this endpoint.
So, these are the steps to get the total length of time of a playlist, plus caching:
- get playlist id
- lookup etag in cache by playlist id
- call /playlists with the etag in the
If-None-Match
header (should work even if etag is empty) - if the api returns 304, use cached playlist length
- if the api returns 200, save the new etag in cache
- You can do more caching!
- call /playlistItems with playlist id (with all the pageTokens)
- lookup each videoId in cache to get video length
- Cache is defined as a dictionary of
videoId:videoLength
- if videoLength not found, add videoId to a
videos
array - if videoLength is found, add to a
lengths
array
- Cache is defined as a dictionary of
- call /videos with all the video id's that are not found in cache up to 50 elements
- Could be done right after /playlistItems call or when all calls are done, I think it's ok to be lazy right now and do it right after each call
- Also you can cache video calls with etags and save that to check if the length hasn't changed, but then you would have to call the api per each video. I dunno, but I think this is over-optimizing. Still might want to keep in mind that video length can change via YouTube's editing tools when debugging
- (continued from 7) For each video in the response, cache the video length in a dictionary as a
videoId:videoLength
pair, then add length to alengths
array - Reduce
lengths
array into a moment.js duration object - Save a formatted string of the length of the playlist to cache by etag as key
- Return the formatted string of the length of the playlist
Here is the implementation on my github
I've found, that etag works correctly only when you use "part=id" and do not use "maxResults=NN". Otherwise every call to API returns new etag.
The eTag does not change every request. But you get a specific amount of different eTags for a specific request. Reason for that is that some elements inside the response change their order and therefore the algorithm creating the eTag produces different eTags. As soon as two responses have exactly the same order of their content elements the eTag will be the same again. I recorded a bunch of requests and came exactly to this conclusion. Tested with channel request by id and part=brandingSettings,snippet . If you select multiple parts with lots of nested elements in die response, you'll get more different combinations and therefore different eTags.