JavaScript: Writing to download stream

New solution has arrived: showSaveFilePicker/FileSystemWritableFileStream, supported in Chrome, Edge, and Opera since October 2020 (and with a ServiceWorker-based shim for Firefox—from the author of the other major answer!), will allow you to do this directly:

async function streamDownloadDecryptToDisk(url, DECRYPT) {

    // create readable stream for ciphertext
    let rs_src = fetch(url).then(response => response.body);

    // create writable stream for file
    let ws_dest = window.showSaveFilePicker().then(handle => handle.createWritable());

    // create transform stream for decryption
    let ts_dec = new TransformStream({
        async transform(chunk, controller) {
            controller.enqueue(await DECRYPT(chunk));
        }
    });

    // stream cleartext to file
    let rs_clear = rs_src.then(s => s.pipeThrough(ts_dec));
    return (await rs_clear).pipeTo(await ws_dest);

}

---

Depending on performance—if you're trying to compete with MEGA, for instance—you might also consider modifying DECRYPT(chunk) to allow you to use ReadableStreamBYOBReader with it:

…zero-copy reading from an underlying byte source. It is used for efficient copying from underlying sources where the data is delivered as an "anonymous" sequence of bytes, such as files.

This is only going to be possible with a combination of service worker + fetch + stream A few browser has worker and fetch but even fewer support fetch with streaming (Blink)

new Response(new ReadableStream({...}))

I have built a streaming file saver lib to communicate with a service worker in other to intercept network request: StreamSaver.js

It's a little bit different from node's stream here is an example

function unencrypt(){
    // should return Uint8Array
    return new Uint8Array()
}

// We use fetch instead of xhr that has streaming support
fetch(url).then(res => {
    // create a writable stream + intercept a network response
    const fileStream = streamSaver.createWriteStream('filename.txt')
    const writer = fileStream.getWriter()

    // stream the response
    const reader = res.body.getReader()
    const pump = () => reader.read()
        .then(({ value, done }) => {
            let chunk = unencrypt(value)

            // Write one chunk, then get the next one
            writer.write(chunk) // returns a promise

            // While the write stream can handle the watermark,
            // read more data
            return writer.ready.then(pump)
        )

    // Start the reader
    pump().then(() =>
        console.log('Closed the stream, Done writing')
    )
})

There are also two other way you can get streaming response with xhr, but it's not standard and doesn't mather if you use them (responseType = ms-stream || moz-chunked-arrayBuffer) cuz StreamSaver depends on fetch + ReadableStream any ways and can't be used in any other way

Later you will be able to do something like this when WritableStream + Transform streams gets implemented as well

fetch(url).then(res => {
    const fileStream = streamSaver.createWriteStream('filename.txt')

    res.body
        .pipeThrogh(unencrypt)
        .pipeTo(fileStream)
        .then(done)
})

It's also worth mentioning that the default download manager is commonly associated with background download so ppl sometimes close the tab when they see the download. But this is all happening in the main thread so you need to warn the user when they leave

window.onbeforeunload = function(e) {
  if( download_is_done() ) return

  var dialogText = 'Download is not finish, leaving the page will abort the download'
  e.returnValue = dialogText
  return dialogText
}