Send data by parts in node.js express
The response
object is a writable stream. Just write to it, and Express will even set up chunked encoding for you automatically.
response.write(chunk);
You can also pipe to it if whatever is creating this "file" presents itself as a readable stream. http://nodejs.org/api/stream.html
Express extends Connect which extends HTTP I believe. Looking at the HTTP API, it seems that what you are looking for is response.write
, which takes a chunk of data to send as a response body. I then believe you can use response.end
on the end signal.
the easy way to do this is to convert the data into a readable stream and pipe it to the response object. here is my nodejs code. hope this might help
var express= require('express');
var bodyParser = require('body-parser');
const JSONStreamStringify = require('json-stream-stringify');
var app=express();
var jsonParser = bodyParser.json();
app.post('/node_test_pipe', jsonParser, function (req, res) {
console.log("Request from:"+req.url);
var obj={
my:1,
two:2
}
JSONStreamStringify(obj).pipe(res);
var obj={
my:3,
two:4
}
JSONStreamStringify(obj).pipe(res);
});
app.listen(8080);
Besides using Response.write()
method to chunk send and Response.end()
to mark the end!
If you have a stream! You can pipe it directly to the Response object! And all get handled automatically!
See the example bellow for a file readable stream:
var http = require('http');
var fs = require('fs');
http.createServer(function(req, res) {
// The filename is simple the local directory and tacks on the requested url
var filename = __dirname+req.url;
// This line opens the file as a readable stream
var readStream = fs.createReadStream(filename);
// This will wait until we know the readable stream is actually valid before piping
readStream.on('open', function () {
// This just pipes the read stream to the response object (which goes to the client)
readStream.pipe(res); // <====== here
});
// This catches any errors that happen while creating the readable stream (usually invalid names)
readStream.on('error', function(err) {
res.end(err);
});
}).listen(8080);
You can do that with any Stream. Just pipe it to Respond object!
The example above come from here:
https://nodejs.org/en/knowledge/advanced/streams/how-to-use-fs-create-read-stream/
Bottom line
Use pipe! And it's nice too look and understand better!
Here the link from the doc!
https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options
The readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable.
And note about chaining
it is possible to attach multiple Writable streams to a single Readable stream.
The readable.pipe() method returns a reference to the destination stream making it possible to set up chains of piped streams
example:
const fs = require('fs');
const r = fs.createReadStream('file.txt'); // read stream
const z = zlib.createGzip(); // write stream
const w = fs.createWriteStream('file.txt.gz'); // write stream
r.pipe(z).pipe(w);
Read the doc for more!