Piping stream from busboy to request post
The problem here is that you need to provide 'Content-Length' for the multipart upload manually, because request
(and underlying form-data
) can't figure it out by themselves.
So request sends invalid Content-Length: 199 (the same for any incoming file size), which breaks the java multipart parser.
There are multiple workarounds:
1) Use incoming request 'Content-Length'
request.post({
url: server.baseURL + 'api/data',
formData: {
file: {
value: fileStream,
options: {
knownLength: req.headers['content-length']
}
}
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
This will produce a bit incorrect request though, because incoming length includes other upload fields and boundaries, but busboy was able to parse it w/o any complaints
2) Wait until file is completely buffered by the node app then send it to java
var concat = require('concat-stream')
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
fileStream.pipe(concat(function (fileBuffer) {
request.post({
url: server.baseURL + 'api/data',
formData: {
file: fileBuffer
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
}))
})
This will increase app memory consumption, so you needed to be careful and consider using busboy limits
3) Buffer file to disk before uploading (just for the reference)
- express + multer - I recommend using express for webservers, it makes things more manageable, and multer is based on the busboy
- formidable