node.js - NodeJS stream response to prevent out of memory errors -

i have following code:

 response.writehead(200 , {    'content-type': 'application/vnd.ms-excel',    'content-encoding': 'gzip',    "content-disposition": "attachment; filename="+getfilename() });  var zlib = require("zlib"); var data = getdata(); //some arbitrary data method var outputstr = ""; var header = "some header"; var footer = "some footer";  outputstr+= header;  for(i=0;i<data.length;i++) {   outputstr+= data[i].utils.generatestring(); //xml parser , various whatnots resulting in string }  outputstr+= footer;  zlib.gzip(outputstr, function(err, result) {   response.end(result); //works on small strings }) 

sometimes, string length exceeds 512mb limit (which fine), thus, on 32 bit system, i'll get:

fatal error: call_and_retry_0 allocation failed - process out of memory 

i've been reading streams, examples talk abour reading files, rather generated values.

how can create stream , push data during iteration (my loop) instead of writing inside string , pushing response.

it's important output invoke file download, same today, along gzip support (which asynchronous so, needs taken consideration)

you want create streaming response, create gzip stream outside of convenience functions.

example:

var zlib=require('zlib'); var fs=require('fs');  var gz=zlib.creategzip(); var res=fs.createwritestream('./test.gz');  gz.pipe(res);  var data=['hi','test','data']; var i; (i=0;i<data.length;i++) {     gz.write(data[i]); } gz.end();  // implicitly call 'res.end()' default