Download lengthy data as a CSV file

Problem

I need to download the contents of a table shown in the view as a CSV file. I have code to format the contents of the table properly so that it can be stored as a .csv file (for example, the column separator is a comma, each record of the table is in a new line).

So, in my controller, I have the following code:

window.location.href = '/download?data=' + encodeURIComponent($scope.csvString);

I am using NodeJs for the server part. Here, I have the following route handler:

app.get('/download', function (req, res) {
    var data = req.query.data;

    res.attachment('data.csv');
    res.setHeader('Content-Type', 'text/csv');
    res.end(data);
});

This works perfectly fine. The download works and the data in the file downloaded is in perfect CSV format. No issues.

The issue arises when there are too many records in the table, that is when the data is huge. Due to this, the query parameter is really huge and I assume that it exceeds the limit and thus, the GET request returns with a 400 (Bad Request) error - note that according to my analysis, the request never reaches the server, instead I observed that the request failed using the Chrome Developer Tools. In the browser, the URL simply is www.my_domain.com/download?data=the_lengthy_data. That's it. The web page is blank / empty and no file is downloaded.

Thus, I wish to know how I can download the lengthy data? All examples that I came across do a GET request and thus I have to pass the lengthy data as a query parameter. Is there some other approach?

Problem courtesy of: callmekatootie

Solution

Not really an answer to your question, but from the looks of it, you are trying to "force" a download of data generated on the client. Then in this case, you could create a data URI, that has the data directly in it.

Your angular JS template:

<a href="data:base64,{{btoa(csvString)}}" download>Download CSV</a>

Your controller:

$scope.btoa = function (val) {
    return btoa(val);
};

As a result, your DOM should look like:

<!-- The link contains data to the CSV string "a,b,c" -->
<a href="data:text/csv;base64,YSxiLGM=" download>Download CSV</a>

And clicking the link downloads the CSV file.

EDIT

You could also force download of strings using a library called FileSaver.js.

So as usual, you would get the CSV, but this time, you would convert it to a BLOB.

var csvBlob = new Blob([$scope.csvString], { type: 'text/csv;charset=utf-8' });

And then, you can call FileSaver.js' saveAs function.

saveAs(csvBlob, 'mydata.csv');
Solution courtesy of: Salehen Rahman

Discussion

we need to change nodejs code as below. Nodejs code

    function downloadableFile(req, res) {
        res.attachment(document.title);
        var bufferStream = new stream.PassThrough();
        // we read the file in buffer.
        bufferStream.end(new Buffer(document.buffer));
        return bufferStream.pipe(res);
    }

Angularjs do not understand the binary data. so at front end we can use third party tools like FileSaver.js'

var csvBlob = new Blob([$scope.csvString], { type: 'text/csv;charset=utf-8' }); saveAs(csvBlob, 'mydata.csv');

or we can use angularjs code like this. Angularjs Code.

    downloadDocument = function (path,id,type) {

    $http.get(path).then(function (data) {
    if (data.status === 404) {
        mpToastService.showToastHeader('error', 'Error downloading document');
    } else {
        var blob = new Blob([data.data],{
        type: type + ";charset=utf-8;"
        });
        $("#" + id).attr("href", window.URL.createObjectURL(blob));
        document.getElementById(id).click();
    }
    });
    }    

    //In html file, we need to include these two lines.

    <a ng-click="downloadDocument(document.path,document.id,document.mimeType)" >document.title</a>    
    <a id="document.id"></a>

This angular code will download the binary data at front end and then pop up download modal without any data limit in popular browsers. But generally this is not the good practice except in case of client special requirements.

Discussion courtesy of: Atif Hussain

You're right about the character limit in the URL. Generally, lengthy data is passed through the body as this has no character limit in theory.

However, you should avoid trying to pass a body in a GET request. The XMLHttpRequest specs on the send method say - "If the request method is GET or HEAD, set data to null.".

So I suggest, you use a POST request and pass your parameters in the body. You'll therefore also need to rewrite the handler using app.post('download', ...) and subsequently parse request.body for those params.

Also from my understanding what you're trying to do, POST seems like the more appropriate HTTP verb as you're providing a block of data to be processed and handed back by the server.

Discussion courtesy of: C Blanchard

This recipe can be found in it's original form on Stack Over Flow.