1

I have a really annoying issue, whereas PDF's I'm trying to download from the site get corrupt. They are generated fine (I can see that if I download it via SFTP from the temp folder they are put in).

    location ~ \.cgi$ {
        #try_files $uri =404;
        gzip off;
        proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:8181;
    }

The Perl code is pretty simple (I won't bother putting the PDF generating code in as well, as that is working fine);

my $filepath = qq|/home/fatpeter/web/site.com/cgi-bin/hotels/admin/tmp_pdf/| . CORE::time() . q|.pdf|;

$pdf->to_file($filepath);

$file_name =~ s/\-/_/sig;
my $size = -s $filepath;

my $file = \do { local *FH; *FH };
open $file, "<$filepath" or die "Unable to open file '$filepath': $!";
binmode $file;
binmode STDOUT;

print $IN->header($IN->file_headers(
    filename => $file_name,
    mimetype => 'application/pdf',
    inline   => 0,
    size     => $size
));

{
    local $\;
    while (read($file, my $chunk, 4096)) {
        print $chunk;
    }
}

So basically nginx passes the request to Apache2.4, then Perl runs and creates the PDF. The script then passes the PDF back to them.

[Tue Apr 25 13:41:36.810922 2017] [:error] [pid 2901] :Apache2 IO write: (104) Connection reset by peer at /home/fatpeter/web/site.com/cgi-bin/hotels/invoices.cgi line 285

Line 285 is the while () { } loop:

    while (read($file, my $chunk, 4096)) {

I'm at a loss as to why it would be cutting it off before it's downloaded. Any ideas? For what its worth, the test file I'm using is on 363 KB (372,596 bytes), so not exactly huge! My proxy settings in the main nginx.conf file are:

proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass_header Set-Cookie; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffers 32 4k;

This is what the script is sending out as the headers:

Content-type: application/pdf
Content-Disposition: attachment; filename="foo.pdf"; size=372596
Content-Length: 372596

UPDATE: Interestingly, I now get this error message in Firefox when downloading:

C:\Users\Andy\AppData\Local\Temp\Jy4tj5pr.pdf.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator.

I also now get this in the access log:

127.0.0.1 - - [25/Apr/2017:15:29:38 +0000] "GET /cgi-bin/hotels/invoices.cgi?action=download;id=60d90acf677e9c81;invoice=1055;t=french HTTP/1.0" 200 372861 "https://www.example.com/cgi-bin/hotels/invoices.cgi?t=french" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"
81.174.134.133 - - [25/Apr/2017:15:29:38 +0000] "GET /cgi-bin/hotels/invoices.cgi?action=download;id=60d90acf677e9c81;invoice=1055;t=french HTTP/2.0" 200 372616 "https://www.example.com/cgi-bin/hotels/invoices.cgi?t=french" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"

UPDATE 3: OK, so I'm getting a little bit closer here. The issue seems to be with the "chunking" in the script:

while (read($file, my $chunk, 4096)) {

If I change this to, for example:

while (read($file, my $chunk, 409600)) {

(greater than the file size in question), it WORKS! The problem with that though, is what do I do with larger files?

So here is the working code:

{
    local $\;
    while (read($file, my $chunk, -s $filepath)) {
        print $chunk;
    }
}

I just don't like the idea of not being able to send it back in blocks. Is there any disadvantage to doing it this way?

  • When you say that the PDF files are corrupt what does that mean exactly? Does the client see anything? A truncated file? Something else? – Paul Haldane Apr 25 '17 at 14:11
  • Does the code to generate the PDF file explicitly close the file? Does the size in the outgoing header match the size of the file? (though I'm not sure how an unflushed file with that code would lead to the error you see in the logs). Could you try a simple test script just using a static PDF file rather than generating a file on the fly? – Paul Haldane Apr 25 '17 at 14:14
  • @PaulHaldane - I've just put some links in for you to look at the PDFs (bottom of my post). I'll remove them after, as they have my address in :) The browser based one downloads, and both show as the same size (almost), but the one downloaded from the browser seems to have 1000 less lines when I look at it in WinMerge – Andrew Newby Apr 25 '17 at 14:14
  • @PaulHaldane , I just gave that a go. Uploaded a sample PDF that I know was fine, and then used the same code (with some hard-coded paths in) to download it via the browser. It came out the same size as the original, but was missing lots of lines. – Andrew Newby Apr 25 '17 at 14:19
  • How many requests do you see in the server log for each invoice? Could it be that there are overlapping requests trying to use the same file name? Perhaps try with a higher resolution time function (eg Time::HiRes) or something else that will generate a unique file name. – Paul Haldane Apr 25 '17 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.