Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am building a new proxy server with Squid & SquidClamAV.

I am facing problems with downloading large files. I have set a maxsize of 52428800 bytes in squidclamav.conf - which should prevent the scanning of files bigger than 50MB.

Downloading anything up to 2GB through the proxy works fine, but files bigger than that produce the following error: ICAP protocol error - The system returned: (32) Broken pipe

In c-icap server log i get this: Fri Dec 2 10:19:36 2011, general, Bug in the service. Please report to the servive author!!!!

Downloading a file at sizes below 2GB produces the download window instantly to the browser. As expected.

When starting to download anything above 2GB, it takes some time before producing the error. Following /var/tmp I see that it actually starts to download the file up to 50MB (the maxsize set in squidclamav.conf!) and then stops and produces the error.

I don't understand why this happens with files that are bigger than the maxsize in squidclamav.

The server is a CentOS 6 32bit machine. C-icap is compiled from source (v0.1.7), squid is from CentOS, clamd+squidclamav from repoforge.org. Below I've added related config files.

/etc/squid/squid.conf:

...
icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Authenticated-User
icap_service service_req reqmod_precache \
     bypass=1 icap://127.0.0.1:1344/squidclamav
adaptation_access service_req allow all
icap_service service_resp respmod_precache \ 
     bypass=1 icap://127.0.0.1:1344/squidclamav
adaptation_access service_resp allow all

/etc/squidclamav.conf:

maxsize 52428800
redirect http://localproxy/detecteio.php
clamd_local /var/run/clamav/clamd.sock
timeout 1
logredir 0
dnslookup 1

/etc/c-icap.conf:

PidFile /var/run/c-icap/c-icap.pid
CommandsSocket /var/run/c-icap/c-icap.ctl
Timeout 300
MaxKeepAliveRequests 100
KeepAliveTimeout 600
StartServers 3
MaxServers 10
MinSpareThreads     10
MaxSpareThreads     20
ThreadsPerChild     10
MaxRequestsPerChild  0
Port 1344
ServerAdmin root@localhost
ServerName localproxy.domain.local
TmpDir /var/tmp
MaxMemObject 131072
DebugLevel 1
ModulesDir /usr/local/lib/c_icap
ServicesDir /usr/local/lib/c_icap
TemplateDir /usr/local/share/c_icap/templates/
TemplateDefaultLanguage en
LoadMagicFile /usr/local/etc/c-icap.magic
RemoteProxyUsers off
RemoteProxyUserHeader X-Authenticated-User
RemoteProxyUserHeaderEncoded on
ServerLog /var/log/c-icap_server.log
AccessLog /var/log/c-icap_access.log
Service squidclamav squidclamav.so
Service echo srv_echo.so

/etc/clamd.conf:

LogFile /var/log/clamav/clamd.log
LogFileMaxSize 0
LogTime yes
LogSyslog yes
PidFile /var/run/clamav/clamd.pid
TemporaryDirectory /var/tmp
DatabaseDirectory /var/clamav
LocalSocket /var/run/clamav/clamd.sock
FixStaleSocket yes
TCPSocket 3310
TCPAddr 127.0.0.1
MaxConnectionQueueLength 30
MaxThreads 50
ReadTimeout 300
User clamav
AllowSupplementaryGroups yes
ScanPE yes
ScanELF yes
DetectBrokenExecutables yes
ScanOLE2 yes
ScanMail yes
ScanArchive yes
ArchiveBlockEncrypted no
share|improve this question
    
This might not help at all, but the only time I've encountered an issue with downloading files bigger than 2GB was with 32bit internet explorer (which uses a single data stream).. –  pauska Dec 2 '11 at 12:09
    
What do your squid logs show for the failed downloads? –  Flimzy Dec 3 '11 at 18:30
    
Thanks for the replies. Squid log shows a TCP_MISS/500: TCP_MISS/500 3217 GET ftp.funet.fi/pub/Linux/INSTALL/Centos/6/isos/x86_64/… - DIRECT/193.166.3.2 text/html –  mikkoko Dec 5 '11 at 8:15
add comment

3 Answers

Generally, 32bit systems will have trouble dealing with files larger than 2GB, however, having the download capped at 50MB would suggest ClamAV is at fault.

Couple of suggestions;

1) Ensure Squid has been compiled with '--with-large-files'

2) Test the download locally on the box using curl or wget (rules out browser/network issue)

3) Try increasing the 'maxsize' directive in '/etc/squidclamav.conf' to see if you can download larger files.

4) Disable file scanning and re-test.

share|improve this answer
add comment

Based on the configuration provided, it seems that this issue is not because of squid as the configured limit is 50MB which is the limit to prevent scanning files larger than this but it has nothing to do with the size of download file. Can you please confirm the name and version of Web browser using for download, as some Internet explorer versions allow only less than specific size of files to download. For Example, this behavior may occur if you try to download a file that is larger than 2 GB in IE6 or is larger than 4GB in Internet Explorer 7 in this case you may switch to Firefox or any other web browser. Note: This download limit has been removed in IE8. Therefore, you should not experience this behavior in IE8. You can refer this link for more information http://support.microsoft.com/kb/298618 I hope this information will be helpful for you

share|improve this answer
    
Sorry but it's not a client/browser problem (bypassing the proxy allows me to download the tested files from the same client/browser). Somehow its squid, clamav or c-icap that is figuring the file size wrongly when the size exceeds 2GB. –  mikkoko Dec 7 '11 at 12:01
    
Have you tried by using Firefox, by doing some further investigation, I have found the same issue discussed in the following web link, please have a look: www2.us.squid-cache.org/mail-archive/squid-users/200705/… –  Asif Hussain Dec 7 '11 at 16:57
add comment
up vote 0 down vote accepted

The proxy server was rebuilt with 64bit Centos 6. The same configuration was used, but the problem has gone away.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.