2

Right now, I have an Apache setup sitting in front of Tomcat to handle caching. This setup has been given to an external service to manage, and since the transition, I've noticed odd behavior. Specifically, when I request a swf file from the web server, I hit the Apache cache (good), but occasionally I'll receive a truncated file. Once I receive this truncated file, the cache will NOT refresh until I manually delete the cache and let the swf pull down from tomcat again.

The external service claims that the configuration is fine, but I don't see any way this could be happening aside from improper configuration. Now, there are two apache and two tomcat servers under a load balancer, and occasionally one apache cache will break while another does not (leading to 50% of all requests getting bad, truncated data).

Where should I start looking to debug this issue? What could POSSIBLY be causing this odd behavior?

Edit: Inspecting the logs, tomcat throws this:

java.io.IOException: Bad file number
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:199)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        at java.io.FilterInputStream.read(FilterInputStream.java:90)
        at org.apache.catalina.servlets.DefaultServlet.copyRange(DefaultServlet.java:1968)
        at org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:1714)
        at org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:809)
        at org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:325)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:209)
        at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:347)
        at org.terracotta.modules.tomcat.tomcat_5_5.SessionValve55.invoke(SessionValve55.java:57)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:619)

followed by 

access_log.2009-12-14.txt:1.2.3.4 - - [14/Dec/2009:00:27:32 -0500] "GET /myApp/mySwf.swf HTTP/1.1" 304 -
access_log.2009-12-14.txt:1.2.3.4 - - [14/Dec/2009:01:27:33 -0500] "GET /myApp/mySwf.swf HTTP/1.1" 304 -
access_log.2009-12-14.txt:1.2.3.4 - - [14/Dec/2009:01:39:53 -0500] "GET /myApp/mySwf.swf HTTP/1.1" 304 -
access_log.2009-12-14.txt:1.2.3.4 - - [14/Dec/2009:02:27:38 -0500] "GET /myApp/mySwf.swf HTTP/1.1" 304 -

So apache is caching the bad file size. What could possibly be causing this, and possibly separate, how do I ensure that this exception does not get written to cache?

2
  • How's the memory usage on this machine? Jan 8, 2010 at 19:43
  • Very low. The server has 32GB of ram. Jan 8, 2010 at 20:06

2 Answers 2

0
+150

wget, curl or file save as this file?

Do other large files work?

If there's a support contract, I would consider requesting the vendor debug this collaboratively.

4
  • This seems to be the answer: mail-archives.apache.org/mod_mbox/tomcat-dev/200808.mbox/…
    – user31170
    Jan 8, 2010 at 20:17
  • Is that still not fixed in 6.0.20? Jan 8, 2010 at 20:21
  • wget and save as both pull down the cached apache file and fail. There aren't many other files on the server that get hit often as large as this file (in fact this file probably gets hit most frequently). Jan 9, 2010 at 3:08
  • If it's still occuring, I would ask the service provider run a sandbox with apache interactively with strace enabled. Also I won't mention a related service we're launching, that would be douchetabular.
    – user31170
    Apr 29, 2011 at 21:20
0

Tomcat's inability to serve large files has been fixed in trunk, 5.5.x, and the fix fixing the bug is committed to 6.0.27.

1
  • That's great news. I hear 6.0.32 seems the most stable.
    – user31170
    Apr 29, 2011 at 21:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .