0

I have been trying to set up squid 3.5.27 on a windows box using the store-id-program setting.

However, it seems that however I set it up the python script I'm running immediately gets end-of-input (as can be seen by adding trace) and then squid stops running, complaining

Squid Cache (Version 3.5.27): Terminated abnormally.
FATAL: The store_id helpers are crashing too rapidly, need help!

I've tried all sorts of variants (not at the same time obviously):

store_id_program /cygdrive/c/apps/squid/local/bin/texture_rewrite.bat
store_id_program /cygdrive/c/apps/squid/local/bin/texture_rewrite.py
store_id_program /cygdrive/c/apps/Python27/python.exe -u c:\apps\squid\local\bin\texture_rewrite.py

(The .bat file consists of @<path-to-python> -u <path to script>)

Squid works fine if I don't try setting up url rewriting.

For reference the python script looks like this:

while True:
    line = sys.stdin.readline()
    print >>sys.stderr, line
    if not line:
        break
    line = line.strip()
    if line != '':
        process(line)

print >>sys.stderr, 'exit'

process(line) never gets called

1

If you are referencing our Squid for windows build from squid.diladele.com - then most probably this is due to the fact how Cygwin models piping between Cygwin binary (squid) and your native Windows binary (python). If I am not mistaken the pipes are modelled as non blocking overlapped IO over files - which gets ERROR_IO_PENDING right after read - that is interpreted as EOF in any console program.

Partially this problem is discussed in https://www.cygwin.com/ml/cygwin/2006-03/msg00330.html, https://github.com/nodejs/node/issues/3006 and many others.

The best bet is to compile the urlrewriter as C++ code from within cygwin - in this case the code will automatically use the same pipe implementation as squid and thus work. Or use read byte by byte from overlapped files like we do in our urlrewriter (just a sample)

read_result read(char& c)
{
    // this is the value to return
    CHAR  value = 0;
    DWORD read  = 0;
    DWORD len   = sizeof(CHAR);

    // construct overlapped structure
    OVERLAPPED ovl = {};
    {
        ovl.hEvent = hevent_;
    }

    // read one byte from input
    BOOL bres = ::ReadFile(stdin_, &value, len, &read, &ovl);
    DWORD err = ::GetLastError();
    if (bres)
    {
        // see if we read 0 bytes
        if (0 == read)
        {
            // yes, read of 0 bytes from the redirected pipe in cygwin means eof???
            return reached_eof;
        }

        // debug check
        _ASSERTE('\0' != value);

        // otherwise store char
        c = value;

        // nice 
        return read_success;
    }

    // we have encountered and error, see which one
    switch (err)
    {
    case ERROR_HANDLE_EOF:
        return reached_eof;

    case ERROR_IO_PENDING:
        {
            // wait until overlapped read completes
            BOOL bres = GetOverlappedResult(stdin_, &ovl, &read, TRUE);
            if (bres)
            {
                // good, we have read one byte
                c = value;

                // nice
                return read_success;
            }

            // overlapped result failed, there might be a lot of reason for this, see if EOF is there
            if (ERROR_HANDLE_EOF == GetLastError())
                return reached_eof;

            // in case of all other errors we fail the operation
            return read_error;
        }
        break;

    default:
        // some unknown error, we also fail the operation
        return read_error;
    }
}
| improve this answer | |
  • Umm. Is there a cygwin version of python? Writing c++ for this is maybe a little too hard – Tom Tanner Nov 10 '18 at 6:39
  • I installed cygwin with python but that seems to not even run - at least I get no trace output from the script at all – Tom Tanner Nov 10 '18 at 10:03
  • Converted the above to python and seems to be working – Tom Tanner Nov 10 '18 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.