0

I'm in trouble while performing cat/tail/head operation on large files on Aix 5.3. When asking for a cat of several 1Go file redirected to another one:

cat file1 file2 file3 > outputfile

The outputfile is limited to 2Go (cat: output error and result file is 2147483647 bytes)

Filesystem is jfs2. I successfully uploaded through ftp 10Go files on the filesystem without problem.

I found nothing relevant in etc/security/limits:

default:
fsize = -1
core = 2097151
cpu = -1
data = 262144
rss = 65536
stack = 65536
nofiles = 20000

ulimit -a

core file size (blocks)     unlimited
data seg size (kbytes)      245759
file size (blocks)          unlimited
max memory size (kbytes)    unlimited
open files                  2000
pipe size (512 bytes)       64
stack size (kbytes)         32768
cpu time (seconds)          unlimited
max user processes          2048
virtual memory (kbytes)     278527

The problem does not occur on another AIX 5.3 server, I'm just looking for a different configuration that might be the source of the problem.

/etc/security/limits on the server without the problem:

default:
        fsize = -1
        core = 2097151
        cpu = -1
        data = 262144
        rss = 65536
        stack = 65536
        nofiles = 20000

ulimit -a on the server without the problem:

core file size        (blocks, -c) 1048575
data seg size         (kbytes, -d) 131072
file size             (blocks, -f) unlimited
max memory size       (kbytes, -m) 32768
open files                    (-n) 20000
pipe size          (512 bytes, -p) 64
stack size            (kbytes, -s) 32768
cpu time             (seconds, -t) unlimited
max user processes            (-u) 262144
virtual memory        (kbytes, -v) unlimited
2
  • Are you using the same shell on both servers?
    – jftuga
    Sep 20, 2010 at 12:55
  • Thank you for your interest. I'm using bash on both servers
    – snowflake
    Sep 21, 2010 at 14:22

1 Answer 1

2

You've said you're using ext2, but that's a Linux filesystem. I assume you meant JFS2?

JFS2 and cat both support >2GB files, so I think the most likely thing is the ulimit values - you've put in the contents of /etc/security/limit, but what happens if you run

ulimit -a

as the person executing cat?

If you're actually using JFS, not JFS2, then you'll need to check the filesystem is large file enabled,

Enter smit fs.
Select Add/Change/Show/Delete File Systems.
Choose Journaled File Systems.
Select Change/Show Characteristics of a Journaled File System.
Choose the file system that you are using.
Scroll down until you get to Large File Enabled.

Finally, you can check that you're running the 64-bit kernel, not the 32-bit one, by running the following command as root:

bootinfo -K

That's an upper-case K there. If it returns 32, then you should switch to the 64-bit one.

Ewan

4
  • My bad, it is jfs2
    – snowflake
    Sep 21, 2010 at 14:18
  • I'm running the 64-bit kernel
    – snowflake
    Sep 21, 2010 at 14:26
  • I added the ulimit information. I tried to compute something from the pipe size 64*512*32768=1Gbyte, I guess it is linked to my problem and will look for pipe size configuration
    – snowflake
    Sep 21, 2010 at 14:28
  • @snowflake you're showing an output of "Pipe size" in the ulimit output, that normally means you're not running the standard Ksh shell? Are you running Bash on that machine? If so, that's likely the issue, it could well be a 32-bit bash shell. Can you run "echo $SHELL" to verify which shell you're in?
    – Ewan Leith
    Oct 1, 2010 at 12:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .