3

I am getting memory related error when I try to split a big file into 8GB files.

# split -C 8000000000 r_mini_20120503.txt  
split: memory exhausted

I am however able to create 4GB files using the same command. Is there any way to increase memory for this purpose?

Improve this question
2
  • 2
    I doubt it's a memory issue, more likely the version of 'split' you are using doesn't support large files. Grab the src and recompile with '-D FILE_OFFSET_BITS=64' (this will work on recent 32bit as well as 64bit)
    – symcbean
    May 14, 2012 at 11:20
  • That's not the question though.
    – Tom O'Connor
    May 14, 2012 at 11:29

3 Answers 3

Reset to default
3

Maybe split is the somewhat wrong tool for this task. You can try it with dd (untested):

infile=r_mini_20120503.txt
filesize=1234567890 # manual set
skip=0
chunk=8000000000
bs=1000000000
count=8
part=1

while [ $skip -lt $filesize ] ; do
    dd if=$infile of=$infile.$part bs=$bs count=$count skip=$skip
    skip=[[ $skip + $chunk ]]
    part=[[ $part + 1 ]]
done

As I wrote, untested.

Improve this answer
4
  • 1
    This is interesting, but deeply, deeply scary.
    – Tom O'Connor
    May 14, 2012 at 11:29
  • Why is that scary?
    – ott--
    May 14, 2012 at 11:44
  • I find dd scary.
    – Tom O'Connor
    May 14, 2012 at 12:11
  • especially the "untested" part along with dd is scary :)
    – the-wabbit
    May 14, 2012 at 13:28
0

I'd say you're trying to use more memory than is on your system.
You could have a go at doing this in Single User mode, so there's less services, and other stuff (like X and GDM) running.

To do that, run telinit 1

OR.. Install more RAM. It's cheap.

Improve this answer
0

How much RAM is in your box? You could add some more swap to your system for the time being, but that's going to be dog slow :P

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .