Reputation
348
Top tag
Next privilege 500 Rep.
Access review queues
Badges
3 13
Impact
~25k people reached

  • 0 posts edited
  • 0 helpful flags
  • 14 votes cast
Dec
28
comment Can i delete oplog local.* of Mongo backup fsyncLock() + LVM snapshot?
and about MMS, we tried, and it was great. Unfortunately, their pricing is completely inadequate for us: we do constant updates. the data size grows slowly, but the oplog operations are numerous, which is why their pricing per oplog operation is killing.
Dec
28
comment Can i delete oplog local.* of Mongo backup fsyncLock() + LVM snapshot?
The fact is that mongo dump takes more than 24h to restore. The data size is around 300GB. Tt takes so long, that even if it needs to recreate the indexes, it does not make sense, the restore server is not running at full capacity.
Dec
27
asked Can i delete oplog local.* of Mongo backup fsyncLock() + LVM snapshot?
Nov
25
awarded  Popular Question
Nov
21
awarded  Curious
Sep
30
answered Shellshock: How do I know if my server is compromised + suspicious files to look out for
Sep
30
accepted ES fills up heap slowly over time and hangs at 14GB, while biggest index is 164MB?
Sep
30
answered ES fills up heap slowly over time and hangs at 14GB, while biggest index is 164MB?
Sep
24
asked ES fills up heap slowly over time and hangs at 14GB, while biggest index is 164MB?
Aug
22
awarded  Nice Answer
Apr
9
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
I managed to get NORAID/JBOD on the rented server. Configuration is soft raid1 with mdadm except for one, which is of course ZFS's own raid1 with ashift=12. Prefetching disabled, ARC max 1/3 of RAM, compress lz4, zfs_txg_timeout=5. XFS Zvol created with volblocksize=128K, mounted with noatime,logbufs=8,logbsize=256k as well. I can fsyncLock Mongo(), xfs_freeze and zfs snapshot. I automatically send snapshots diffs hourly to the staging machine so that it verifies content by mounting it and we are confident that snapshots are correct. I also clone the snap, mount+fsck, and send to S3.
Apr
9
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
I managed to get NORAID/JBOD on the rented server. Configuration is soft raid1 with mdadm except for one, which is of course ZFS's own raid1.
Apr
7
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
That works rather well now, thank you all for your help
Apr
7
accepted MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
Apr
1
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
@ewwhite Can you explain why the -o volblocksize=128K and the logbufs=8,logbsize=256k mount options, if you made a special calculation for it ? I also see nobarrier, so i guess your raid has a battery ?
Apr
1
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
Disabling prefetching improved a lot. It reversed my previous ratio of 5 reads for 1 write to 1 read to 3 writes (Sar data). Much better for my write intensive app to Mongo
Apr
1
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
Now testing with ashift=12, prefetching disabled and arc_max at 12GB, and xfs on zvol, and it seems great so far. I still need to test it on zfs's own raid 1.
Mar
30
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
That is interesting and worth trying. Your zpool creation cmd line reminded me that my first tests that showed small disk activity were with ashift=12. My latest tests with disastrous iowait were without, because i did not know what were the disks behind the hardware raid. Maybe that is a big part of the solution
Mar
30
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
I will try to raise ARC to 12GB to see. MongoDB is also memory hungry
Mar
26
comment MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
@adam-c What does "checked" mean "ZFS is not even checked" ? Do you simply mean that the Mongo server is testing on which filesystem its data files are located to choose an efficient storage mechanism, but this had not been implemented for ZFS on Linux ? (sorry, i am not a native english speaker and i find that ambiguous to understand)