1

Situation

  • xen 4.0.1 dom0 (debian squeeze)
  • domUs all with LVs as disks: disk = [ 'phy:/dev/vg-00/domu-swap,xvda1,w', 'phy:/dev/vg-00/domu-disk,xvda2,w' ])
  • one VG (vg-00) with 2 PVs

Goal

  • Move all LVs from one PV to the other (pvmove) and remove the "empty" PV (vgreduce)
  • Not disturbing any running machine (domU od Dom0)

Problem(s)

When i start pmove (i even tried to ionice -c3 it) my domUs get very high loads or even get stuck. I think this happens, when pvmove is moving the extends from the domUs LV from one PV to the other. I also saw the domU really freaking out an fireing the OOM-killer. Long story short: i had to interupt the procedure (pvmove --abort), because my domU(s) started to get unusable, major server components were killed or they even died/froze completely.

Questions

  • I'm aware of higher IO loads while the transition and can cope with this. But even when ionice -c3d the IO load is so high, that inside the domU tasks get blocked. Why isn't the ionice working here? If i understand this correct, all IO is done by the dom0 (by the blkback driver) so the dom0 should see all IO done by every dom(0|U) and should be able toschedule IO for my renice process - are my assumptions wrong here?

  • Why does my domU fire up the OOM-killer? How can this process affect the domU's memory? BTW: when the domUs go crazy my dom0 works fine. High IO, but that's obvious.

  • Is there any way to remove one PV without the hassel above? Would it be better to shutdown/pause one domU after the other and pvmove only LVs from this machine?

Thank you in advance for every input - i even would be glad to get some "debugging ideas"!

Improve this question

1 Answer 1

Reset to default
0

It should work - but it seems that XEN imposes some kind of exclusive locking in "w" mode. Perhaps that locking is not that strict in "w!" (or was is "!w"?) mode. That mode should allow write requests from more than one source.

Which memory cosumption goes up before the killer kicks in in the DomU? Buffer memory?

Improve this answer
3
  • I don't really understand, why 'w!' should make a difference, but i will give it a try. I can't really tell, which memory consumption was going up - as i noticed the oom killer kicking in, i aborted the whole process. But next time i will look into this ... Thank you very much for your input!
    – m.sr
    May 16, 2011 at 8:57
  • You are welcome. Please keep us updated. I could get into situations where I would try the same what you did...
    – Nils
    May 16, 2011 at 21:06
  • Yesterday i moved some of my LVs. I moved one LV after the other (pvmove -n lvname ...). Most of them moved fine - had no drastic impact on any machine. Now i have 4 LVs left (unfortunately the biggest ones). When i try to move them, load on a domU (always the same one(s)) rises, machine is stuck and i have to abort. Will investigate ... P.S.: Thist was done with running domUs and 'w'-disks
    – m.sr
    May 20, 2011 at 6:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .