Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Folks,

I've had a CENTOS box running two xen VMs for a couple of years. Yesterday the ISP had a heat problem and the server went down. Now the host OS on the box is back up but I can't restart the XEND service. I do the 'service xend start' and the only output I see in the log file is:

    [2010-11-08 08:25:57 xend 2970] INFO (SrvDaemon:283) Xend Daemon started
[2010-11-08 08:25:57 xend 2970] INFO (SrvDaemon:287) Xend changeset: unavailable .
[2010-11-08 08:25:57 xend 2970] ERROR (SrvDaemon:297) Exception starting xend ((111, 'Connection refused'))
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDaemon.py", line 291, in run
    servers = SrvServer.create()
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvServer.py", line 108, in create
    root.putChild('xend', SrvRoot())
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvRoot.py", line 40, in __init__
    self.get(name)
  File "/usr/lib/python2.4/site-packages/xen/web/SrvDir.py", line 82, in get
    val = val.getobj()
  File "/usr/lib/python2.4/site-packages/xen/web/SrvDir.py", line 52, in getobj
    self.obj = klassobj()
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py", line 39, in __init__
    self.xd = XendDomain.instance()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 655, in instance
    inst.init()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 71, in init
    xstransact.Mkdir(VMROOT)
  File "/usr/lib/python2.4/site-packages/xen/xend/xenstore/xstransact.py", line 345, in Mkdir
    complete(path, lambda t: t.mkdir(*args))
  File "/usr/lib/python2.4/site-packages/xen/xend/xenstore/xstransact.py", line 351, in complete
    t = xstransact(path)
  File "/usr/lib/python2.4/site-packages/xen/xend/xenstore/xstransact.py", line 20, in __init__
    self.transaction = xshandle().transaction_start()
  File "/usr/lib/python2.4/site-packages/xen/xend/xenstore/xsutil.py", line 18, in xshandle
    xs_handle = xen.lowlevel.xs.xs()
Error: (111, 'Connection refused')
~

When I Ctrl+c the start command I see this trace which makes it look like maybe the issue is with xenstored?:

Starting xend: Traceback (most recent call last):
  File "/usr/sbin/xend", line 134, in ?
    sys.exit(main())
  File "/usr/sbin/xend", line 111, in main
    start_xenstored()
  File "/usr/sbin/xend", line 90, in start_xenstored
    s,o = commands.getstatusoutput(cmd)
  File "/usr/lib/python2.4/commands.py", line 54, in getstatusoutput
    text = pipe.read()
KeyboardInterrupt

Any thoughts?

share|improve this question
    
What versions are you on? –  pehrs Nov 8 '10 at 16:24
    
rpm shows:rpm -qa | grep xen kernel-xen-2.6.18-8.1.15.el5 xen-libs-3.0.3-25.0.4.el5 xen-3.0.3-25.0.4.el5 –  user59538 Nov 8 '10 at 20:57
    
Some more information, the xenstored process is taking 100% of the CPU and apparently not completing. It's been running for 17 minutes now and counting. I'm going to try just leaving it but it seems likely it's just stuck. –  user59538 Nov 9 '10 at 3:06

1 Answer 1

Just in case it is of use to future generations. The issue appeared to be a corrupt install of the Xen libraries. Likely when the server crashed they got subtly munged. Doing an upgrade using a new RPM solved the problem. The existing VM images could then be started and used unchanged.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.