Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This probably has a simple explanation, but I certainly can't think of it.

I've got corosync installed (via yum), with it's default init script. Something is strange on this particular CentOS installation as I often need to manually link /etc/rc.d/init.d/ to /etc/init.d.

The issue is that it fails when run via it's symbolic link, yet it runs fine through /etc/rc.d/init.d

What's even weirder is it fails to run if run using the full path, and only if actually run in the /etc/rc.d/init.d directory.

Example:

[~]# /etc/rc.d/init.d/corosync start
     Starting Corosync Cluster Engine (corosync):               [FAILED]
[~]# service corosync status
     corosync is stopped
[~]# cd /etc/rc.d/init.d/
[init.d]# /etc/rc.d/init.d/corosync start
          Starting Corosync Cluster Engine (corosync):          [FAILED]
[init.d]# corosync start
[init.d]# service corosync status
          corosync (pid  1985) is running...

Any explanation?

Edit:

Not sure what I've changed exactly, but it now works when started from /rc.d/init.d, but not with service corosync start.

[root@server2 mirror]# /etc/rc.d/init.d/corosync start
Starting Corosync Cluster Engine (corosync):               [  OK  ]

[root@server2 mirror]# /etc/init.d/corosync start
Starting Corosync Cluster Engine (corosync):               [FAILED]

[root@server2 mirror]# service corosync start
Starting Corosync Cluster Engine (corosync):               [FAILED]

edit 2:

Made a symbolic link from /etc/rc.d/init.d to /etc/init.d .. and now it works when run via service corosync start.. yet doesn't start on boot, argh.

Edit 3:

It's working with every command except on boot.

I've changed the run level to 99, which it still fails on, and I've changed the path inside the script to the absolute path : /usr/sbin/corosync

I've also done diffs of the environmental variables:

On service corosync start:

_=/bin/env
LANG=en_US.UTF-8
PATH=/sbin:/usr/sbin:/bin:/usr/bin
PWD=/
SHLVL=1
TERM=xterm

On boot:

_=/bin/env
LANG=en_US.UTF-8
PATH=/sbin:/usr/sbin:/bin:/usr/bin
PWD=/
SHLVL=2
TERM=linux
CONSOLETYPE=vt
LANGSH_SOURCED=1
previous=N
PREVLEVEL=N
runlevel=3
RUNLEVEL=3
UPSTART_EVENTS=runlevel
UPSTART_INSTANCE=
UPSTART_JOB=rc

Boot log:

Starting Corosync Cluster Engine (corosync):    [FAILED]
share|improve this question

5 Answers 5

up vote 1 down vote accepted

So now the script works when the system is already up but not during boot.

Is there perhaps a third version of the corosync script? Is the version in /etc/rc2.d/ linked to the one in /etc/init.d/ or is it different?

Incidentally, we have got this far without even considering your cluster setup. If this is part of a cluster, there may be clues in the log files of the other nodes.

share|improve this answer
    
All the same unfortunately. It's just a 2 node setup, and the first works just fine despite it being almost exactly the same. –  user705142 May 10 '12 at 8:59
    
Nothing in the other node's log about this node? Are nodenames in /etc/hosts or dependent on DNS? Check the time on the hardware clock and correct it if necessary. –  ramruma May 10 '12 at 10:23
    
Nothing in the log file of the other node on boot, although there is when connecting normally. It's unicast so nodes are defined by IP, both correct. Hardware clock is correct, and the same on both. –  user705142 May 10 '12 at 10:38
    
Use cksum to check that files that ought to be the same on each node really are the same. If this is not in production, try rebooting the other node to check it comes up cleanly. Change the startup script (take a copy first) to include set -x at the top in order to track down where it fails. If appropriate, do the same sort of thing to any failing child processes. Also check corosync log and set debug. –  ramruma May 10 '12 at 11:36
    
Chksums are all exactly the same. Just checked, first node starts up properly. Literally nothing in the corosync log, it doesn't write a line on boot, even in debug mode. The permissions on the log file are all the same as they are on the first node, so it can't be that either. It fails after running the executable and checking it's status. It won't even start when put in rc.local or in crontab, but it'll start as soon as the user manually runs it ! –  user705142 May 10 '12 at 12:04

I had the same issue...

check:

# getenforce

if it returns "Enforcing" then you have to disable SELinux in file: /etc/selinux/config

and dynamicaly:

# setenforce 0

share|improve this answer

So the problem now is that /etc/init.d/corosync start works with bash -x but not without, and not on boot. Is that right?

Does bash /etc/init.d/corosync start (without the -x work)?

Probably there is an environment variable set in your profile which is not there when the system runs the script during the boot process. Add a line like env |sort > /tmp/env.$$ to /etc/init.d/corosync and then diff the resulting files.

One other possibility is a hidden dependence on another service which starts later in the boot process. Try changing to S99...

share|improve this answer
    
Sorry, it's getting confusing - it's now (for some reason) working with every command except on boot, looking in the boot.log it's just returning the same FAILED message. Will do a diff and report back. –  user705142 May 10 '12 at 6:05
    
I've changed it to 99 with no luck unfortunately, edited the main post to include the env. variables. –  user705142 May 10 '12 at 6:44

If corosync start works in any directory but /etc/rc.d/init.d/corosync start fails, then probably you are running two different scripts. Run: which corosync

If not, check inside the corosync script for relative paths that should be absolute paths.

share|improve this answer
    
Ah, seems like that's running the bin file directly, /usr/sbin/corosync –  user705142 May 10 '12 at 5:25

Try to debug the init script with -x

# bash -x /etc/rc.d/init.d/corosync start

Also try to use service start|stop|status corosync.

share|improve this answer
    
That works without a problem - and using it directly now works as well, ack –  user705142 May 10 '12 at 5:19
    
It works with both /etc/rc.d/init.d/ and /etc/init.d/ when run via -x, how bizarre –  user705142 May 10 '12 at 5:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.