Currently there is a directory that is mounting over another directory on boot. I cannot figure out for the life of me where this is coming from. The problem mount is the last mount on the df -ah output...

UPDATE: A little more color. The /gfs2 directory below is a legacy directory on the root partition. I want to mount the /dev/mapper/vg_prodapi01-LogVol01 directly to the /sitelogs directory without it getting mounted over by this legacy /gfs2 directory on boot.

df -ah output

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_prodapi01-LogVol01
                       18G   12G  5.9G  66% /
proc                     0     0     0   -  /proc
sysfs                    0     0     0   -  /sys
devpts                   0     0     0   -  /dev/pts
tmpfs                 3.9G   32M  3.9G   1% /dev/shm
/dev/sda1             291M   65M  212M  24% /boot
none                     0     0     0   -  /proc/sys/fs/binfmt_misc
none                     0     0     0   -  /sys/kernel/config
sunrpc                   0     0     0   -  /var/lib/nfs/rpc_pipefs
/dev/mapper/vg_prodcluster-lv_gfs2
                       18G   12G  5.9G  66% /sitelogs
/gfs2/sitelogs         18G   12G  5.9G  66% /sitelogs #problem mount

Here are my configs:

/etc/fstab

#
# /etc/fstab
# Created by anaconda on Fri Jun 28 08:12:10 2013
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_prodapi01-LogVol01 /                       ext4    defaults        1 1
UUID=37913101-3d09-4eec-9c99-608827d5e9a7 /boot                   ext4    defaults        1 2
/dev/mapper/vg_prodapi01-LogVol00 swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/mapper/vg_prodcluster-lv_gfs2  /sitelogs   gfs2    defaults    0 0

rc.local

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

Here is also my boot log file

/var/log/boot.log

G       Welcome to CentOS
Starting udev: G                                           [  OK  ]
Setting hostname prodapi01.domain.com:                [  OK  ]
Setting up Logical Volume Management:   Skipping clustered volume group vg_prodcluster
  2 logical volume(s) in volume group "vg_prodapi01" now active
                                                           [FAILED]
Checking filesystems
_CentOS-6.4-x86_: clean, 393237/1163264 files, 3049229/4638720 blocks
/dev/sda1: clean, 48/76912 files, 75666/307200 blocks
                                                           [  OK  ]
Remounting root filesystem in read-write mode:             [  OK  ]
Mounting local filesystems:                                [  OK  ]
Enabling local filesystem quotas:                          [  OK  ]
Enabling /etc/fstab swaps:                                 [  OK  ]
Entering non-interactive startup
Calling the system activity data collector (sadc):
Starting monitoring for VG vg_prodapi01:   2 logical volume(s) in volume group "vg_prodapi01" monitored
                                                           [  OK  ]
   Checking acpi hot plug                                  [  OK  ]
Starting VMware Tools services in the virtual machine:
   Switching to guest configuration:                       [  OK  ]
   VM communication interface:                             [  OK  ]
   VM communication interface socket family:               [  OK  ]
   Guest operating system daemon:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:                                [  OK  ]
Bringing up interface eth2:                                [  OK  ]
Bringing up interface eth4:                                [  OK  ]
Starting auditd:                                           [  OK  ]
Starting portreserve:                                      [  OK  ]
Starting system logger:                                    [  OK  ]
Starting irqbalance:                                       [  OK  ]
Starting rpcbind:                                          [  OK  ]
Starting kdump:                                            [  OK  ]
Starting lldpad:                                           [  OK  ]
Starting cluster:
   Checking if cluster has been disabled at boot...        [  OK  ]
   Checking Network Manager...                             [  OK  ]
   Global setup...                                         [  OK  ]
   Loading kernel modules...                               [  OK  ]
   Mounting configfs...                                    [  OK  ]
   Starting cman...                                        [  OK  ]
   Waiting for quorum...                                   [  OK  ]
   Starting fenced...                                      [  OK  ]
   Starting dlm_controld...                                [  OK  ]
   Tuning DLM kernel config...                             [  OK  ]
   Starting gfs_controld...                                [  OK  ]
   Unfencing self...                                       [  OK  ]
   Joining fence domain...                                 [  OK  ]
Starting FCoE initiator service:                           [  OK  ]

Starting system message bus:                               [  OK  ]
Starting clvmd:
Activating VG(s):   1 logical volume(s) in volume group "vg_prodcluster" now active
  2 logical volume(s) in volume group "vg_prodapi01" now active
                                                           [  OK  ]
Starting NFS statd:                                        [  OK  ]
Initializing OpenCT smart card terminals:                  [  OK  ]

Starting RPC idmapd:                                       [  OK  ]
Starting cups:                                             [  OK  ]
Mounting other filesystems:                                [  OK  ]
Starting acpi daemon:                                      [  OK  ]
Mounting GFS2 filesystem (/sitelogs):                      [  OK  ]
Starting HAL daemon:                                       [  OK  ]
Retrigger failed udev events                               [  OK  ]
Starting PC/SC smart card daemon (pcscd):                  [  OK  ]
Loading autofs4:                                           [  OK  ]
Starting automount:                                        [  OK  ]
Enabling Bluetooth devices:
Starting snmpd:                                            [  OK  ]
Starting sshd:                                             [  OK  ]
Starting ntpd:                                             [  OK  ]
Starting postfix:                                          [  OK  ]
Starting puppet-agent: Loading launcher_prod_tier_2_rc...
Starting crond:                                            [  OK  ]
Starting atd:                                              [  OK  ]
Cleaning up /opt/simpana/Base/Temp ...
Starting Simpana services for Instance001 ...
Starting saslauthd:                                        [  OK  ]
Starting Cluster Module - cluster monitor:                 [  OK  ]
Starting Cluster Service Manager:                          [  OK  ]
Starting oddjobd:                                          [  OK  ]
Starting ricci:                                            [  OK  ]
  • This looks "right" to me, do you not intend for /sitelogs to be written to a shared GFS2 file system? – TheFiddlerWins Jun 29 '15 at 14:36
  • Let me add a little more color. The /gfs2 directory is a legacy directory that the previous sysadmin had set up as a loopback mount.What I want to do is just mount the disk directly to the /sitelogs directory as indicated in the fstab. – Blanco Jun 29 '15 at 16:44

We have pinpointed the issue. This is ridiculous. The previous sysadmin had created an init script to make the directory and mount it on boot without outputting anything.

Lesson for everyone -- please use standard configurations and save the sanity of the next person ahead of you.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.