I have a long involved script that is intended to install a Solaris system based on an image. This script was in the "almost working with a couple of minor bugs (like Tee does not exit after pipeline it's on has finished)" state, and now has devolved to "not working", in large part because of the following error.
Everything on the "image" side of the script works just fine. I prepare the system by stopping some services, shutting down and detaching the zones, removing some extraneous files (audit, mail, that kind of stuff), then going to single user mode. There I flar the system, take tarballs of the zones, then reattach the zones and restart the services. That all works fine. Including the zones; they reattach and then boot just fine.
On the target system, however, things get weird. I restore the system from flar, boot it, and then run the script. The script creates volumes on a direct attached array, formats and newfs's the volumes, mounts them, unzips and untars all the tarballs I created, then attachs, boots, reconfigures, and reattaches the zones. This was all working fine, and I cannot for the life of me find anything that I've changed in this part of the script. Now, though, it has started to do the following:
Upon attaching the zone, it complains that it (I'm not using zfs at all).
failed to initialize zfs library
Then, when attempting to login to the zones, I get errors with the /dev system (i.e., /dev/null: cannot open; /tmp/spinning: No such file or directory). Then, after this, when I try to login to the zones to do troubleshooting, I get
zlogin: failed to open /zones/APP1/dev/pts/1: No such device or address
I found some hints that it might be a "device major number" mismatch between the zone and the global zone, but doing
ls -lL, the major numbers look the same. The zones show as running with no issues. There are no broken services.
I have been googling for 3 days and I can't even find any hints to follow. Any ideas?