I always have to look this up, so I’m wiring it down. To mount an ISO image on Solaris derived systems, do the following:
lofiadm -a /path/to/image.iso /dev/lofi/1 mount -F hsfs -o ro /dev/lofi/1 /path/to/mountpoint
So you have used SVM to mirror your disk, and one of the two drives fails. Aren’t you glad you mirrored them! You don’t have to do a restore from tape, but you are going have to replace the failed drive.
Many modern RAID arrays just require you to take out the bad drive and plug in the new one, while everything else is taken care of automatically. It’s not quite that easy on a Sun server, but it’s really just a few simple steps. I just had to do this, so I thought I would write down the procedure here.
Basically, the process boils down to the following steps:
Let’s look at each step individually. In my case, c0t1d0 has failed, so, I detach all meta devices on that disk and then delete them:
# metadetach -f d0 d2
# metadetach -f d10 d12
# metadetach -f d40 d42
# metaclear d2
# metaclear d12
# metaclear d42
Next I take a look at the status of my meta databases. Below we can see the the replicas on that disk have write errors:
# metadb -i
flags first blk block count
a m p luo 16 8192 /dev/dsk/c0t0d0s3
a p luo 8208 8192 /dev/dsk/c0t0d0s3
W p luo 16 8192 /dev/dsk/c0t1d0s3
W p luo 8208 8192 /dev/dsk/c0t1d0s3
r - replica does not have device relocation information
o - replica active prior to last mddb configuration change
u - replica is up to date
l - locator for this replica was read successfully
c - replica's location was in /etc/lvm/mddb.cf
p - replica's location was patched in kernel
m - replica is master, this is replica selected as input
W - replica has device write errors
a - replica is active, commits are occurring to this replica
M - replica had problem with master blocks
D - replica had problem with data blocks
F - replica had format problems
S - replica is too small to hold current data base
R - replica had device read errors
The replicas on c0t1d0s3 are dead to us, so let’s wipe them out!
# metadb -d c0t1d0s3
# metadb -i
flags first blk block count
a m p luo 16 8192 /dev/dsk/c0t0d0s3
a p luo 8208 8192 /dev/dsk/c0t0d0s3
The only replicas we have left are on c0t0d0s3, so I’m all clear to unconfigure the device. I run cfgadm to get the c0 path:
# cfgadm -al
Ap_Id Type Receptacle Occupant Condition c0 scsi-bus connected configured unknown c0::dsk/c0t0d0 disk connected configured unknown c0::dsk/c0t1d0 disk connected configured unknown c0::dsk/c0t2d0 disk connected configured unknown c0::dsk/c0t3d0 disk connected configured unknown c1 scsi-bus connected configured unknown c1::dsk/c1t0d0 CD-ROM connected configured unknown usb0/1 unknown empty unconfigured ok usb0/2 unknown empty unconfigured ok usb1/1.1 unknown empty unconfigured ok usb1/1.2 unknown empty unconfigured ok usb1/1.3 unknown empty unconfigured ok usb1/1.4 unknown empty unconfigured ok usb1/2 unknown empty unconfigured ok
I run the following command to unconfigure the failed drive:
# cfgadm -c unconfigure c0::dsk/c0t1d0
The drive light turns blue
Pull the failed drive out
Insert the new drive
Configure the new drive:
# cfgadm -c configure c0::dsk/c0t1d0
Now that the drive is configured and visible from within the format command, we can copy the partition table from the remaining mirror member:
# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2
Next, I install the bootblocks onto the new drive:
# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s2
Create the state replicas:
metadb -a -c 2 c0t1d0s3
Recreate the meta devices:
metainit -f d2 1 1 c0t1d0s0
metainit -f d12 1 1 c0t1d0s1
metainit -f d42 1 1 c0t1d0s4
And finally, reattach the metadevices which will sync them up with the mirror.
metattach d0 d2
metattach d10 d12
metattach d40 d42
At least in RHEL 4, the fdisk command does not support the creation of filesystems larger than 2TB. In order to get around it, you have to use the parted command. I found the basic info here, but this is the long and short of how to cut off a big ol’ slice of disk using parted:
Run parted
# /sbin/parted
It’s interactive, so the following commands are issued within the utility.
1) Make the disk label
(parted) mklabel gpt
2) Create the partition
(parted) mkpart primary 0 -1
3) Verify
(parted) print
Disk geometry for /dev/sda: 0.000-38146.972 megabytes
Disk label type: msdos
Minor Start End Type Filesystem Flags
1 0.031 101.975 primary ext3 boot
2 101.975 38146.530 primary lvm
4) Exit the GNU Parted command shell
(parted) quit
5) Finally, make the filesystem:
# mkfs.ext3 -m0 -F /dev/sdb1
6)Finally, you don’t want to wait for that big filesystem to fsck from time to time, so make sure it does not get checked unless you run the command yourself:
# tune2fs -c0 -i0 /dev/sdb1
That should just about do it. Remember that only RHEL 4 and higher can support filesystems larger than 2TB. If I remember correctly RHEL 3 can go up to 2TB, RHEL4 can handle 8TB, and RHEL 5 can make a whopping 16TB chunk of disk. Have fun!
Getting Solaris 8 to light up a Qlogic QLA2310 Fibre Channel card using the SUNWqlc and SUNWqlcx drivers can be frustrating enough, but the headaches are only beginning if you want to connect it to a SAN and you don’t have all the right packages installed.
Last week, I installed the QLA2310 in a Sun Fire V210 running Solaris 8. I installed the latest versions of SUNWqlc, SUNWqlcx and SUNWsan. After doing a reboot -- -r, the system came up and attached the driver to the card. I zoned it in the fabric and logged into Navisphere, where the WWN showed up, but neither Power Path or the Navisphere host agent could communicate with the CLARiiON. I also could not see any of the LUNS I had presented.
I thought it was strange that the CLARiiON could see the host, but the host could not see the CLARiiON.
I ran:
luxadm -e port
Which returned:
Found path to 1 HBA ports
/devices/pci@1d,700000/SUNW,qlc@1/fp@0,0:devctl CONNECTEDClearly, it could see the HBA.
I ran:
ls -l /dev/cfg
total 8
lrwxrwxrwx 1 root root 38 Nov 30 14:31 c0 ->
../../devices/pci@1e,600000/ide@d:scsi
lrwxrwxrwx 1 root root 39 Nov 30 14:31 c1 ->
../../devices/pci@1c,600000/scsi@2:scsi
lrwxrwxrwx 1 root root 41 Nov 30 14:31 c2 ->
../../devices/pci@1c,600000/scsi@2,1:scsi
lrwxrwxrwx 1 root root 48 Dec 4 13:49 c3 ->
../../devices/pci@1d,700000/SUNW,qlc@1/fp@0,0:fcThe card was C3… This becomes useful later when we have to config it.
I ran:
cfgadm -al -o show_FCP_dev
Which retuned:
cfgadm: Configuration administration not supported
There it was… I didn’t have the complete SAN package installed. I hadn’t done this in a few years, so I had forgotten all the packages I had to add to get the Sun SAN package working correctly… There are many.
Happily, Sun has now packaged them in a nice “SAN_4.4.12_install_it.tar.Z”, which you can get from their website if you have a username. It installs everything for you in the right order.
The only thing left to do was another reboot -- -r and run cfgadm -c configure c3 to config the device. After this everything started working nicely.
When you install RHEL, the filesystems are labeled for you. Usually you won’t have to mess with it anymore, but on occasion, you may want to change them to more accurately represent the data that is stored on that partition. If, for instance, you used to have all of your database files on a partition labeled “/database”, but you have now moved them somewhere else, and you now wish to house your user account data there, it would make sense to change the label to something like “/users”.
Labels are, of course, arbitrary, so there is no technical need to do this, and you could, instead simply change the mount point in the fstab file, mounting the partition by device name rather than label, but it is usually cleaner to change the label. Here is how you do it:
First, let’s figure out what the partition is currently labeled as:
[root@calvin /]# /sbin/e2label /dev/hda4
/database
[root@calvin /]#
It’s current label is “/database”, and, since we have moved the database data somewhere else, we now want to store our user account data here, we need to change it to “/users”.
[root@calvin /]# /sbin/e2label /dev/hda4 /users
[root@calvin /]#
That’s all there is to it, now we check to make sure we have done what we think we have done.
[root@calvin /]# /sbin/e2label /dev/hda4
/users
[root@calvin /]#
Sure enough, it’s now labeled “/users” and the data on the disk remains intact. Now all we have to do is change the appropriate entry in the “/etc/fstab” file to represent the change.
Change this:
LABEL=/database /databases ext3 defaults 1 2
To this:
LABEL=/users /users ext3 defaults 1 2
And you’re all set to go. Make sure you have unmounted “/databases” before making the change.
Now, just run:
[root@calvin /]# mount /users
[root@calvin /]#
And you’re all set to go. As always, change the values here to represent those in your environment.
If you manage a UNIX system with a large number of directories that vary in size, chances are that you’ve needed to figure out which ones are using up the most disk space. Of course if the directories are user accounts, the best way to do this is to enable quotas and use the “repquota” command. If you just have a bunch of directories, however, you can easily figure out which ones are largest by giving the correct arguments to “du” and “sort”. Here is how:
du -sk * | sort +0nr
This will display the size of all directories and sort them from largest to smallest. If you want to sort them from smallest to largest, simply remove the “r”.
du -sk * | sort +0n
If you have nested directories, you will need to incorporate foreach to recurse through and get all the directory names.
Kids… DO NOT TRY THIS AT HOME! If this is not done exactly right, you will render your system unbootable and corrupt your data. That being said, under some circumstances you can take some space from your swap partition and add it to an unused one without initializing your entire disk. This is particularly useful if you decide you want to use DiskSuite to mirror your system disk, but have not allocated the 100MB partition that is needed to hold the state databases. As always, BACK EVERYTHING UP FIRST. Better yet, make two backups and store them on two different systems. This is a risky procedure, and you don’t want to lose any data!
You can also use my instructions for copying a Solaris boot drive to a disk with a different partition layout as a safer alternative.
The first thing you need to do is figure out if your disk layout will allow for this procedure. Usually the swap partition is the second one on the disk, making it partition number 1 (Partition number 0 is root). If partition number 1 is swap on your system, and partition number 3 or 4 are unused, you are in good shape, and this should work. To figure this out, you should do something like this:
# format
Select the boot disk – usually disk 0
Specify disk (enter its number): 0
format> partition
format> print
This will show you the current disk layout.
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9436 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
Here we see that partitions 3 and 4 are unused and directly after partition 1, so we can take some space from swap and assign it to one of these. Partition 2 is, of course the entire disk. I have not tried it, so I don’t know if you could assign non-sequential cylinders to a partition that is not directly after swap.
So to take some space from partition 1 and add it to partition 3, the first thing we have to do is disable swap, so the format utility will let us change it.
Comment out the following lines in your /etc/vfstab file and reboot the system.
#/dev/dsk/c1t0d0s1 - - swap - no -
#swap - /tmp tmpfs - yes -
This will bring the system up without swap enabled. You can now edit the disk label. Remember that our cylinders need to be sequential, so always work in cylinders when using the format utility.
Re-enter the format utility, select your system disk and view the partition table:
# format
Select the boot disk – usually disk 0
Specify disk (enter its number): 0
format> partition
format> print
Again we wee that partitions 3 and 4 are unused.
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9436 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
The first thing we need to do is take some cylinders away from partition 1. In this example, we are looking to make partition 3 roughly 100MB, so we need to take about 75 cylinders from partition 1 so that we can add it to partition 3. Parititon 1 ends at cylinder 9436, so we need to subtract 75 from that number. 9436 – 75 = 9361, so that is the new ending cylinder for partition 1. We then subtract the beginning cylinder (726) from that number to give us the new total number of cylinders for partition 1. 9361 – 726 = 8635, so this is the number we enter when format asks for the size of the partition. Like so:
partition> 1
Part Tag Flag Cylinders Size Blocks
1 swap wu 726 - 9360 11.90GB (8635/0/0) 24946515
Enter partition id tag[swap]:
Enter partition permission flags[wu]:
Enter new starting cyl[726]:
Enter partition size[24946615b, 9436c, 12880.92mb, 12.00gb]: 8635c
partition>
Now we have to add these 75 cylinders to partition 3.
partition> 3
Part Tag Flag Cylinders Size Blocks
3 unassigned wm 0 0 (0/0/0) 0
Enter partition id tag[unassigned]:
Enter partition permission flags[wm]:
Enter new starting cyl[0]:9361
Enter partition size[0b, 0c, 0.00mb, 0.00gb]:75c
partition>
Print out the new partition table to make sure everything lines up correctly:
partition> print
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9360 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 9361 - 9436 107.21MB (76/0/0) 219564
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
Partition 1 ends at cylinder 9360, and partition 3 picks right up at cylinder 9361. Partition 3 ends at cylinder 9436, and partition 5 begins at cylinder 9437. Partition 4, of course, remains unused. Since none of the cylinders overlap, we can go ahead and write the disk label out. DO NOT DO THIS if you have any doubt at all about what you have just done. By writing out the disk label, you could corrupt the data on your formated filesystems if any cylinders overlap into them. The format utility is usually pretty smart about keeping you from making mistakes, but be very careful anyway! You don’t want to end up with scrambled eggs on a disk that has valuable data on it.
partition> label
This writes out the disk label, so you can now exit the format utility and re-enable swap in your /etc/vfstab file. Simply uncomment out the following two lines and reboot the system.
/dev/dsk/c1t0d0s1 - - swap - no -
swap - /tmp tmpfs - yes -
Reboot your system, and if all goes well, it will come up, and you will see that partition 3 will have a little over 100MB on it. Usually people want to do this so they can store the DiskSuite meta database on the newly created partition. If this is the case for you, you can now move on to mirroring the system disk.
