Creating Linux Partitions for CLARiiON
Creating a properly offset slab of disk for Linux systems on your CLARiiON is not just a matter of creating a partition using the default fdisk values. The reason for this is that disk management utilities for Intel based systems generally write 63 sectors of metadata directly at the beginning of the LUN. The addressable space begins immediately after these initial sectors causing the CLARiiON to cross disks, especially when writing larger IO because it doesn’t match up with the stripe element size (usually 64k).
To get around this, you have to align the partition in such a way that it will start writing data on a sector that will mesh up nicely with the stripe element size. In this case, 128. Below is an example of how I create partitions on our CLARiiON for Linux systems. Check out the EMC Best Practices for Fibre Chanel storage white paper for more detail.
/sbin/fdisk /dev/emcpowera Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 39162. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-39162, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-39162, default 39162): Using default value 39162 Command (m for help): x Expert command (m for help): b Partition number (1-4): 1 New beginning of data (63-629137529, default 63): 128 Expert command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
How to Make Gnarly Big Linux Filesystems
At least in RHEL 4, the fdisk command does not support the creation of filesystems larger than 2TB. In order to get around it, you have to use the parted command. I found the basic info here, but this is the long and short of how to cut off a big ol’ slice of disk using parted:
Run parted
# /sbin/parted
It’s interactive, so the following commands are issued within the utility.
1) Make the disk label
(parted) mklabel gpt
2) Create the partition
(parted) mkpart primary 0 -1
3) Verify
(parted) print
Disk geometry for /dev/sda: 0.000-38146.972 megabytes
Disk label type: msdos
Minor Start End Type Filesystem Flags
1 0.031 101.975 primary ext3 boot
2 101.975 38146.530 primary lvm
4) Exit the GNU Parted command shell
(parted) quit
5) Finally, make the filesystem:
# mkfs.ext3 -m0 -F /dev/sdb1
6)Finally, you don’t want to wait for that big filesystem to fsck from time to time, so make sure it does not get checked unless you run the command yourself:
# tune2fs -c0 -i0 /dev/sdb1
That should just about do it. Remember that only RHEL 4 and higher can support filesystems larger than 2TB. If I remember correctly RHEL 3 can go up to 2TB, RHEL4 can handle 8TB, and RHEL 5 can make a whopping 16TB chunk of disk. Have fun!
Working With Disk Labels in RHEL
When you install RHEL, the filesystems are labeled for you. Usually you won’t have to mess with it anymore, but on occasion, you may want to change them to more accurately represent the data that is stored on that partition. If, for instance, you used to have all of your database files on a partition labeled “/database”, but you have now moved them somewhere else, and you now wish to house your user account data there, it would make sense to change the label to something like “/users”.
Labels are, of course, arbitrary, so there is no technical need to do this, and you could, instead simply change the mount point in the fstab file, mounting the partition by device name rather than label, but it is usually cleaner to change the label. Here is how you do it:
First, let’s figure out what the partition is currently labeled as:
[root@calvin /]# /sbin/e2label /dev/hda4
/database
[root@calvin /]#
It’s current label is “/database”, and, since we have moved the database data somewhere else, we now want to store our user account data here, we need to change it to “/users”.
[root@calvin /]# /sbin/e2label /dev/hda4 /users
[root@calvin /]#
That’s all there is to it, now we check to make sure we have done what we think we have done.
[root@calvin /]# /sbin/e2label /dev/hda4
/users
[root@calvin /]#
Sure enough, it’s now labeled “/users” and the data on the disk remains intact. Now all we have to do is change the appropriate entry in the “/etc/fstab” file to represent the change.
Change this:
LABEL=/database /databases ext3 defaults 1 2
To this:
LABEL=/users /users ext3 defaults 1 2
And you’re all set to go. Make sure you have unmounted “/databases” before making the change.
Now, just run:
[root@calvin /]# mount /users
[root@calvin /]#
And you’re all set to go. As always, change the values here to represent those in your environment.
Taking Disk Cylinders From Swap on Solaris 8
Kids… DO NOT TRY THIS AT HOME! If this is not done exactly right, you will render your system unbootable and corrupt your data. That being said, under some circumstances you can take some space from your swap partition and add it to an unused one without initializing your entire disk. This is particularly useful if you decide you want to use DiskSuite to mirror your system disk, but have not allocated the 100MB partition that is needed to hold the state databases. As always, BACK EVERYTHING UP FIRST. Better yet, make two backups and store them on two different systems. This is a risky procedure, and you don’t want to lose any data!
You can also use my instructions for copying a Solaris boot drive to a disk with a different partition layout as a safer alternative.
The first thing you need to do is figure out if your disk layout will allow for this procedure. Usually the swap partition is the second one on the disk, making it partition number 1 (Partition number 0 is root). If partition number 1 is swap on your system, and partition number 3 or 4 are unused, you are in good shape, and this should work. To figure this out, you should do something like this:
# format
Select the boot disk - usually disk 0
Specify disk (enter its number): 0
format> partition
format> print
This will show you the current disk layout.
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9436 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
Here we see that partitions 3 and 4 are unused and directly after partition 1, so we can take some space from swap and assign it to one of these. Partition 2 is, of course the entire disk. I have not tried it, so I don’t know if you could assign non-sequential cylinders to a partition that is not directly after swap.
So to take some space from partition 1 and add it to partition 3, the first thing we have to do is disable swap, so the format utility will let us change it.
Comment out the following lines in your /etc/vfstab file and reboot the system.
#/dev/dsk/c1t0d0s1 - - swap - no -
#swap - /tmp tmpfs - yes -
This will bring the system up without swap enabled. You can now edit the disk label. Remember that our cylinders need to be sequential, so always work in cylinders when using the format utility.
Re-enter the format utility, select your system disk and view the partition table:
# format
Select the boot disk - usually disk 0
Specify disk (enter its number): 0
format> partition
format> print
Again we wee that partitions 3 and 4 are unused.
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9436 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
The first thing we need to do is take some cylinders away from partition 1. In this example, we are looking to make partition 3 roughly 100MB, so we need to take about 75 cylinders from partition 1 so that we can add it to partition 3. Parititon 1 ends at cylinder 9436, so we need to subtract 75 from that number. 9436 - 75 = 9361, so that is the new ending cylinder for partition 1. We then subtract the beginning cylinder (726) from that number to give us the new total number of cylinders for partition 1. 9361 - 726 = 8635, so this is the number we enter when format asks for the size of the partition. Like so:
partition> 1
Part Tag Flag Cylinders Size Blocks
1 swap wu 726 - 9360 11.90GB (8635/0/0) 24946515
Enter partition id tag[swap]:
Enter partition permission flags[wu]:
Enter new starting cyl[726]:
Enter partition size[24946615b, 9436c, 12880.92mb, 12.00gb]: 8635c
partition>
Now we have to add these 75 cylinders to partition 3.
partition> 3
Part Tag Flag Cylinders Size Blocks
3 unassigned wm 0 0 (0/0/0) 0
Enter partition id tag[unassigned]:
Enter partition permission flags[wm]:
Enter new starting cyl[0]:9361
Enter partition size[0b, 0c, 0.00mb, 0.00gb]:75c
partition>
Print out the new partition table to make sure everything lines up correctly:
partition> print
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 725 1.00GB (726/0/0) 2097414
1 swap wu 726 - 9360 11.90GB (8635/0/0) 24946515
2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180
3 unassigned wm 9361 - 9436 107.21MB (76/0/0) 219564
4 unassigned wm 0 0 (0/0/0) 0
5 usr wm 9437 - 10888 2.00GB (1452/0/0) 4194828
6 var wm 10889 - 18148 10.00GB (7260/0/0) 20974140
7 unassigned wm 18149 - 24619 8.91GB (6471/0/0) 18694719
Partition 1 ends at cylinder 9360, and partition 3 picks right up at cylinder 9361. Partition 3 ends at cylinder 9436, and partition 5 begins at cylinder 9437. Partition 4, of course, remains unused. Since none of the cylinders overlap, we can go ahead and write the disk label out. DO NOT DO THIS if you have any doubt at all about what you have just done. By writing out the disk label, you could corrupt the data on your formated filesystems if any cylinders overlap into them. The format utility is usually pretty smart about keeping you from making mistakes, but be very careful anyway! You don’t want to end up with scrambled eggs on a disk that has valuable data on it.
partition> label
This writes out the disk label, so you can now exit the format utility and re-enable swap in your /etc/vfstab file. Simply uncomment out the following two lines and reboot the system.
/dev/dsk/c1t0d0s1 - - swap - no -
swap - /tmp tmpfs - yes -
Reboot your system, and if all goes well, it will come up, and you will see that partition 3 will have a little over 100MB on it. Usually people want to do this so they can store the DiskSuite meta database on the newly created partition. If this is the case for you, you can now move on to mirroring the system disk.
Rebuilding the Solaris Device Tree
If you ever shift around any bootable drives within a Sun Solaris box, you may find that either the device names (cxtxd0sx) do not follow the disk position within the server, or, the system just fails to boot because it can’t mount the other disk slices.
Let’s assume you are booting off of target 8 (c1t8d0s0), but wish to move that disk to the appropriate slot to make it target 0 (c1t0d0s0). You have changed all references in the /etc/vfstab file to reflect the new disk position, physically moved the drive from the target 8 slot to the target 0 slot, and changed the boot-device variable within the OBP to the appropriate disk. You should now be all set to boot from the disk in target 0, right?
Not quite yet.
Solaris creates a device tree with links to all the disks it knows about, and these don’t get rebuilt upon reboot. If you simply tried to boot the disk now in target 0, it would find the kernel, but fail to mount any of the other filesystems, because these device links are still pointing to the disk slices on target 8.
In order to boot off the drive in the new position, you will have to remove these device links and rebuild them. Here is how we do that:
1. Insert a Solaris 8, 9 or 10 cd into the hosts cdrom
2. From the ok prompt, enter boot cdrom -s
ok> boot cdrom -s
3. fsck the boot disk
# fsck -y /dev/rdsk/c1t0d0s0
Remember that your boot disk may differ than the example above. Since in our example above, we have put the disk into the slot for target 0 (c1t0d0), that is what we are using here.
4. Mount the root slice on /mnt
# mount /dev/dsk/c1t0d0s0 /mnt
Note that your root slice may differ than the above example.
5. Move path_to_inst
# mv /mnt/etc/path_to_inst /mnt/etc/PATH_TO_INST_ORIG
6. Remove all old device links
# rm /mnt/dev/rdsk/c* ; rm /mnt/dev/dsk/c* ; rm /mnt/dev/rmt/* ; rm
/mnt/dev/cfg/c*
7. Rebuild path_to_inst and devices
# devfsadm -r /a -p /mnt/etc/path_to_inst
8. Unmount the root slice and reboot
# umount /mnt ; init 6
You should now be able to boot off your old drive in its new slot.
Problems With Multiple MetaDB Partitions
UPDATE TO “Solaris Disk Partition Layout & Mirroring Scripts“
Several Months ago, I tried to use my old mirroring scripts on a new Solaris 9 install. I found that the the kernel would panic upon reboot because it was unable to mount /. I tried many things, including opening a support call with Sun. They reviewed my scripts and said that they should work, but despite repeated tries, they did not.
In the end, I created only one metadb partition instead of two, and found that the system would boot. I attributed this to a problem with the mirror disk, until it happened to me again this week. For some reason, the implementation of Disk Suite on Solaris 9 does not accept multiple metadb partitions.
Previously, in Solaris 8, I always created a total of four metadb partitions. Two on each drive…
#!/bin/sh
#Mirrorme.sh
prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2
metadb -a -f -c2 /dev/dsk/c1t0d0s3 /dev/dsk/c1t1d0s3
metadb -a -f -c2 /dev/dsk/c1t0d0s4 /dev/dsk/c1t1d0s4
Currently, with Solaris 9, that method does not work, and results in a kernel panic. To resolve this issue, you must create only one metadb partition on each disk. I’ve been using s3 for this, although you could use any slice you wish.
#!/bin/sh
#Mirrorme.sh
prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2
metadb -a -f -c2 /dev/dsk/c1t0d0s3 /dev/dsk/c1t1d0s3
Aside from this change, the mirroring scripts continue to work. Please let me know if you find any other problems not mentioned.
Recovering From a Stuck Symantec Ghost Virtual Boot Partition
It seems that when you manage a bunch of machines with Symantec Ghost, you always end up with one or two that, for whatever reason, get stuck in the Ghost Virtual Boot Partition, and can’t boot back into windows. This can be really frustrating because you usually know what went wrong, and need to get back into Windows to fix the problem.
The good news is that setting the machine is really easy once you know how. I went through years of booting into DOS, and running fdisk from a floppy whenever this happened to me before I realized that fixing it is just a simple command that exists on the Ghost Virtual Partition.
Here is how it is done:
- Stop Ghost and go to a DOS prompt:
- If running the Ghost DOS client, NGctdos.exe, type Ctrl+X
- If running the Ghost executable, Ghost.exe, type Ctrl+C
- Change to the Ghost directory.
- Type cd Ghost
- Type ngctdos -hide

This last command hides the Ghost Boot Partition, makes the Windows partition active, and restarts the computer. You can then move on to fix the problem with minimal fuss, and get on with distributing the virus known as “Windows”.
