Using Sort to List Directories by Size

If you manage a UNIX system with a large number of directories that vary in size, chances are that you’ve needed to figure out which ones are using up the most disk space. Of course if the directories are user accounts, the best way to do this is to enable quotas and use the “repquota” command. If you just have a bunch of directories, however, you can easily figure out which ones are largest by giving the correct arguments to “du” and “sort”. Here is how:

du -sk * | sort +0nr

This will display the size of all directories and sort them from largest to smallest. If you want to sort them from smallest to largest, simply remove the “r”.

du -sk * | sort +0n

If you have nested directories, you will need to incorporate foreach to recurse through and get all the directory names.

REL 3 Direct Connect to EonStor A12F-G2221

This summer we have been migrating a bunch of data to our shiny new InfoTrend EonStor A12F-G2221. With 1G battery backed cache, it’s a screaming box of disk, and it looks cool to boot. There is a gotcha though if you want to direct connect it to QLogic QLA2340 card on a REL 3 server. Here is what you have to do.

First, get the new driver from QLogic, or install the one that came on CD with the HBA. The one that Red Hat packages is always old and useless, and one that QLogic provides is better anyways because the installer rebuilds the rdimage for you. Once you get the package just “cd” into the “qlafc-linux-X.XX.XX-X-install” and run “qlinstall”. This will install it all for you, so let it do it’s thing, and reboot the system when it’s done.

Now, go into the management console for your EonStor A12F-G2221. For the most part, the system defaults should work, but InfoTrend sets the default Fibre Connection to “Loop Only”. This is fine if you are dealing with a san, but since we are trying to do a direct connect, we have to change it to either “Auto” or “Direct Connect”. I suggest “Auto”, since that way you can have the other port connected to a loop if you want.

EonStor

That should be all you have to do. You will have to reboot the controller for the change to take effect, so make sure you do this during a scheduled downtime if you have the disk in production.

Sun Talks Some Smack

We all know that Sun is MUCH cooler than Dell, but the company is finally coming out and saying it in no uncertain terms. Apparently They have tried to get these advertisements into several major industry publications, but have been censored at every attempt because someone may find the material offensive.

Yeah… Sure… Much more likely, the editors of these publications are just afraid of loosing Dell as a client. I, for one, think the ads are pretty funny, and I’m glad to see Sun taking on Dell, because I truly feel they have a better product.

Sun says:

Sun Censored but Not Silent
Top business publications refused to run our bold ad concepts because the headlines were thought too controversial. At Sun, we’re the radical engineers that build “ass-whoopin” technology – we’re not Miss Manners and we never want to be. We ask all you contrarians out there to e-mail us your own provocative ad headlines: my-headline@sun.com. See the remarkable benchmarks for the new Sun Fire tm X2100, X4100, and X4200 64 bit X86 servers and you’ll understand why we have license to brag.

They’re not quite as cool as “Benchmark Studies Prove That Dell Sucks” or “100% More Bitchin’ Than Dell”, but Sun is currently running some fairly edgy ads that are along the same lines. I just wish I could pick up “SysAdmin” and find the ads below in them.

Click on the small images below to take you to the larger versions.

NetBackup, Solaris 9, and LTO2 drives

If you are using Veritas NetBackup on Solaris 9 with LTO Ultrium-2 tape drives, you will be constantly annoyed by slow tape write performance unless you use blocks of at least 256KB.

To resolve this, the first thing you’ll want to do is increase both the number of buffers and the buffer size on the media manager host:

Create and edit the file: /usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS and add 262144 on the first line.
Create and edit the file: /usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS and add 16 on the first line.

These numbers with their respective quotes should be the only thing in these two files.

Next, since Solaris still has insanely low limits on its default shared memory subsystem, we have to increase these limits as to not exhaust them with the increased NetBackup Buffer sizes. We do this by editing the /etc/system file and adding the following lines.

set msgsys:msginfo_msgmni=1024
set msgsys:msginfo_msgtql=1024
set semsys:seminfo_semmni=2048
set semsys:seminfo_semmns=2048
set semsys:seminfo_semopm=128
set shmsys:shminfo_shmmax=33554432
set shmsys:shminfo_shmmni=512

It is now necessary to reboot the system for the kernel parameters to become active.

You should now notice a dramatic increase in tape write speed during your backups.

Interview with the Brothers Chaps


Interview with The Brothers Chaps from Cliff Pearson on Vimeo.

Several months ago my friend Chuck found this interview with the guys that put together homestarrunner.com. The “Brothers Chaps”, as they refer to themselves, have created an internet cartoon series with flash that has quickly gained popularity. Most new cartoons come out on Mondays, which gives us all something to look forward to, and helps make the first day of the week tolerable.

As fans of the site will discover, the coolest thing about this video is that you finally get to see the people behind the voices. On the screen is a guy you’ve never seen, but out of his mouth is coming Strong Bad’s voice. The voice of Marsipan looks nothing like I had imagined she would, but why should she? I don’t really know why I expected her to look like a broom in the first place.

Sorry it’s an WMV file. We don’t always get to pick the way these things come to us, but check out the video… I think you’ll enjoy it.

Watch the video >
Check out homestarrunner.com >

What’s up with the new CMS?

There are quite a few reasons I decided to move away from PhPNuke, but the biggest of the all was the fact that it is simply FULL of security holes. Day after day I read about SQL injection vulnerabilities that would allow a user to gain administrative access to the site, and go at it with a hatchet. I was simply tired of always worrying that someone would hack my site and leave me with a big headache.

I was also getting increasingly frustrated by the fact that PhPNuke never supported pingbacks, trackbacks, or xmlrpc. I read something months ago about someone developing trackbacks for PhPNuke, but it would seem he went the way of Jimmy Hoffa, and nobody ever heard from him again. As time went on, it was becoming more and more clear that PhPNuke needed to be abandoned for a better product, and the only question that remained was which product to chose.

I’ve always liked Mambo, but while it is much more secure than PhPNuke, it still lacks pingbacks, trackbacks and xmlrpc. Drupal is a very nice product that does everything I wanted it to do and more. There is even a script that will migrate all the users and data from PhPNuke into Drupal, but in the end, I was just not happy with the look and feel of the site under Drupal.

Casey over at maisonbisson.com had recently moved his site from P Machine to Word Press, so once I confirmed that it had Gallery2 Integration I decided to take a look to see what he liked so much about it. He had written a really cool statistics plugin called bstats, and overall the CSM presented things very cleanly. Overall, I’m pretty happy with the software. It does everything I want it to, and it’s quite nice to have the option of managing my stories through xmlrpc with Ecto.

Casey Bisson’s bstats plugin > 
Word Press Gallery2 integration >

 

Copy a Solaris Boot Drive to a New Disk

If you’ve ever gone to mirror a system drive with Solstice Disk suite, you know how frustrating it can be when you either don’t have any more slices to use for your meta database partitions, or all the space on the disk has already been allocated to existing partitions. While Disk Suite only requires one partition be reserved for its meta database information on boot drives, two are really suggested for redundancy purposes, and in the example below, I found myself needing to mirror a system disk that had only one remaining partition, and no space left that could be used for the meta database.

While I could have taken a small amount of space from the swap partition and re-allocated it to a new meta database partition on slice 7, this solution would not have been elegant, and I would have still only had one meta database partition. As it stood, the system had the following filesystems on the following disk slices:

c1t0d0

Part Tag
0 root
1 swap
2 backup
3 usr
4 usr/local
5 opt
6 var
7 unused

In order to bring the system into line with my standards and prepare it for proper mirroring, I would have to carve up another disk, and migrate the data to it.

Here is what the partition table on the new disk looked like:

c1t2d0
Current partition table (original):
Total disk cylinders available: 24620 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks
0 root wm 0 1088 1.50GB (1089/0/0) 3146121
1 swap wu 1089 6896 8.00GB (5808/0/0) 16779312
2 backup wu 0 24619 33.92GB (24620/0/0) 71127180
3 wm 6897 6967 100.16MB (71/0/0) 205119
4 wm 6968 7038 100.16MB (71/0/0) 205119
5 opt wm 7039 8853 2.50GB (1815/0/0) 5243535
6 usr wm 8854 12483 5.00GB (3630/0/0) 10487070
7 var wm 12484 24619 16.72GB (12136/0/0) 35060904

Now that everything is all laid out, we can start moving all the data from c1t0d0 to c1t2d0, keeping in mind that we will be merging /usr/local onto /usr on the new system disk… Here we go

Make a new filesystem for /:

# newfs /dev/rdsk/c1t2d0s0
newfs: /dev/rdsk/c1t2d0s0 last mounted as /
newfs: construct a new file system /dev/rdsk/c1t2d0s0: (y/n)? Y

Mount the new / filesystem as /mnt:
# mount -F ufs -o rw /dev/dsk/c1t2d0s0 /mnt
Move the data from c1t0d0s0 to c1t2d0s0:
# ufsdump 0f – / | ( cd /mnt ;ufsrestore xvf – )
Add links
Set directory mode, owner, and times.
set owner/mode for ‘.’? [yn] y
Directories already exist, set modes anyway? [yn] y
DUMP: 405886 blocks (198.19MB) on 1 volume at 406 KB/sec
DUMP: DUMP IS DONE

Unmount /mnt
# umount /mnt

That’s the general idea… Now we just have to do the same thing for the other partitions, leaving out swap, backup, and our two meta database partitions of course. These partitions (1,2,3 and 4) should be left alone for the time being, as they are never mounted as filesystems.

# newfs /dev/rdsk/c1t2d0s5
# mount -F ufs -o rw /dev/dsk/c1t2d0s5 /mnt
# ufsdump 0f – /opt | ( cd /mnt ;ufsrestore xvf – )
# umount /mnt
# newfs /dev/rdsk/c1t2d0s6
# mount -F ufs -o rw /dev/dsk/c1t2d0s5 /mnt
# ufsdump 0f – /usr | ( cd /mnt ;ufsrestore xvf – )
# umount /mnt
# newfs /dev/rdsk/c1t2d0s7
# mount -F ufs -o rw /dev/dsk/c1t2d0s5 /mnt
# ufsdump 0f – /var | ( cd /mnt ;ufsrestore xvf – )
# umount /mnt
Finally, the /usr/local partition
# mount -F ufs -o rw /dev/dsk/c1t2d0s5 /mnt
# ufsdump 0f – /usr/local | ( cd /mnt/local ;ufsrestore xvf – )
# umount /mnt

Now that we have all the data moved, we still don’t have a disk that is bootable. Since the whole idea here is for us to end up with a new bootable system disk, we have to install bootblocks onto the new system disk. This is done with the installboot command:

# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t2d0s0

Now that we have the bootblocks needed to boot the solaris kernel, the last thing we have to do is make sure our new vfstab file points to all the right partitions.

Mount the new / partition:
# mount -F ufs -o rw /dev/dsk/c1t2d0s0 /mnt
Edit the new vfstab file:
# vi /mnt/etc/vfstab
For the information given in this example, this file will contain the following entries:

#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd /dev/fd fd no
/proc /proc proc no
/dev/dsk/c1t2d0s1 swap no
/dev/dsk/c1t2d0s0 /dev/rdsk/c1t2d0s0 / ufs 1 no
/dev/dsk/c1t2d0s5 /dev/rdsk/c1t2d0s5 /usr ufs 1 no
/dev/dsk/c1t2d0s6 /dev/rdsk/c1t2d0s6 /var ufs 1 no
/dev/dsk/c1t2d0s7 /dev/rdsk/c1t2d0s7 /opt ufs 2 yes
swap /tmp tmpfs yes

Notice that the target number will remain 2, not move to 0 when we swap the disks and boot from the new one. To resolve this, it is strongly suggested that you rebuild the solaris device tree and change the vfstab file to reflect the new disk position.

That is everything! We now shutdown the system, swap the positions of c1t0d0 and c1t2d0 and reboot off our new system disk. We are now ready to move onto the mirroring process.