Bare Metal Linux Restore

Several weeks ago we started seeing some pretty scary errors showing up on the main system disk for our Blackboard server. We had an extra server hanging around, so we decided to move all the data off the failing disk and onto our spare server. The only question was how to make the new server as close to a perfect copy of the old one as possible. Simply restoring all the filesystems failed for a variety of reasons, mostly related to GRUB and the kernel, so I had to find a way of excluding only the files and directories that were tied to the specific model of server.

To do this, I started by installing a minimal copy of RHEL 4, making sure to lay the filesystems out in exactly the same way as they were on the old server. I then went through several experiments, leaving just the bare minimum files and directories required for the hardware and booting, but formatting all other filesystems and restoring the data from our old server. In the end, the below process resulted in system that worked perfectly, and very closely mirrored the original server.
Continue reading

RHEL System Configuration Changes for Oracle 10G

Below is a list of RHEL system configuration changes that Oracle 10G requires before it is installed.

First, check the following kernel parameters using the commands below:


/sbin/sysctl -a | grep kernel.shmall
/sbin/sysctl -a | grep kernel.shmmax
/sbin/sysctl -a | grep kernel.shmmni
/sbin/sysctl -a | grep kernel.sem
/sbin/sysctl -a | grep fs.file-max
/sbin/sysctl -a | grep net.ipv4.ip_local_port_range
/sbin/sysctl -a | grep net.core.rmem_default
/sbin/sysctl -a | grep net.core.rmem_max
/sbin/sysctl -a | grep net.core.wmem_default
/sbin/sysctl -a | grep net.core.wmem_max

If any parameters are lower than the examples below, you will have to increase them by editing “/etc/sysctl.conf” file, adding the appropriate lines as expressed below. If the current value is higher, leave it as is.

kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144

Next, edit your “/etc/security/limits.conf” file, adding the following lines:

oracle          soft    nproc           2047
oracle          hard    nproc           16384
oracle          soft    nofile          1024
oracle          hard    nofile          65536

If your current “/etc/pam.d/login” file does not already contain the following line, add it:

session    required     pam_limits.so

Finally, add the following lines to your "/etc/profile" file:

#Tweaks for Oracle
if [ $USER = "oracle" ]; then
    if [ $SHELL = "/bin/ksh" ]; then
    ulimit -p 16384
    ulimit -n 65536
    else
    ulimit -u 16384 -n 65536
    fi
fi

These are just the basic steps I take. See the “Oracle Database Installation Guide” for more complete instructions.

How to Make Gnarly Big Linux Filesystems

At least in RHEL 4, the fdisk command does not support the creation of filesystems larger than 2TB. In order to get around it, you have to use the parted command. I found the basic info here, but this is the long and short of how to cut off a big ol’ slice of disk using parted:

Run parted

# /sbin/parted

It’s interactive, so the following commands are issued within the utility.

1) Make the disk label

(parted) mklabel gpt

2) Create the partition

(parted) mkpart primary 0 -1

3) Verify

(parted) print


Disk geometry for /dev/sda: 0.000-38146.972 megabytes
Disk label type: msdos
Minor    Start       End     Type      Filesystem  Flags
1          0.031    101.975  primary   ext3        boot
2        101.975  38146.530  primary               lvm

4) Exit the GNU Parted command shell

(parted) quit

5) Finally, make the filesystem:

# mkfs.ext3 -m0 -F /dev/sdb1

6)Finally, you don’t want to wait for that big filesystem to fsck from time to time, so make sure it does not get checked unless you run the command yourself:

# tune2fs -c0 -i0 /dev/sdb1

That should just about do it. Remember that only RHEL 4 and higher can support filesystems larger than 2TB. If I remember correctly RHEL 3 can go up to 2TB, RHEL4 can handle 8TB, and RHEL 5 can make a whopping 16TB chunk of disk. Have fun!

RHEL useradd Syntax

Unlike other flavors of UNIX, RHEL does not have a command like adduser which walks you through the process step-by-step, so you have to remember the four flags useradd requires, and in what order it expects to receive them. Since I don’t manually add users unless I’m installing a new server, I don’t run the command enough to remember the syntax… It’s basically the same as it is on Solaris.

useradd -g group -c 'User Name' -d /path/to/home/directory -s /bin/bash username

VMware Fusion Evaluation

Since much of my job involves rolling out Linux solutions I’ve been experimenting with VMware Fusion Beta for the Macintosh in my development environment. Given that the product is still in beta, I have very few complaints about its actual stability. Most of the features work reliably as advertised, but there are some basic points of functionality that I feel the software is lacking. More on that later.

First, let’s take a look at exactly what VMware Fusion is. At its core, the package allows the user to create and run virtual machines on the Macintosh. For those who are new to virtualization, it is a way to run multiple virtual computers on one actual computer. The hardware resources are abstracted and shared to the virtual machines through the virtualization software — in this case VMware Fusion. A complete description on virtualization can be found here.

Previous to Fusion, only VMware player was available to Macintosh users, so it is nice to actually be able to create virtual machines locally. The snapshot feature is also very nice for development purposes since you can instantly roll back to a previous working state should you corrupt the software on the virtual machine.

Perhaps the problem that annoyed me most, however, was the fact that there is no clear way to delete virtual machines from within the software. I actually tried to get rid of one by deleting this folder:

/Volumes/Macintosh HD/Users/myaccount/Documents/Virtual Machines/Mymachine.vmwarevm

But I just ended up breaking the “Virtual Machine Library” application and having to uninstall and reinstall everything from scratch. The process detailing how to delete a virtual machine did not exist anywhere in the VMware Fusion FAQ or documentation as far as I could tell. Granted, it’s beta software, but I would think this should be a core feature of any virtualization product. At least they provide an “Uninstaller” script.

VMware Fusion is a basic piece of software that succeeds in fulfilling the most fundamental of virtualization requirements. If all you want to do is be able to run a virtual machine or two on your Mac, it will most likely work for you. If you are looking to deploy it as part of an enterprise solution, I would suggest letting the product mature a while and using something like Parallels instead.

Disable SSH Root Logins on RHEL

For one reason or another RHEL does not disallow incoming ssh connections as root. This is, of course a glaring security problem which should be addressed for all systems that allow ssh connections to be made from any but the most restricted networks.

The best practice, of course, would be to make the initial ssh connection as an unprivileged user and then use the “su” command to promote yourself to root. This way, even if an attacker managed to get into the system, it would be as an unprivileged user and they would not able to do much harm. Allowing incoming ssh connections at root leaves you much more exposed to attack. Granted your root password is still protecting you, but it becomes your only layer of defense.

Ok, so how do we disallow incoming ssh connections as root on our RHEL box?

First, edit “/etc/ssh/sshd_config”

Find the section of the file that looks like this:

# Authentication:
#LoginGraceTime 2m
#PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6

Change this line:
#PermitRootLogin yes

To this:
PermitRootLogin no

Restart sshd:
/sbin/service sshd restart

RHEL Winbind Authentication Against Active Directory

So you have a RHEL system and you want to authenticate it against your active directory. The good news is that Red Hat has made it easy for you to do this. The bad news is that they only get the most basic structure working for you. Here I will show you how to get WinBind authentication working using Authconfig, and how make it a little more seamless than this utility leaves it off.

It should be noted that while this works perfectly well, it is really not the best way to authenticate users against a UNIX host. Given the option, having your users in Open Ldap and PAM authenticating them against that would be a much better option. However, we don’t live in a perfect world, and sometimes we just have to make things work.

Let’s start by using authconfig to join your machine to the domain. This should all be done as the root user.

# authconfig

  • Select “Use Winbind” and Use “Winbind Authentication”. Remember to leave “Cache Information”, “Use MD5 Passwords” and “Use Shadow Passwords” selected.
  • Select “Next”
  • Under “Security Model” select “ads”
  • “Domains:” examplead (substatute with the name of your Active Directory)
  • “Domain Controllers:” adserver.domain.com (Again, substitute with the name of your Active Directory server)
  • “ADS Realm:” ADSERVER.DOMAIN.COM
  • “Template Shell:” /bin/bash
  • Select “Join Domain”
  • Select “OK”

Now your machine should be be on the domain. Test it to make sure you can see your AD users:

# wbinfo -u

You should see your users in the list.

The only problem is that to do anything with them, you have to express their user name in that annoying way Windows likes you to. Something like this:

“EXAMPLEAD\\username”

Not very usefull. To get around this, simply edit “/etc/samba/smb.conf” and change this line:

winbind use default domain = no

to this:

winbind use default domain = yes

You should now be able to express AD usernames without the domain nonsense before it. Try it:

# finger username
Login: username                            Name: Username
Directory: /home/EXAMPLEAD/username        Shell: /bin/bash
Never logged in.
No mail.
No Plan.

Finally check your “/etc/nsswhich.conf” file to make sure RHEL knows to use WinBind. Authconfig should have set this up for you, and it should have lines that look like this:

passwd:     files winbind
shadow:     files winbind
group:      files winbind

That should do it you should be able to create home directories for all your AD users and let them authenticate away. Have fun.