Archives For 30 November 1999

Whether you’re using Mac OSX, Windows or Linux, we’re all using a so-called “window manager”. Most are graphical user interfaces; and that’s a good thing 🙂 But as a sysadmin I need to manage many servers. Servers without a graphical user interface. So, how to handle that?

One could just ssh into a server when you need to do work on a given server. Depending on the terminal program you use, you might be able to have multiple sessions at the same time, preferably in tabs. It looks like this when 3 tabs are open:

Although this works well, it has one drawback for me: it only works on one computer. And when you turn off your computer everything is gone and disconnected. Since I’m working on multiple computers (desktop/laptop), multiple OS’es (OSX, Ubuntu) and multiple places (Work, Home), this no longer worked for me and I started looking for a better solution.

GNU Screen to the rescue! GNU Screen is a full-screen window manager, but terminal based. That is, it works in interactive shells such as a ssh session and is able to keep running while disconnected.

Starting a screen is easy:

screen -S screenname

You can attach and detach a session when needed. To detach, press Ctrl+a+d. To reattach enter:

screen -r screenname

This means that no matter on what computer I login, on any place, I always am able to attach a running screen. It looks like this:

Note the bar at the bottom where the tabs are. You can even give them a name!

GNU Screen can be a bit obscure to configure. After googling a lot and some help from co-workers, I have now configured GNU Screen as you can see on the above image. Configuration is read from the .screenrc file in your homedir. In the image above you can see my .screenrc file.

Commands in GNU Screen are all prefixed by a control command, the default is Ctrl+a. This means that all commands you type will be entered in the terminal you’re connected to, except commands followed directly after you press Ctrl+a. In the manual page you’ll find C-a, which is short for Ctrl+a.

When you want to create a new tab, enter Ctrl+a+c. To change from one tab to the other, you press Ctrl+a+2 to go to tab #2. Ctrl+a+spacebar brings you to the next tab. To name that tab, enter Ctrl+a+A, etc. It takes a bit of time to get used to it, but for me it works very well.

GNU Screen is a window manager, so apart from multiple tabs, you can even split the window to host multiple screens next to each other. To split vertically you enter Ctrl+a+|. Ctrl+a+tab brings you to the newly created space. You then create a new tab there, like Ctrl+a+c. It looks like this:

Like I said, Ctrl+a+tab switches between the left and right screen. In each you can call any tab that is below with Ctrl+a+tabnumner, or toggle between them using the spacebar.

Need to go away and want to protect your screens? Enter Ctrl+a+x and you’re screens will be locked. You need to enter the password of the connected user to unlock.

This is just a quick introduction of what is possible with screen. Have a look at the man page:

man screen

or use Google to get some more configuration examples. Have fun!

Since I’m working with CloudStack I’m also working with CentOS. As I’ve been working with Debian for over 10 years, it sometimes takes some extra time to get things done. Like when you want to install a package that is not in the default repo’s.

This is how to enable the RPMForge repository:

Start to import the key:

rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt

Then download the RPMForge release file:

wget http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm

Then all you need to do is install this rpm:

rpm -i rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm

You only need to do this once. Now the packages from RPMForge can be installed using yum. Have fun!

Remote Linux systems usually have to be available all the time. Although Linux is rock solid and stable, an occasional crash can occur. For example when there’s a problem with hardware or software. The last thing you want, is that a Linux server got a kernel panic and then waits forever for someone to reboot it.

Of course there’s monitoring, and there are APC’s too, which can reboot the server. This is like pulling the plug. It usually takes some time for monitoring to notify a sysadmin and then for the sysadmin to reboot the server. And, when using IPMI devices (especially the older ones that share a network connection with the server), a kernel panic could make them unavailable, too. I’ve had that on many occasions. Then you end up driving to the data center or call someone to reset the server. I hate that 😉

In our setup, most servers are in clusters. This means losing one server should not be a problem. But you still want a server to be available again as soon as possible, to be able to handle future problems.

There’s a way to configure Linux to reboot automatically, say 10, seconds after a kernel panic occurs. This will quickly and automatically have the server up again. Should rebooting not help, then bad luck and there’s probably some failing hardware part. You’ve then to drive to the data center anyway.

So, how to do that? Well, there are several options to set this parameter. To test this out, use this command:

/sbin/sysctl -w kernel.panic=10

Note that this setting will not survive a reboot. If you want it to remain active, add this line to /etc/sysctl.conf:

kernel.panic=10

To check the current setting, issue:

cat /proc/sys/kernel/panic

It is wise to implement some sort of monitoring on reboots or uptime. You definitely want to read the logfiles and find out what exactly happened that led to the kernel panic.

Networking in CloudStack 3.0 is awesome; the Virtual Router provides many cool features like LoadBalancing, PortForwarding, (s)NAT, DHCP, VPN and so on. When a new network is created and being used, a Virtual Router is automatically launched to support these features. Since the Virtual Router is a Single Point of Failure, you should turn on the HA (High Available) option; which actually addes a 2nd Virtual Router on each network. While this is pretty cool, it makes the number of System VM’s go up and when you don’t need them it’s kind of wasting resources.

For example, when I was creating a network for the web servers to talk privately to the database, I didn’t need a Virtual Router. All I want is them to be able to do networking to each other and that is all. When using the default settings, a Virtual Router is launched anyway.

So how to tell CloudStack you don’t need a Virtual Router? Well, this is done through Service Offerings. You find this option on the menu at the left, it’s the last option. Select Network Offerings and a list is displayed.


Click Add Network Offering at the right and fill in the form. When you do not select any service, you’ll create a Network Offering for which CloudStack does not spin off Virtual Routers.

Now, when you create a new Guest Network, make sure to select the Network Offering you just created. This will make sure your new Guest network will have no Virtual Router launched when in use 🙂

Update: I’ve written another blog with more details on how to use this network. Also have a look at the comments in both blogs for some examples and idea’s. Feel free to ask me any questions you have below!

In a previous post I described howto restore a OpenLDAP server from backup . But how to backup Open LDAP?

The backups I make consist of two parts:

1. First backup the LDAP database itself using a program called ‘slapcat.’ Slapcat  is  used  to generate an LDAP Directory Interchange Format (LDIF) output based upon the contents of a given LDAP database. This is a text version of your database which can be imported later. Think of it as a SQL-backup for relational databases. Anyway, here’s how to run slapcat on the OpenLDAP server:

slapcat -l backup.ldif

This will backup the whole database into the file called ‘backup.ldif’. You can then use this file to restore an OpenLDAP server later, using slapadd. Be sure to run this in a backup script from crontab and have a backup at least once per day.

2. Second thing I do, is backing up the config of the OpenLDAP server. This config is usually in /etc/ldap. Back it up using a tar, or using a technique like rsnapshot.

When you have this in place (and save the backups on a different place), you’ll be able to rebuild an OpenLDAP server without problems.