SysAdmin Adventures

My DRBD SlackBuild has been updated for 8.3.7 and split into two: drbd-tools and drbd-kernel.

They have been submitted to SlackBuilds.Org and are pending submission. They have been updated for proper Slackware64 compatibility and include notes regarding DRBD entering the Linux Kernel as of kernel 2.6.33.

TAKE NOTE:
Slackware{,64}-current has moved to kernel 2.6.33. DRBD’s kernel code is now part of the Linux kernel. You should not attempt to install drbd-kernel on a system with a 2.6.33 or later kernel. While it is technically possible, version 8.3.7 will not compile externally against 2.6.33. A fix is pending for the next release of DRBD but is nonetheless not required.

Users of Slackware{,64}-13.0 and earlier should use both packages.

DRBD v8.3.7 Kernel Module SlackBuild
DRBD v8.3.7 Userland Tools SlackBuild

Have you just updated your WordPress.com Stats plugin? Does your stats page no longer show the graph? Would you like to know why?

The reason is because the plugin now includes a .htaccess file (/plugins/stats/.htaccess) and the .htaccess file contains:

By default, in a reasonably secure setup, the .htaccess file does not have permission to specify an Allow directive. This really shouldn’t be a problem as it should just be ignored, but it seems that for some reason, the fact that the directive is not allowed causes apache to actively deny access instead even though it would work perfectly if the .htaccess file weren’t there at all. I don’t know why, it just does.

One solution would be to delete the .htaccess file, but I prefer not to adjust the plugin as a simple adjustment to the apache configuration will solve the problem. Apache needs to be told to allow the .htaccess file to specify an Allow directive using the AllowOverride directive in the main configuration.

Those who have implemented Permalinks will already have included:

All you need to do is extend it to include Limit as well:

You could achieve the same result with:

I heavily recommend against it as it has the potential to be a security threat.

Let me start by clarifying that I’m actually talking about slackware64-current as of Tue Mar 2 19:07:31 UTC 2010, but for longevity I have titled the post 13.1 because I believe -current to be pretty close to a 13.1 release and I will be updating this server to 13.1 at release and halting updates there.

That said, Symantec have released (Veritas) NetBackup 7. Woo. As insane as it sounds, NetBackup 6.5’s Linux server component was only available for x86 and IA64, but NetBackup 7’s Linux server component is only available for x86_64. Way to screw everybody over, Symantec(!)

Thankfully, this just happens to be beneficial for me as I am at the point of being able to move our NetBackup server from Solaris10-SPARC to Slackware Linux and it means that I get to deploy it on Slackware64. In this case, the server I am using has had alienBOB’s Multilib Conversion; but this is not necessary to support NetBackup 7.

Perversely, the new “OpsCenter” (x86_64) that Symantec won’t shut up about does require 32-bit libs, however it’s also much more tightly bound to RHEL/SLES. NetBackup Server works well out of the box in Slackware, but OpsCenter won’t install on anything but RHEL or SLES and manually extracting the RPMs and forcing it in is much more work that I can be bothered with for a component I probably won’t even need. As far as I can tell, the main reason they are giving away OpsCenter is because they want managers to see it so that they can persuade them to buy OpsCenter Analytics which I’m certain will not be cheap.

That’s the ranting out of the way. Onto the details…

Installation really is a breeze. I used the RHEL version of the installer (NetBackup_7.0_LinuxR_x86_64_GA.tar.gz) not the SLES version (NetBackup_7.0_LinuxS_x86_64_GA.tar.gz) because I am more familiar with RHEL than SLES and I have always been successful in the past using Veritas’ RHEL releases on Slackware. Feel free to play with the SLES version and report back.

Extract the tarball and run the installer. I used the defaults for pretty much every option. Feel free to change the install path from /usr/openv to /opt/openv for correctness, but I am personally very familiar with /usr/openv (as is the software, obviously) and the software lives almost entirely in that subdirectory without polluting the rest of the machine; so it’s a safe choice.

The installer will modify the inetd configuration and add entires to your /etc/services file and put two startup scripts (netbackup and vxpbx_exchanged) in /etc/rc.d/init.d. It will also put symlinks to these in the run-level numbered /etc/rc.d/rc*.d directories. Worth remembering that under Slackware /etc/init.d is a symlink to /etc/rc.d/init.d. In my previous post about running NetBackup client software on Slackware (http://blog.tpa.me.uk/2009/08/30/veritas-netbackup-slackware/) I suggested that you should remove the symlinks and move the netbackup startup script to /etc/rc.d/rc.netbackup. Technically I still believe that advice as I am somewhat of a purist, but I can also be pragmatic when necessary. Slackware has a startup script deisgned to deal with SysV-style scripts called /etc/rc.d/rc.sysvinit and there’s no real reason you shouldn’t just let it do its job and leave the NetBackup scripts where they are. It will certainly help out when it comes to upgrade time; you just have to remember to use /etc/init.d/netbackup (shortest option) rather than /etc/rc.d/rc.netbackup.

Now it’s time for…
The bit that will catch you out:

As is widely known, Slackware still does not contain PAM. There are reasons this is good and there are reasons this is bad. Pat is well aware of all of these reasons and continues to consider his options. In the meantime, we have to be aware of the lack of PAM and work around it where necessary (assuming you don’t choose to install PAM yourself, I know I don’t).

Awkwardly, there is one component of NetBackup server that requires PAM: /usr/openv/netbackup/bin/bpjava-msvc. Unfortunately it’s a critical component. In order to administer a NetBackup server, you use the Java NetBackup Administration Console. When you connect to the NetBackup server with the console, the server authenticates the request by launching the authentication service from inetd; which is bpjava-msvc. Unfortunately for us, and for anyone else that doesn’t have PAM installed (even if they are running RHEL) bpjava-msvc is linked against the main PAM library libpam.so.0 and cannot launch without it:

I have spoken with Symantec about this and they clearly state that NetBackup is not dependant upon PAM, in its absence the Java client authentication service will happily authenticate against shadow, it just happens that the binary is linked to the pam library so needs it to be present to run. While there’s not much they are prepared to do about it right now, there is an “Idea” in the Symantec Connect portal’s Ideas section that requests the development team deal with this issue, and it is the right channel for communicating with product development. If you would like to see the issue dealt with, I heavily recommend you “Vote Up” the idea to bring it to their attention:
https://www-secure.symantec.com/connect/idea/bpjava-msvc-linked-against-pam-unnecessarily-making-it-dependency-entire-application

In the meantime, there is a really ugly workaround. It wants libpam.so.0? Well give it one; just not the rest of the PAM package. If you get the latest PAM source and compile it, you can copy the libpam.so.0 file into the NetBackup lib directory and voilá; everything works perfectly. While you could put it in /usr/lib64 or put it in /opt/pamlib and add /opt/pamlib to /etc/ld.so.conf, I recommend putting it in the NetBackup installation to save from unnecessary modification or pollution of your system. KISS. So put it in /usr/openv/netbackup/lib.

And that’s all there is to it. Now to add your tape/disk/removable devices and setup your backup policies.

Thanks to Symantec, migrating from my Solaris10-SPARC system is not an option. Even changing the hostname of the NetBackup server is not a supported operation and will break everything unless handled very delicately. Symantec say the only way they will help you is if you pay them a bucketload for Symantec Consultation Services. Well I can tell you now that just isn’t going to happen. In practice this means I am going to have to import all historic tapes into the new system (http://seer.entsupport.symantec.com/docs/327873.htm), but thankfully I have a Sun StorageTek C4 Robotic Tape Library that will hold 32 tapes at a time and I only need to import 60 or so tapes. It will probably take the best part of a week, but it’s acceptable.

YMMV

Some days are just better than others.

I’ve just finished completely eviscerating a powerful server that was running Windows 2003 R2.

It’s now running Slackware64-current and very very beautifully so:

Kernel: 2.6.32.7
Kernel Image Size: 2299 KB
Module Tree Size: 33936 KB

CPU: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz

Total RAM Available: 4054384KB
Total RAM Used (+/- b/c): 42240 KB

/dev/sda: 6x SAS disks
MegaRAID hardware controller
4 disk in RAID10.
2 global hot spares.

hdparm -tT /dev/sda:
/dev/sda:
Timing cached reads: 8206 MB in 2.00 seconds = 4106.44 MB/sec
Timing buffered disk reads: 552 MB in 3.01 seconds = 183.66 MB/sec

It’s name according to the Simpsons naming scheme?

Wolfcastle

This is a very simple thing, but it seems not widely known.

You login to, or start, X as a regular user as you rightly should, but you want to be able to su to root and then run a GUI application as root.

There are a number of ways to do this, some better than others, including using “kdesu”, “xhost” etc; the simplest however is:

… where xuser is the user you are running X as. The export line may not be required depending on how you became root.

The root shell you are in now has full authority to communicate with and manipulate X.

I am pleased to report my desktop is no longer Slackware-13.0, but is now Slackware-current and using KDE SC 4.4.0 as provided by alienBOB: http://alien.slackbook.org/blog/its-been-released-kde-sc-4-4-0/

I’m very pleased with the new KDE. With the exception of the polkit argument that’s going on every new release of KDE brings fresh delight, and I hope it never stops.

There’s one little thing it took me a short while to realise and so is worth mentioning here. I use “Desktop Grid” and “Present Windows” a lot and have them mapped to the top corners of the desktop for quick-access. After upgrading to KDE SC 4.4.0, my panel started disappearing whenever I used “Desktop Grid” and, while it’s not exactly a problem, it doesn’t look very good. The panel always disappears in “Present Windows” but that is expected behaviour.

The reason I discovered is that “Desktop Grid” now includes a per-desktop “Present Windows” by default, so if you have multiple windows open in a desktop, they will be “Presented” within that desktop in the grid view, and for this reason the panel disappears. Some may like this feature, and I see how it could be very useful, but I don’t want it. As I have, you can turn it off:

System Settings -> Desktop -> Desktop Effects -> All Effects -> Desktop Grid (Configure)

Uncheck “Use Present Windows effect to layout the windows”.

If following a new install (or an upgrade for people not previously using Akonadi/Nepomuk/Strigi) Akonadi is whinging at you because the mysql log has errors such as this:

[ERROR] Can’t open and lock privilege tables: Table ‘mysql.servers’ doesn’t exist

Then the following should sort you out:

It worked for me. Your Mileage May Vary.

DO NOT follow this if you have Akonadi data you do not want to lose.

Sometimes it is necessary to retain a Slackware installation, but change the hardware it runs on:

• Migrating to or from virtual hardware (VirtualBox, VMware etc)
• Duplicating an installation across multiple new servers
• Using temporary hardware to set up a new OS for a server to minimise downtime
• Plain old hardware failure
• etc.

Under many other Operating Systems, especially Windows, this can be painful and perhaps not even worth doing. As usual it is easy with Slackware.

In the simplest case (default install, default kernel, desktop environment) there’s literally nothing you need to do. Just put the disk or image in the new hardware and boot.

If you are using a custom kernel (as you should be) then you will need to create a new kernel and update lilo either before or after you do the migration. If you have any sense of self-preservation, your lilo config will include the default “huge” kernel and so the minimum you need to do is just boot the huge kernel on the new hardware, then you can go about making a new custom kernel later.

The bit that will catch you out:
There is one thing that might make you stumble: you moved the installation, booted up and all is well.. but the network isn’t working. You run `ifconfig -a` and find your two network interfaces are now called eth2 and eth3 and neither is configured.. “What’s going on?”, you ask. The answer is udev.

udev knows that your two new network interfaces are different to your old ones because they have different MAC addresses and decides you might not want to have them configured the same; perhaps you are going to swap in the old cards later and have four, so it reserves the previously used “ethX” labels in case it sees those cards again.

Since you are migrating to new hardware, you want it to forget about your previous network interfaces and re-use the labels with your new hardware. Head to /etc/udev/rules.d and find the file called 70-persistent-net.rules. Take a look at it.. it should look something like this:

See how it has an entry for both the old interfaces and the new ones? That’s what’s meant by persistent net rules. If you moved to a new machine again, it would add the two new interfaces as eth4 and eth5.

You have two choices for fixing the situation:

1. Delete the file. It will be re-created on next boot and start again from eth0.
2. Manually modify the file to reflect the configuration you want.

Note: There is also a 70-persistent-cd.rules file that it’s worth keeping an eye on during hardware migration, but usually it’s only the net rules that actually cause people problems.

Finally got around to making a SlackBuild for DRBD. Made a little easier by the introduction of a configure script into the DRBD source.

The script is available below and has also been submitted to SlackBuilds.Org.

Have fun.

DRBD SlackBuild (v8.3.6)

• SlackBuild updated on 12/11/2009 for better handling of kernel source dir.

UPDATE 20100304:
DRBD 8.3.7 has been released and new SlackBuilds are available:
http://blog.tpa.me.uk/2010/03/04/drbd-8-3-7-slackbuilds/

OpenLDAP Software is an open source implementation of the Lightweight Directory Access Protocol.
The suite includes:

* slapd – stand-alone LDAP daemon (server)
* libraries implementing the LDAP protocol, and
* utilities, tools, and sample clients.

There are a few things I’d like to mention about OpenLDAP in Slackware-13.0 and so I might as well deal with them all in one post.

Running an OpenLDAP Server

I can’t remember why, but Pat made the decision not to provide slapd, the OpenLDAP server daemon in Slackware by default. This has been the case for some time. Instead, what you get is openldap-client which is just the client libraries enabling you to use OpenLDAP code in LDAP client applications and LDAP-dependant software.

If you want to run an OpenLDAP server you are going to need slapd. Thankfully this is incredibly easy to do because all Pat did to remove slapd was to configure the OpenLDAP source with:

If you want slapd (and perhaps slurpd) then you just need to rebuild the openldap sources using the SlackBuild Pat provides in the Slackware source, but with the word disable replaced with enable. However, there is a catch. In Slackware-13.0 OpenLDAP will not build with the source and SlackBuild Pat shipped because the OpenLDAP package has not been rebuilt since 24/07/2008 and since then a change has been made to the C compiler that prevents OpenLDAP from building correctly unless you append the flag -D_GNU_SOURCE. This wasn’t detected early enough for it to be fixed for the 13.0 release. Normally this isn’t an issue because the package still works perfectly, but if you want to rebuild it you have to modify the source or the SlackBuild to do so.

Assuming your Slackware source is located at /mirror/slackware/slackware{64}-13.0:

Working in vim so you can see what you’re doing.. (you could replace however you wanted, e.g. with sed, or manually, or whatever):

Then go and have a sleep for half an hour as it builds and then tests just about every conceivable LDAP operation.
Once it’s done:

Job done.

Using OpenLDAP for User Authentication

In other distributions, setting up LDAP user authentication for the OS is done with Linux-PAM. While, mercifully, Slackware does not have Linux-PAM (and probably never will) it does mean that LDAP user authentication is harder or more impractical.

Note: I do not recommend doing LDAP user authentication for your Operating System. It’s not necessary in most cases. LDAP authentication should be done at the application level for specific services provided to users. The Operating System should only be authenticating users that actually log in to the system and in most cases LDAP is not needed for this. (Don’t argue if it is truly necessary in your case – you’re an exception).

If you are absolutely sure you definitely need LDAP user auth for Slackware, your options are:

Install PAM

This is an epic challenge and few succeed. Since PAM is evil anyway, do not bother. If you choose to take this route and succeed, then document your adventures and link to them.

Use nss_ldap

The resolution of the entities defined in RFC 2307 is generally performed by a set of UNIX C library calls (such as getpwnam() to return the attributes of a user). The nss_ldap module provides the means for Solaris and Linux workstations to this information (such as users, hosts, and groups) from LDAP directories. The module is the reference implementation of RFC 2307, and has been studied by vendors such as Sun (who developed the original Name Service Switch interface).

This basically allows you to use nsswitch.conf to redirect user entry queries to an LDAP server and seems the natural way to do LDAP auth in Slackware, but there is drawback. In normal circumstances LDAP authentication is done with a bind. The user supplies a username and password and the application submits a login request (a bind operation) to the LDAP server and the LDAP server returns success or failure depending on whether the credentials are valid. This is not possible with nss_ldap because nss_ldap doesn’t replace the authentication method, it just redirects it to a different storage back-end. The effect of this is that to use nss_ldap you have to store the users’ passwords in the LDAP server using the same hashing algorithm you would use in the passwd file (crypt()).

I’m not going to go into any further detail on this, you are going to have to do the research yourself, but suffice it to say nss_ldap is not ideal, but it is the cleanest method and probably the one I would use if I had to.

Use NIS and a ypldapd

The NIS/LDAP Gateway, or ypldapd, is a Network Information Service (NIS) server which uses LDAP as its information source. It permits existing NIS clients to transparently use LDAP to resolve user, group and host information. Enterprises can thus realize the benefits of LDAP, such as its distribution and being scalable, without upgrading clients.

I personally haven’t tried this and can give no reasoned opinion on it, but if you are migrating from NIS to LDAP it seems like a reasonable way to get going with it and may also cover the drawbacks of the nss_ldap solution – but it has it’s own drawbacks.

1. You must have or implement a NIS network.
2. It’s not free.