1 2 3 ... 10

Size of a blank disk

I was trying to figure out how to get the raw size of a blank disk (no MBR). For the Windows readers, I am talking about disks that have not been “signatured” yet.

Intuitively, you only need to get the c/h/s or lba values and some 5th grade Math knowledge. This is how generally tools like parted work.

The tricky part is to get these values.

If you try to read the geometry values from the MBR, as parted does, it won’t work with a blank disk. cfdisk and grub also read the geometry from the partition table but fall back to ioctl()’s HDIO_GETGEO if they couldn’t read the head/sectors from the MBR. Have a look at the following patch:

http://svn.debian.org/wsvn/pkg-grub/grub/trunk/debian/patches/geometry-26kernel.diff?op=file&rev=0&sc=0http://svn.debian.org/wsvn/pkg-grub/grub/trunk/debian/patches/geometry-26kernel.diff?op=file&rev=0&sc=0

But, as described by hdio.txt, the values returned by ioctl() are not meaningful for modern drives:

Not particularly useful with modern disk drives, whose geometry is a polite fiction anyway.  Modern drives are addressed purely by sector number nowadays (lba addressing), and the
drive geometry is an abstraction which is actually subject to change.  Currently (as of Nov 2004), the geometry values are the "bios" values -- presumably the values the drive had when Linux first booted.

In addition, the cylinders field of the hd_geometry is an unsigned short, meaning that on most architectures, this ioctl will not return a meaningful value on drives with more than 65535 tracks.

The start field is unsigned long, meaning that it will not contain a meaningful value for disks over 219 Gb in size.

(a thread on the linux-kernel ml on the topic: http://www.uwsg.iu.edu/hypermail/linux/kernel/0602.1/0898.html)

The so called “bios” values are the one returned by INT13 (see arch/i386/boot/edd.S in the kernel source):

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=arch/i386/boot/edd.S;h=4b84ea216f2b81ede738e80206a5fafd8e5ef994;hb=2c3ca07d2f691a463b715c83122c87f57c2a7b7c

So basically, the safe way to go is only to rely on the number of blocks (for modern drives and bios supporting lba, this should be the ‘real’ value; for c/h/s, you need to trust your bios) and assume 512 bytes for the block size. Note that the sysfs knows the number of blocks ( see /sys/block/*/size).

I’m still wondering how you can check for the block size. Is 512 considered as an industry standard?

Autotools and private toolchain

How do you compile code using autotools against a private toolchain?

This is the issue I faced today. I needed to build bash (why does Ubuntu link bash against ncurses?!) against an external toolchain (tree containing specific version of gcc, libc, ld, nm, as, ...).

Sure you can specify CC='your-version-of-gcc' but what about PERL? or even LD? You don’t want to use the gcc from the toolchain and link against your local libc (what about `ldconfig’?).

Here is the solution I came with. Tell me if you have a better one.

In my main makefile, I first define my utils:
YACC            := $(TOOLCHAIN)/bison-2.3/bin/yacc
(do the same for nm, gcc, perl, cp, ...) I then declare explicitly a PATH:
YACC_PATH       := $(shell dirname $(YACC))
TOOLCHAIN_PATH  := $(TOOLCHAIN_PATH):$(YACC_PATH)
And then run the configure script, explicitly redefining PATH in a subshell:
bash-configure:
  PATH=$(TOOLCHAIN_PATH); pushd bash-3.2; \
  $(SHELL) configure $(MYCONFIGUREFLAGS) CC=$(CC) LDFLAGS=$(MYLDFLAGS); \
  popd

Passing the PATH will break the build if a utility is called which has not been explicitly declared (sed, awk, ...) which is good. I don’t want the build to use any of my local tools.

I also pass explicitly when I `make all’ AR, NM, OBJCOPY, STRIP, ... basically all the binutils. It is probably unnecessary, but it is ‘in case of’. Note that you need to double check `ld’ and `as’. If they are configured when you compiled `gcc’ (—with-ld=...), you don’t want to specify them again (that could lead to some strange behaviors).

Also, be sure to check `gcc -print-search-dirs’ for header and library files paths.

More is Less -E (More or Less)

I had an interesting issue today: how to deal with the environment variable PAGER?

Here’s the big picture: not all distributions use the same default pager. You have on one side the more fans (Debian) and on the other side the less ones (Suse). I’m not taking into account the geeks who use something else (probably well tweaked).

Imagine now you want to display to the user your EULA in an installation script. How should you display it? Using more by default? Or PAGER? The second solution seems intuitive to me. After all, if I use a special, personal pager, I would be totally frustrated to have to deal with more when reading the EULA.

However, think about the beginner Suse user, who doesn’t know much about less. Reaching EOF, less will stop scrolling, if neither -e nor -E are specified. You need to type q to exit. The beginner would have no idea what to do to exit less.

The solution I came up with is to either use more if no pager is specified, and use PAGER otherwise. In the latter case, if is it less, -E is appended at the command line, with the regular flags the user might have specified in PAGER.

The solution works well. If no pager is specified or less is, the behavior will be to exit the pager when EOF is reached and the script can continue (“Do you accept…”). If a special, tweaked, pager is specified, it is very likely that the user specified it, so he know how to use it. And he won’t be frustrated when reading the text.

Emulate a repository with rsync

I was working the other day on Dateslider, a fancy Javascript library to create sliding date pickers (see the demo here).

After having refactored 300 lines of Javascript and added two classes, I realized it was time to backup my work! Unfortunately, I have no access to the initial repository. I could have simply zipped it and sent it to an external hard drive but I wanted to keep an history of my changes. I could have created a personal repository but sending files to the upstream team would be painful with all the .svn/ subdirectories…

I finally decided to go with rsync and emulate a repository.


#!/bin/sh
# svn.sh

# Directory to backup
DIRECTORYTOBACKUP=${1%/} # Remove trailing slash

# The name of the backup machine
# Don't even try to hack this CNAME, it doesn't exist.
BACKUPSERVER=backups.mouraf.org

# Auto-generated directory containing modified files
INCREMENTALDIR=`date +%m_%d_%y-%H_%M_%S`
OPTS="--progress --force --ignore-errors --delete --backup --backup-dir=/BAK/$DIRECTORYTOBACKUP/$INCREMENTALDIR -a --compress" 

rsync $OPTS $DIRECTORYTOBACKUP backups@$BACKUPSERVER:/BAK/$DIRECTORYTOBACKUP/current

Note: refer to the rsync man page for details about available options. Have a look to “delete-excluded” and “exclude-from” which are especially helpful to avoid transferring Virtual Machines ram disks for instance…

With the command ./svn.sh dateslider, you end up backuping the entire directory dateslider to /BAK/dateslider/ on $BACKUPSERVER (you don’t have to send it to another server, you may simply backup it to an external drive for example). The current up-to-date version is in the current directory.

Each time you modify a file and fire up the script, another directory is created ($INCREMENTALDIR) with the old version of the modified file and the new version is updated in current.

I have aliased this script to commit on my machine. Once in a while, I run it. Hence I always have a current copy of my data and all the diff I may need. Not as powerful as a real repository, but helpful and secure for small and/or temporary projects.

watch - display a progress bar

As Julien pointed out, the Debian Administration website has some nice articles about “Commands you might have missed”. I came across this article ab out watch, which repeatedly run a command.

I personally use it in some backup scripts to display a progress bar. Assuming I want to copy a directory with 18 files , I can monitor the progress on the remote machine with this simple command line:

watch 't=`ls -la | wc -l`;perl -e "print \"[\",\"x\"x($t),\".\"x(18-$t),\"]\n\";"'
which outputs something like:

Every 2.0s:
[xxxxxxxxxxxxx.....]

Each x represents a file which has already been transferred. In this example, I can see quickly that 5 files still need to be transferred.

By the way, this snippet to draw bars is quite handy. Here is what I use to get visually the space remaining on a drive:

server ~% t=`df -h | grep hdr42 | awk '{print $3/$2*50}'`;perl -e "print \"[\",\"x\"x($t),\".\"x(50-$t),\"]\n\";" 
[xxxxxxxxxxxxxxxxxxxxxx................]