Tuesday, September 28, 2004

NetBSD/mac68k soft-float support

Two days ago I commited some code to NetBSD's CVS HEAD to let the mac68k port be built with soft-float support. All this work was originally done by Bruce O'Neel, so if you want to thank somebody, thank him. In order to build a release with this feature enabled, it's as easy as passing the -V MKSOFTFLOAT=yes argument to the build.sh script.

But what's known as soft-float? Simply put, it's a way to build binaries that do not use the FPU at all. The GCC compiler has an special option to do this, named -msoft-float. When used, it translates all FPU instructions into calls to regular subroutines (provided in the C library), which provide emulation code.

Ok but... why is soft-float useful in the mac68k port? The 60840LC chip has a hardware bug that makes it impossible to implement an FPU emulator in kernel space; this chip is included in many Macintosh boxes, like the Performa 630 I have. (AFAIUI, there is a way to workaround this issue, but it's not implemented in NetBSD...) So, by using a NetBSD release built with soft-float, you can run it on one of those computers! Isn't it cool? ;-)

Sunday, September 26, 2004

Why function names appear in column 1?

Have you ever wondered about why many people writes C/C++ function definitions separating the return type from the function name? And why even style guides suggest you to do so? I'm referring to something like:

int
main(void)
{
....
}

The reason behind this is to simplify searching for functions. If the function name appears in column 1, it is very easy to construct a regular expression that matches the code you are looking for.

For example: suppose that you have to locate the main function; a simple grep ^main *.c will give you a very accurate match (often unique). And in case it gives more results than desired, a grep '^main[ \t]*(' *.c will return the exact match. (When using egrep, you'll need to quote the left bracket.)

Saturday, September 25, 2004

Preformatted manual pages

Some time ago, a friend of mine installed FreeBSD on his old laptop and told me that it was very slow. Specially, a simple man ls took forever, while the same command on a Linux system went faster. Why was this specific command slower? Probably because he missed to install the preformatted manual pages, included in the catman series.

We'll see what these files are in a minute, but first of all you have to understand how man works. Manual pages are stored under /usr/share/man/man[1-8] (among other customizable locations). The files in these directories are the manual pages in their source code form, that is, in groff format (or some other flavour of roff, depending on the system).

Whenever you ask for a manual page, man invokes groff to generate a pretty formatted page: it indents and fills paragraphs, changes some text into boldface, etc. (As an analogy, it is like the conversion from plain HTML code to the rendered page.) When the conversion has finished, more (or the program pointed by the PAGER variable) is invoked to show you the generated page.

This process can be very slow on old systems. So, if the source pages don't change often, why not store a copy of the generated files somewhere in the system? This way, they can be accessed at will, without having to call groff. This is exactly what preformatted manual pages are for.

On a BSD system, if you look at the files stored in /usr/share/man/cat[1-8] (assuming you installed them), you'll see that they are the pretty formatted version of your manual pages. These where generated when the whole system was built, so that the conversion has to happen only once. If they are there, man will prefer them over the source pages to speed up its process.

On a Linux system, things work in a slightly different way. The preformatted pages are stored under /var/cache/man/cat[1-8], and are generated on the first read; therefore, further reads of the same manual page will be very fast. However, this only happens if the man program has the setuid bit set, or if you have enough permissions to write to that directory. Check the man(1) manual page for more information about how this works. (I'm not completely sure that this is the exact process... correct me if I'm wrong, please.)

Friday, September 24, 2004

Bugs and questions

Some time ago I talked about how to write good bug reports. Today, I've found an essay that describes this process in great detail; you can find it here, written by Simon Tatham.

Related to this, I also found an article that explains how to make good questions; i.e., to not annoy the person that may answer you - it focuses on hackers answering - and to provide enough information to make your question complete. You can find it here, written by Eric S. Raymond.

These documents are not short, but they are worth reading, really. Even if you think you do good questions and write good bug reports, you'll surely find something in them to improve your style.

Wednesday, September 22, 2004

VCS Made Easy: sources imported

I have imported the preliminar sources of VCS Made Easy into the repository; you can view them trough ViewCVS by clicking here. To download them, just follow the regular procedure for any project hosted at Sourceforge.net, which is summarized here:

$ mkdir vcsme
$ cd vcsme
$ cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/vcsme
login
$ cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/vcsme
co -P src

The status of this code is close to a possible 0.1 release, although it still lacks some important features and cleanups: for example, it doesn't work very well if you don't have a previous .cvsrc file. I will be more confident to publish it when it works correctly out of the box for a completely clean user acount. But for now, you can play with the code in the repository.

Tuesday, September 21, 2004

GNOME 2.8.0 hits pkgsrc

After more than two weeks of work, I've been able to commit to the pkgsrc tree all the required changes to bring the GNOME Desktop to its 2.8.0 version.

The update has resulted in 73 new entries added to the doc/CHANGES file; of these, 2 are new packages, 70 are package updates and 1 is a removal of a meta package. Quite a bit, eh? ;-)

Unfortunately, not all the official components that form the GNOME Desktop are in pkgsrc yet; these include gnome-nettool, gnome-system-tools, gnome-volume-manager, gnomemeeting, vino, ximian-connector and evolution-webcal. This is because I have had not enough time to work on them, and I expect they'll carry multiple portability issues. If you want to help, there's always pkgsrc-wip where you can add any of these missing packages!

Now, to install this new and shining GNOME release, just get into pkgsrc/meta-pkgsrc/gnome, type make install and you're done! Oh, and in case you don't know, this is not restricted to NetBSD; you can use pkgsrc even under Linux!

Sunday, September 19, 2004

Writing portable code

Portability problems can be seen from two points of view: the operating system and the architecture. Depending on the kind of application you are developing, you may hit these problems. An example of OS-portability can be the use of a specific hardware subsystem through kernel facilities; on the other hand, an example of architecture-portability can be the direct use of assembly code.

Often, you will be aware that a chunk of your code is not portable; for example, when accessing Linux's ACPI subsystem, you can be sure that that part of the code won't work outside Linux.

Some people will do an extra research effort and see how other systems do what they need, and will add the right code to make their program portable. Unfortunately, many people opts to keep the code as is within the program.; that is, it gets compiled unconditionally, thus causing build failures on any other OS. And a wrong desing at this point will make your code unreadable when adding portability workarounds.

In this situation, you can mitigate the problem by isolating the unportable code, even if you won't be implementing support for other platforms. This way, anyone who builds the package will quickly notice why it fails, and he may be able to provide a better patch with less hassle. How to do it is up to you: either create a source file for each platform you want to support, use preprocessor conditionals or use inheritance in an OOP language. Whichever case you choose, keep scalability in mind: adding support for another OS/architecture should be easy (I'm referring to the code structure, not to how easy is to port the feature).

Resuming the previous ACPI example, you could organize your code by using a file per OS, like in: acpi-linux.c, acpi-netbsd.c, etc. Or you could do something like:

/* Prototypes for portability functions defined below. */
static void _acpi_init(void);

/* Public functions that are called from other .c files. */
void
acpi_init(void) {
...
_acpi_init();
...
}

#ifdef __linux__
/* Inclusion of Linux-specific headers */
/* Private functions to use Linux's ACPI */
static void
_acpi_init(void) { ... }
#elifdef __NetBSD__
/* Inclusion of NetBSD-specific headers */
/* Private functions to use NetBSD's ACPI */
static void
_acpi_init(void) { ... }
#else
# error "Sorry, this code has not been ported yet."
#endif

As you can see, the "generic" code is kept simple: it calls the system specific function and does any other common stuff; no conditionals are inserted there to handle OS details. Moreover pay special attention to the #error directive: it will tell the preson building your software why it failed, and won't lead him to confusion.

These comments come from lots of portability issues I've hit while making some programs work under NetBSD; fortunately, I'm not using anything else than i386. Hope you find these suggestions helpful and apply them to your code; they'd make the task of packagers a bit easier. Ah, and remember: the world is not Linux/i386 only!

Wednesday, September 15, 2004

GNOME 2.8 published

According to the release schedule, the GNOME Project is pleased to announce the 2.8 version of its GNOME Desktop. You can find the official annoucement here, as well as the discussion in FootNotes.

I've been using the first release candidate (2.7.92) for around two weeks, and all I can tell is that it's marvelous (well, almost) ;-) Lots of bugs have been fixed, some of them regarding to portability to NetBSD (which I filed). It also has several new features and applications, like Evolution 2.0, Vino or the GNOME System Tools, just to name a few.

What I like specially is that Metacity starts to be usable. Believe me, it sucked in GNOME 2.6, specially its focus management. It got lost over and over again, which was very annoying when using the keyboard to switch between desktops and windows. There are still some issues remaining (compared to other window managers), but they don't get in the way in daily use (and I'm not sure if it's Metacity's fault, or if it's really doing what's expected).

With respect to pkgsrc, I already have 2.7.92 in my local tree. Updating it to 2.8 should be painless, given that it will be very similar, so I should have the update ready by the end of the current freeze (which is to prepare the 2004Q3 branch, in case you don't know). I hope to commit the big update just after the freeze is over, so keep tuned! If you are curious, here are some shots I took a week ago; they are about the first release candidate running under NetBSD 2.0G.

Tuesday, September 14, 2004

VCS Made Easy - Project registered

Yay! Sourceforge.net has accepted my new project, called VCS Made Easy. This program aims to simplify the management of several directory hierarchies controlled by a version control system (such as CVS or Subversion) up to date.

The program reads a configuration file, written in XML, which describes the set of directories to be managed together with some properties associated to them. It then later processes the action requested by the user, which at the moment can only update such trees. For example, a simple vcsme update call from the command line could result in all trees being updated with the right permissions and ownerships for each of them.

It also aims to be safe enough to be run through sudo(1) or with the setuid bit set, so that regular users can update (or manage in other ways) system-wide trees. Furthermore, it is able to generate nice reports, to be mailed or, in a future, be uploaded to a web server. This is specially useful to run the program from a cron job without any trouble.

The code is being written in C++, with a design that makes it easy to add support for other VCS systems and other ways to generate reports. Aside from the libxml2 library, the program has no other build time dependencies.

In a maybe not-so-distant future, I'd like to create a GNOME frontend for it, so that users can easily keep control of their source trees - in fact, this was the initial idea of the project.

Note that the project has just been created. ATM, the repository is empty, and no releases have been done. However, I have the source tree in my disk, which I hope to upload to the repository soon (specially, to keep it under revision control).

Monday, September 13, 2004

One time passwords

When you are away from home, you may need to access your machine through an SSH (or telnet, if you are still using it) client. The client machine will often be public and not administered by you, so you can't trust it. Who warrants you that it does not contain any key sniffer that can capture your password?

In this situation, you can still log in by using a one time password (OTP). These passwords let you log in your machine only once, so even if somebody else is able to sniff it, it won't be of any use to them. The disadvantage is that you have to carry a list of valid OTP passwords and be sure that nobody can see them until they have been used.

There are many OTP systems; the one I'm using is skey, which comes with the default NetBSD installation - on other systems, check if it is included or install it from the packages system. Furthermore, it is well integrated with SSH, so its setup is painless.

Start by enabling OTP passwords for you account, issuing skeyinit from the command line. It will ask you your account's password and a passphrase: the first one is used to authenticate you and the second one is used to generate the OTP sequence. Once introduced, the program will let you know the actual sequence number and an identifier for your keys. Be sure to store take note of the identifier, as you will need it to generate further keys (if you are unsure, just store the entire output of the program). Furthermore, it will also let you know your first OTP.

But one OTP is not enough, specially when you are away (since you can't generate more from there). So, run the skey program, which takes two arguments: the sequence number and the key identifier you stored above. Additionally, pass it the -n 10 parameter to generate 10 OTPs (replace the number with the amount you want). Then, take note of all listed passwords in a paper and store them in your wallet.

Now you can safely log into your system 10 times. If you need more passwords, you can generate them with another call to skey, replacing the sequence number with the next one.

Edit (18:37): In fact, you don't need to remember the identifier printed by skeyinit; simply running skeyinfo at any time will tell you your next OTP sequence and identifier. Sorry for the confusion, but I was posting from a computer that didn't have these utilities and I couldn't verify it.

Sunday, September 12, 2004

The old new thing

I would like to recommend you a blog that I find very interesting: it's called The old new thing, and is published by Raymond Chen. You know, I'm not a Windows fan... and this blog's main subject is Windows... so why am I recommending it?

The articles in this blog explain curiosities about why Windows behaves the way it does in some scenarios, how some features are implemented, what is considered when making some decisions, why some things that look strange (in the API, for example) are done that way, etc. Moreover, it also shows some programming tips and also includes some non-Windows related articles. Enjoy it!

Added RSS feed

As I already said in previous posts, LiveJournal provides a quite good service. However, free accounts do not have the RSS service, which is a pity. Having a sindication method is a very important thing to make a blog popular among people.

So... I have implemented it externally: a small Perl script fetches the main page, parses its contents and extracts the recent entries. When it has gathered all the information, it generates the RSS file and stores it in a friend's server - nopcode.org, owned by Brainstorm -, who is providing me space to store it (thanks very much!). This script is run from cron(8) at specific times during the day to keep the file up-to-date (although, unfortunately, not in real time).

You can find a link to the RSS feed in the Links section of this page (see the left bar). Hope it doesn't break soon! :)

Edit (13th September 2004): an anonymous reader has told me that Livejournal provides an RSS feed for everybody, including free accounts. So, forget everything about the custom script and just use the link in the left side of this page to get the feed. It now points to the RSS file generated by Livejournal, which should be updated in real time and is more accurate than the one generated by my script (i.e., it won't break if I change the style). Moreover, I've removed the link pointing to my custom RSS from the post, so that you don't use it by mistake.

Saturday, September 11, 2004

Why pkgsrc uses static file lists

In the pkgsrc package system, each package comes with a PLIST file which describes the files and directories that belong to it. Its contents are used at deinstallation time to cleanly remove the package from the system, among other tasks.

Now you may be scared because this looks like a real maintenance nightmare (it used to be, though). Almost all other packaging systems do not have such static lists; instead, they are automatically generated at installation time (only when installing from the sources) and stored inside the binary package.

Even though, there are good reasons to keep static lists:

  • You can search for a file in all available packages without having to install them. This is useful, for example, when you need a header file and you don't know where it is; a simple cd /usr/pkgsrc; grep file_name */*/PLIST* will do the trick. With dynamic lists, this can be implemented by having a machine that builds all available packages and builds a database with all available files.
  • If the package installs more (or less) files than expected, pkgsrc will emit an error to let you know that something bad happened. (Well, this is not completely true until we have staged installs... which I'm interested in implementing.) It is difficult (if not impossible) to implement this if you only have dynamic lists.

These are the main two reasons, although I recall that there are some other minor ones. And, as we have to maintain these lists, pkgsrc includes several tricks to make the process easier: for example, the print-PLIST target produces an almost-exact list - which can be made exact if you use the PRINT_PLIST_AWK extensions.

Friday, September 10, 2004

About the Finder...

Today I've been reading a very interesting article which talks about the advantages of spatial interfaces. It is titled About the Finder... and can be found here. The paper starts explaining what spatial interfaces are, how they mimic reality and why they are good in usability terms.

To provide implementation examples, it focuses on Mac OS's Finder for the good ideas and Mac OS X's Finder for the bad ideas because, in the oppinion of the author, the later is worse in terms of spatial behaviour. However, I find that many of the concepts explained can be applied to Nautilus (the GNOME file manager), and as it is free software, any features not yet implemented could be added by anyone.

If you like spatial interfaces, this article will only make some concepts clearer to you. If you don't - or you are unsure about their purpose -, it may change your mind. Personally, I like the spatial interface quite a bit, and it is not so bad - as you may actually think - when you get used to it.

autogen.sh scripts

The GNU Build System works by generating scripts that have to be later distributed in your distribution file; these include Makefile.ins and configure, among others. Many projects that use it are managed by a version control system (such as CVS), although they don't keep the generated files under revision control (which is a good thing in many scenarios).

When you check out a copy of these sources from the repository, you don't get any of the build scripts you need to build the project on your machine; instead, you have to deal with the task of previously creating them. However, many projects provide a shell script aimed to simplify this task, usually named autogen.sh. This script runs automake, autoconf, aclocal and other utilities in the right order to create the required files. But this is a very bad idea. Consider that aclocal will be replaced with something else in the future, thus breaking your scripts.

The solution is named autoreconf, which comes with GNU Autoconf. This program deals with the task of running autoheader, aclocal, automake, autopoint (known as gettextize too) and libtoolize where appropiate to remake the GNU Build System files. Furthermore, it only remakes those files that are older than their predecessors, or those that don't exist. Just try it: run autoreconf -is on a project and see how the files are automagically created.

But... what if you need to run other tools like intltoolize? Then, in this case, it is still good to provide an autogen.sh script. So all it would do is run intltoolize and later run autoreconf.

Just remember to not run any of the tools automatically executed by autoreconf in your autogen.sh script. Anything else is appropiate.

Thursday, September 09, 2004

Migrating to new versions of the GNU Build System

The GNU Build System is basically composed of GNU Autoconf and GNU Automake. The latest versions of these tools are 2.59 and 1.9.1 respectively, at the moment of this writing. Compared to 2.13 and 1.4, these are far better, although not completely compatible with the previous ones. However, if you are maintaining a software project which uses these two tools, you should consider updating to the latest versions, as your program will be more portable and easier to manage.

Let's discuss some of the changes you'll need to do in the configure.ac or configure.in file (note that the first name is preferred):

  • The AC_INIT macro now takes four arguments (the last two are optional). The first one is the full name of the package in its full form; for example XML Catalog Manager; the second one is its version; the third one an email address where bugs should be reported; and the last one is the package name in its short form, like xmlcatmgr. Check out the GNU Autoconf manual for more details, because the last parameter can be automatically derived from the first if you follow some rules.
  • Be sure to set AC_PREREQ to 2.59, so that GNU Autoconf knows if it should parse the file or not.
  • Since AC_INIT does not take a file name to verify if the script is being run from the appropiate directory, you have to use the AC_CONFIG_SRCDIR macro to achieve the same thing.
  • Optionally set AC_COPYRIGHT to a copyright string that should appear in the generated configure script, to take credit for your work.
  • The AM_INIT_AUTOMAKE macro now takes a whitespace separated list of options to pass to GNU Automake. My suggestion is that you add the following: 1.9, which specifies the minimum version of the utility needed; -Wall, which turns on multiple warning messages; check-news, which verifies that your NEWS file describes the changes for the latest version; and no-define, which avoids defining the PACKAGE and VERSION macros in the configuration header (these two are deprecated anyway).
  • When formatting help strings, required by the AC_ARG_ENABLE and AC_ARG_WITH macros, use the AS_HELP_STRING macro. This ensures that the text appears properly aligned when calling ./configure --help.

The Makefile.am files will also need some tweaks:

  • Remove AUTOMAKE_OPTIONS assignments, since we have specified them in the AM_INIT_AUTOMAKE macro.
  • Rename the INCLUDES variable to AM_CPPFLAGS.

At last, since we are passing the no-define option to GNU Automake, you'll have to do some minor changes in your source files:

  • Where you need to specify the full package name, use PACKAGE_NAME.
  • Where you need to specify the package version, use PACKAGE_VERSION.
  • Where you need to specify the full package name, together with its version, use PACKAGE_STRING.
  • Where you need to specify the short package name (usually, when calling functions like bindtextdomain), use PACKAGE_TARNAME.
  • As a rule of thumb, replacing PACKAGE with PACKAGE_TARNAME and VERSION with PACKAGE_VERSION should produce the same results as before. However, the new macros allow more flexibility, so you can replace some hardcoded strings in your sources.

Well... and that's all you need to get started. You may want to do other changes while you are at it, but you should now have a good idea of what needs to be done. Let's code!

Tuesday, September 07, 2004

Strange USB mouse

I've been trying to setup a new laptop today (unfortunately, not for me); one of the things I had to do was to install an external USB mouse (by Packard Bell). Easy, you'd say. But it has turned to be impossible.

First attempt: plug the mouse in the laptop. Windows XP sees it and tries to set it up. The result is a box saying that the mouse does not work. Ouch, it must be that I haven't installed the drivers from the floppy disk (hmm, drivers? for a HID mouse?). So I install them and try again. Same result.

Eww. Maybe the mouse is broken? I thought. So the next step was to try it on a different computer; I tried on my father's one, which only has Windows XP. Same result, even unplugging the PS/2 mouse it had attached.

Last attempt: plug the mouse in my computer under Fedora Core 2. Voila! Mouse working perfectly. To ensure it was a USB HID mouse, I checked from 'dmesg', which confirmed by thoughts.

Yet another reason not to use Windows ;-) (or to not buy mice made by the brand I mentioned...) If anybody has an idea of what's going on, please share it. If not, the mouse will go back to the shop soon (though I don't care much).

Monday, September 06, 2004

To rely or not to rely on the PATH

It is quite common for a program to need to execute other programs at run time. This can be done in two ways: specifying the full path to the binary or relying on the current path. So which approach is correct? It depends on what you are trying to do.

If the program you need to execute is not a required dependency of your program, using the path to locate it is correct; however, let the user set up a full path in the configuration file, if applicable, so that he can forget about the path. For example, suppose your program can provide extra functionality if the zip command is installed. As it is optional, you can't know its location at configuration time, so it is safe to use the path.

But if the program you have to execute is a required dependency of your program, you should always use a full path to it. How to do it is simple: just add a check in your configure script to locate the binary and use the result to configure your sources. But why you should do this? If the external program is a dependency, your program should always use the same binary - which is detected at configuration time; the only way to ensure this is with full paths. For example, imagine that you have two versions of gettext installed in your system - one that comes with the base OS and one installed from the packages collection. Now suppose that a given program only works with the version in the packages collection; you must ensure that it always runs this version, no matter what the path value is, so that it does not misteriously break for some users.

ATM, I'm "fixing" intltool to use full paths to iconv(1) and all gettext utilities, precisely because these are required dependencies. I'm not sure if the upstream developers will accept this patch, but I'll try ;-) Edit (00:42): the issue is being tracked at bug #152020.

Sunday, September 05, 2004

Back at home

Hi readers! I'm back from my vacations. Well... in fact I came back a week ago but haven't had a chance to write something for the blog.

So, what are my current projects? First of all, I've been updating all the GNOME packages in pkgsrc to the latest published development version, 2.7.92, in preparation for the shortcoming 2.8. pkgsrc enters a code freeze tomorrow, in preparation for the new 2004Q3 stable branch; this will last for two weeks, during which I'll be finishing the update and getting ready to do the mega-commit when the freeze is over.

I've been also working on a new program that simplifies the process of updating file hierarchies managed by a version control system, such as CVS or Subversion. With a very simple command, you will be able to update all predefined directories transparently, and generate pretty reports to be mailed to you or put on a webserver. It's designed to be easily run from cron(8) and to be run from sudo(1), so that you can let your users update system-wide trees but without letting them direct access. The code still needs several improvements, but it might well be released as a 0.1 version.

At last, I have to finish two small articles and publish them somewhere; one is about the GNU Build System and the other about making software package-friendly.

Will keep you informed about any progress ;-)