pkgsrc: Package-related terminology

For a long time, I've been noticing a recurrent terminology mistake in some pkgsrc-related mailing lists and bug trackers. The error is that some people use the "pkgsrc" word to refer to single packages inside the pkgsrc tree; e.g., they send a problem report with a summary line saying: "New pkgsrc for foo-1.0". This is incorrect and should read: "New package for foo-1.0", but we will discuss this below. (A confession: I did this mistake when submitting my first packages, too.)

So, which is the root of this confusion? I can't say for sure, but I can explain what happened to me. In the Linux world, almost all distributions use binary packages (those rpms or debs, to name a few), and users deal with them on a daily basis. However, all the documentation, as well as programs, use the term package to refer to these, as in "Proceed to install the bar-1.0.deb package". This is obviously not wrong, but is the terminology they chose and/or prefer to use.

Then, in the FreeBSD world, there is the ports tree, which is a collection of source packages, each of which is called a "port" (notice the distinction between plural and singular). OTOH, binary packages are called "packages", just as in Linux.

At last, when these people come to NetBSD, they find that: packages in the pkgsrc tree are not in binary form — thus the "package" term (the one from Linux) not being really appropriate —, and as pkgsrc is a singular name, you can't shorten it to get a correct name for a source package. Thus, they end up using the "pkgsrc" term to mention a single package.

Let's summarize this in a list of terms:

• pkgsrc: The whole tree of packages and the infrastructure files, which usually lives in /usr/pkgsrc. There is no other thing known as pkgsrc.
• Binary package: A .tgz file that contains a set of binary files ready to be installed using the pkg_add utility. We prefer to use these two terms to refer to this kind of files, as they result in an unambiguous concept.
• Source package: A collection of files inside a specific subdirectory of the pkgsrc tree that holds a package. E.g., /usr/pkgsrc/meta-pkgs/gnome. Note that this term isn't much used; in fact, I don't remember seeing it anywhere, but I personally use it when I want to make the distinction clear.
• Package: Usually refers to a source package, although, depending on the context, it can also be a binary one.

The TERM variable

After reading some messages posted during the last few days in a NetBSD mailing list, I realized that several people do not know what is the real purpose of the TERM environment variable.

The misinterpretation is the following: they think that TERM is used to specify the terminal type that should be emulated on your terminal. Thus, e.g., if you explicitly set TERM=linux in a NetBSD console, everything should work as in a Linux system. This is far from reality, as things work quite the other way around.

TERM is used to tell the termcap (or terminfo) library which terminal type you are really using. This way, the library, which acts as an abstraction layer for terminal control codes, can generate the correct control sequences to manage the screen. For example, if you remotely log in into any Unix server from a Linux box, TERM should be set to linux (which usually happens automatically) so that the remote applications can be displayed correctly.

Anyway, I'm not going to describe the details in this little post. If you want to learn more, read the TTYs and X Window: Unix Now and Then article by Hubert Feyrer.

pkg-config: Specifying dependencies

One of the good things about using pkg-config in a software package is that its configuration script can easily specify external dependencies by their name and their version. It doesn't need to mess with other requirements, such as specific build flags or tricks to determine whether the installed library is new enough to fulfill its needs.

In other words: a program can focus on detecting its direct dependencies rather than having to look for other underlying libraries it doesn't care about. And most important: the user can easily see which these are.

Consider the following example: your program needs, at least, GTK+ 2.6.4, but it doesn't want to use any specific Pango functionality. In this case, the program depends on a specific version of GTK+ (2.6.4), but Pango's version is irrelevant. In fact, as the program doesn't deal with Pango at all, Pango could be replaced by a similar component in a future version of GTK+ and your program could remain functional. (Hey, this is just an example! I'm by no means suggesting that Pango should be replaced ;-)

In the previous example, our imaginary software package could do something like:

PKG_CHECK_MODULES(gtk2+ >= 2.6.0)

The occasional reader can later see which are the required and direct dependencies of the program by looking at the output of the configure script (or directly at the source script and searching for calls to this macro).

However, if our previous program required to use Pango, say version 1.8.0, it could do the following:

PKG_CHECK_MODULES(pango >= 1.8.0 gtk2+ >= 2.6.0)

Which clearly states that both Pango and GTK+ are required.

Why is this useful? It makes things extremely easy to the package maintainer who has to analyze the dependencies of a given software program and define the correct dependencies for his package.

Books: The Da Vinci Code

I've just finished reading The Da Vinci Code, which I started this past Tuesday. It didn't take my attention at first, as it seemed just like many other mystery stories, but after around 50 pages, I was hooked. I can't remember any other book which has captivated my interest so much.

As you may already know, this is a novel, so the story in it, as well as the characters, are all fiction. The story itself is well explained and full of surprises. But what makes the book interesting, IMHO, is that it's based on true facts: the scenarios, the pictures, the places, the people... most of them are real.

Of course, you shouldn't believe everything explained; it will require research on your side to decide what's real and what's not (and some things only depend on your faith). It's hard to split where the truth finishes and the fiction starts, which is probably a good aspect of the book, as it will raise your interest in some things you may have never thought about.

All I can say is that it has been an amazing read! I truly recommend it. For more information, just go to the official website linked above; I hope there are no spoilers, but the book will be more exciting if you know nothing beforehand ;-)

GNOME 2.10.0 hits pkgsrc

After more than a week of work, I've finally updated the GNOME packages in pkgsrc to the latest and newest stable version; that is, 2.10.0.

You can read more information in the announcement e-mail I sent to the tech-pkg list.

XTerm: Anti-aliasing

One of the things I kept missing in XTerm, compared to Konsole or gnome-terminal, was anti-aliased fonts. But you know what? They are, in fact, supported, as XTerm now — well, I suppose since XFree86 4.0 at least — uses freetype to render text.

So, how to do it? Just add the following lines to your ~/.Xresources file, as we did yesterday with the scrolling stuff:

XTerm*faceName: Bitstream Vera Sans Mono
XTerm*faceSize: 10

You can, of course, choose a different font name or size, but I think these look pretty good (although a bit big, compared to the default fixed font). And since it's using Freetype, you can use any font you like in your terminal! (Although some of them may look really bad.)

As usual, these are also available as command line options; just take a look at the manual page. You may discover other interesting stuff as well.

XTerm: Setting up scrolling

Back to the days when I used KDE (that was more than a year ago), I got used to the Shift+Up and Shift+Down keybindings in Konsole (I have to say it's a great terminal emulator). These keys combinations scroll up and down, respectively, a single line of text.

When I later switched to GNOME, I was quite disappointed to see that these were not supported by gnome-terminal. In fact, this may be false, but I didn't spend too much time looking for it because gnome-terminal is way too slow.

So I switched back to XTerm, the genuine terminal emulator which, at first, seemed to not support these key combinations either. As I was aware of XTerm being highly configurable, I spent some time reading the manual until I figured out how to set scrolling up.

What I found is that XTerm can map any key combination you want to one of its integrated functions, so all I had to do was to map Shift+Up to scroll-back(1) and Shift+Down to scroll-forw(1). In other words, I had to add the following to the ~/.Xresources file:

*VT100.Translations: #override Shift <Key>Up: scroll-back(1) \n                    Shift <Key>Down: scroll-forw(1)

While we are at scrolling, here is another little trick which will increase the amount of lines stored in the terminal backlog:

XTerm*saveLines: 512

Reload your X settings (xrdb -m ~/.Xresources) and you are done!

pkg-config: Mixed state in some libraries

A reply to my previous pkg-config introductory post outlined a real "problem" with "mixed-state" packages. These packages provide pkg-config metadata files in some situations; i.e., not always. This is a quite common situation in libraries that did not use pkg-config in the past, but have been recently converted to do so. Some examples are OpenSSL or the X libraries (which are being converted to the GNU toolchain by Freedesktop.org).

But why is this a problem? Basically because software developers are not aware of this and simply look for the pkg-config metadata files. E.g., when they need OpenSSL, they simply ask for the openssl.pc file through pkg-config; in case of failure, they assume the library is not there, but this assumption can be perfectly wrong. When this happens, those programs should also check for the libraries in a direct manner, bypassing pkg-config.

However, it's safe to use pkg-config in other situations, specially to get information about libraries which have always used pkg-config (like almost all GNOME 2.x libraries).

So, be careful with libraries that have been around for a long while. In case of doubt, try to do a bit of research: check older (API compatible) versions of the same library to see if they have the file, and also check some other operating systems (specially if the library is part of their base system). If you are still unsure, it may be a good idea to ask the software developers or simply add manual detection in your scripts.

TV series: Dark Angel

I've just finished watching the second (and last) season of Dark Angel, a TV series I discovered around past summer. All I can say is that the whole series are great, but IMVHO, too short. In fact, that's because FOX canceled them just after two seasons; what a pity.

These series tell you a science fiction story, set in the not so distant future. The plot is based around a genetically engineered girl, known as Max. She and several other kids escaped from a government facility where they were being trained to be perfect soldiers. After the ran away, she lost track of all her "brothers", and she wants to find them again (as they are the only "family" she has). This is part of the first season's plot, but this develops into something more complex.

I said kids before, but the story starts several years later after they escaped (maybe 15 or 20). Seattle, the city where she lives, is screwed; everything is heavily controlled by the police (some people say it's like the 1984 book, which I haven't read (yet)). Fortunately, she soon meets a man, Logan, who will help her to achieve her "dream". Don't be afraid; I'm not spoiling anything here: all this happens at the beginning of the first episode.

I think the episodes are well done, have a good deal of action and keep you wanting to know what will happen next (all of them are quite related and should be seen sequentially). So, if you like watching TV series, I highly recommend this one to you. The weird thing is that you'll be left wanting more when the second season finishes...

Edit (March 15th): I've changed the post's title.

pkg-config: A quick introduction

If you have ever tried to check for the presence of a library from a configuration script, you know this is not an easy task. Getting the right compiler (CFLAGS) and linker (LIBS) flags can be very difficult, if not impossible, without manual help from the user.

To detect some libraries, you have to create a little test program and link it against the library you are looking for. If the build is successful, the library is available, and possibly ready to be used. In fact, this is what GNU Autoconf's AC_CHECK_LIB macro does. However, this is not necessarily true, as the detected library may be older than you expected, or may need some extra flags that you didn't think of.

To overcome this difficulty, the *-config scripts were born. These scripts, specific to a library, print the required compiler and linker flags to the standard output, so that the configuration script can get them. For example, consider the glib 1.x library. It includes the glib-config script, which produces the following output:

[dawn jmmv] $ glib-config --cflags
-I/usr/pkg/include/glib/glib-1.2 -I/usr/pkg/lib/glib/include
[dawn jmmv] $ glib-config --libs
-L/usr/pkg/lib -Wl,-R/usr/pkg/lib -lglib

This is certainly nicer than manual detection of libraries, but has a problem: each library has to provide its own script, which may be syntactically incompatible with all other ones (e.g., accepting an -ldflags flag rather than --libs).

And here is where pkg-config comes into play: it generalizes the *-config scripts concept and provides a consistent framework to access library information.

In the pkg-config world, each library installs a special metadata file in a centralized directory (typically, lib/pkgconfig relative to the installation prefix). These files specify the version of the installed library, the compiler and linker flags needed to link to it as well as some other extra information. Then, the pkg-config utility can be used to consistently query this information.

The above glib example becomes the following:

[dawn jmmv] $ pkg-config --cflags glib
-I/usr/pkg/include/glib/glib-1.2 -I/usr/pkg/lib/glib/include
[dawn jmmv] $ pkg-config --libs glib
-Wl,-R/usr/pkg/lib -L/usr/pkg/lib -lglib

We can also ensure that a given version is present:

[dawn jmmv] $ pkg-config --print-errors glib-2.0 ">=" 2.6.0
[dawn jmmv] $ pkg-config --print-errors glib-2.0 ">=" 2.10.0
Requested 'glib-2.0 >= 2.10.0' but version of GLib is 2.6.2

Furthermore, this integrates nicely with configuration scripts generated by GNU Autoconf, as pkg-config provides a macro (PKG_CHECK_MODULES) to trivially query package information.

Unfortunately, not all libraries provide pkg-config metadata files (yet?), so you still need to fallback to the previous mechanisms if available. Furthermore, some people don't like pkg-config, but I can't see why. To me, it's a very neat concept (I'm not talking about its internal implementation, which may be messy) that greatly simplifies things.

GNOME 2.10.0 released

The GNOME Project has just released the 2.10.0 version of its Desktop Environment. This is a new major release of the 2.x branch, which is source and binary compatible with all previous versions in the 2.x series.

Aside lots of fixes, improved translations and code cleanups, which are typical in minor releases, this one includes several new features and utilities designed to make your desktop experience better than ever. You can see a summary of changes here, and in case you are impatient to see it working, just download and try the live cd.

Now, the big question: when will this be ready for pkgsrc? Well, I've started working on the update a moment ago, and after a quick look, there are around 70 packages to go (several of which should be straightforward updates); as you can expect, this will take a while.

As we are on a feature freeze in order to prepare the next pkgsrc stable branch, 2005Q1, I'll have more than a week of time to work on it calmly (don't think that that's a long time). Unfortunately, this means that I won't be able to help much to stabilize the current code-base. Shame on me! I don't like to say this, but... pkgsrc freezes always come in bad timing (final exams or new GNOME releases), so I can't usually help :-(

How to get the window size?

Today, a friend of mine raised the issue of how to get the terminal window size from within a program. He had observed in his system an environment variable, called COLUMNS, which was automatically set to the window's width; that variable even changed when resizing the window.

But then we checked on two more systems (SuSE Linux and NetBSD) and we couldn't find the variable anywhere. I don't know where it's coming from in his system, but this is clearly not the way to go. So, how to do it?

The tty(4) driver has an ioctl for this specific purpose, i.e., to get the window size of an specific terminal. It's called TIOCGWINSZ and has to be called over an open descriptor. (I let to the reader the task to get information about TIOCSWINSZ.)

The next question is, which descriptor? It depends, but, usually, you'll want to get it from the descriptor on which you are going to write the output (STDOUT_FILENO). Let's see a bit of code (slightly based on NetBSD's ps(1)):

struct winsize ws;
unsigned short height, width;  
if ((ioctl(STDOUT_FILENO, TIOCGWINSZ, &ws) == -1 &&
    ioctl(STDERR_FILENO, TIOCGWINSZ, &ws) == -1 &&
    ioctl(STDIN_FILENO,  TIOCGWINSZ, &ws) == -1) ||
    ws.ws_col == 0) {
   height = 25;
   width = 80;
} else {
   height = ws.ws_row;
   width = ws.ws_col;
}

Now that we can get the terminal size on purpose, the next question is how to detect if the window size changes due to an external event - e.g., the user resized his terminal window. The only thing you need to do in this case is to catch the SIGWINCH signal (see signal(7)) and act accordingly; i.e., run the code shown above from inside the signal handler (or any other method you prefer to use, like delayed evaluation).

I guess this method is not extremely portable, but I'm sure it'll work on a lot more systems than the COLUMNS trick.