Friday, January 14, 2005

Automatic verification of PLISTs

pkgsrc uses static file lists to register the contents of an installed package. A drawback of this approach is that the file list of a package may occasionally become out of sync with reality. I.e., it may omit some files that are really installed, thus leaving them orphaned, or specify extra files that are not really copied into place.

There are multiple reasons that can cause this desynchronization, including careless updates, incorrect detection of dependencies or, even worse, files that are only installed depending on the OS you are running. When some files are not properly installed, pkgsrc can easily tell you that an error happened, because it will not find all the files listed in the static list. However, the opposite situation is hard to detect, and it was not possible up until now.

Yesterday, I looked at the stale files left after a bulk build under Linux and I was quite disappointed because the list was not anything close to short. So I decided to add a mechanism to capture these problems as soon as possible. You can benefit from these changes with recent pkgsrc's HEAD as of today's evening.

The new feature, called check-files is only run when PKG_DEVELOPER is defined and CHECK_FILES is set to YES. The later is a default (though it might be changed if it continues to show flaws). This will ensure that any package does not install more files than expected.

There is also some extra functionality that is only enabled when CHECK_FILES_STRICT is set to YES (not a default). In this case, check-files will ensure that packages do not touch PKG_SYSCONFDIR nor VARBASE directly. This behavior is completely incorrect from the point of view of binary packages, as they have to use bsd.pkg.install.mk for correct operation. However, these checks cannot be turned on by default (yet) because they will break many packages. Even though, developers should turn on this strict behavior and fix their packages whenever possible.

2 comments:

Anonymous said...

[Originally posted at 2005-01-17 03:57 pm UTC]

Hi,
have you looked at darwinports? They are using autoconf's DESTDIR to install packages in a clean destroot, then register the new files in the destroot (build a 'PLIST') and move them into the system root (/opt/local). No need for a PLIST file.

Julio M. Merino Vidal said...

[Originally posted at 2005-01-17 04:06 pm UTC]

Yes, I once tried to implement that, and worked quite well. However, it's not easy to "destdir-ify" all packages, as many of them do not support this feature (or work in another way).

Furthermore, although PLIST may seem a problem, they are quite useful. They let us catch mistakes and also search for a given file in all available packages. So there are multiple developers against removing them.

(FWIW, pkgviews generates PLISTs on the fly as it installs each program in its own separate directory.)