Major OpenPKG Bootstrap Upgrade: from RPM 4.0.2 (status quo as of March 2001) to RPM 4.2.1 (status quo as of June 2003) This was a long awaited major upgrade, especially necessary on our road to OpenPKG 2.0. So we had to wait until OpenPKG 1.3 was released. Now that this happended recently, we can fully break ;-) OpenPKG-CURRENT with this upgrade. All these changes were previously developed on the OPENPKG_UPGRADE_RPM42 branch and tested in-depth as much as possible. Although all of our tests showed no significant differences while building OpenPKG packages with this, be aware that this potentially can break mostly everything in OpenPKG-CURRENT. If this is the case, we'll fix this soon, of course. This upgrade was made possible through the efforts of the following people: o Conrad Steenberg <conrad@hep.caltech.edu> (intial RPM 4.2.1 patch set) o Michael van Elst <mlelstv@dev.de.cw.net> (patch consolidation, .src.sh bootstrap integration, fixing of --justdb operation, porting remote glob(3) operations, etc) o Ralf S. Engelschall <rse@engelschall.com> (configuration adjustments, patch cleanups and regeneration, code cleanups and bugfixing, new platform id support, etc) ---------------------------------------------------------------------------- This is a list of major changes between RPM 4.0.2 and RPM 4.2.1. See the detailed CHANGES entries below for more details. o Query Pattern Matching RPM 4.2 now alows a glob(3) pattern matching on queries. ("rpm -qa kde-*") o Platform Identification: We finally cleaned up the whole platform identification in RPM. The new approach now is: <prefix>/lib/openpkg/platform is a program which dynamically determines the "<arch>-<vendor>-<os>" id string for the underlying platform. Wxample outputs are: ix86-pc-freebsd4.8, ix86-pc-redhat9, sparc64-sun-solaris9, etc. Then there is <prefix>/etc/openpkg/platform which is the once determined platform id, loaded by RPM to internally provide the macros %_host_{cpu,vendor,os} and %_host. Additionally, we adjusted all other platform id related macros (%_target, %_arch, %_os, etc) to directly depend on these macros. This way we have a new single point of source for the official platform id. o Bugfixes: The upgrade from RPM 4.0.2 (status quo as of March 2001) to RPM 4.2.1 (status quo as of June 2003) is a major code change covering a two year vendor development cycle. This means hundrets of bugs in RPM were fixed and the code was partly refactored and cleaned up. o Package Signatures: RPM 4.2 now uses the BeeCrypt cryptography library for verification of signatures. Previously one needed an externally available GnuPG. GnuPG now is only needed for creating signatures. Additionally, RPM 4.2 manages OpenPGP publuc keys in its RPM database similar to real packages (they even can be seen as pseudo-packages "gpg-pubkey-XXXXX" under query operations). The OpenPKG bootstrap (and also the regular upgrade procedure) now especially imports the OpenPKG OpenPGP public key into this database to make sure that one out-of-the-box can verify signed packages. Finally, "rpm -qi" now especially displays signature information, too. o RPM Database, Berkeley-DB 4.1.24 RPM 4.2 now internally uses Berkeley-DB 4.1.24 instead of version 3.2.4 which RPM 4.0 used. The old DB version 7 database files can still be read without problems and on "openpkg" package upgrade, the database is rebuilded and upgraded to DB version 8 files automatically. Additionally, RPM 4.2 now managed more information than RPM 4.0 and hence uses a few additional database files. o Re-Packaging RPM 4.2 now provides a nifty --repackage feature which can be applied especially to the --erase operation. Before erasing the installed package, RPM then first creates a corresponding binary RPM from scratch. This later can be used to rollback the erase operation. It just has to be remembered that the binary RPM resulting from a "repackage" operation is both not identical to the original binary RPM (because not all information was installed) and that the resulting binary RPM usually has a wrong MD5 digest (so you have to use --nodigest). o POPT Upgrade RPM 4.2 now uses an enhanced version of the POPT library, which especially allows special "argument eating" markers "!#:+" to be used in the rpmpopt file. This allowed us now to provide convinience macros for %option handling on the command line: --with foo => --define 'with_foo yes' => %option with_foo yes --without foo => --define 'with_foo no' => %option with_foo no --option with_foo bar => --define 'with_foo bar' => %option with_foo bar o %files Improvements: If the payload contains an OpenPGP public key, it can be added to the RPM header with a new %files attribute "%pubkey". Additionally, %exclude will now allow fully exclusion ("everything but"). Then there is a new %dev(type,major,minor) directive to permit non-root device files. o rpmbuild(8): A new command line tool rpmbuild(8) was introduced, providing the official RedHat way of building packages by accepting all the -bX options. Those options were removed from the rpm(8) command, too. OpenPKG provides rpmbuild(8) also, but officially sticks with the single command approach using just rpm(8) for all operations by reenabling the legacy option mappings. o Verification: The "rpm --verify" operation was greatly improved and now checks more details and is more accurate. o Performance: Noticable speed improvements were achieved for both query and install operations. o Transactions: RPM internally now uses transactions for its operations (implemented through data structure coloring) and supports full-state rollbacks in case of problems. o Querying: New --queryformat header format modifiers were introduced: ":xml" for dumping the content in XML (e.g. use --qf '[%{*:xml}\n]' to dump header content in XML), ":base64" to dump in Base64 format and ":armor" to dump in hex format. o File Classification: RPM 4.2 internally now uses the file(3) library for classifying files and can use this on auto-dependency generation. OpenPKG does not use this until now. o Dependency Solving and Caching: Although (still) not used by OpenPKG, RPM 4.2 has some limited functionality (used by RedHat up2date) to cache binary RPM headers locally and even store a full separate RPM database with package information ("rpm --dbpath /path/to/somewhere --justdb -U *.rpm") and then resolve dependencies through this information. This is done with the --aid option which implements at least one of the 2 essential elements of a dependency solver. The 2 essential elements are (a) locate a previously uninstalled package that satisfies a new dependency. (b) upgrade an already installed package in order to solve a problem. RPM currently does (a) only. ---------------------------------------------------------------------------- The following is a stripped down version of the CHANGES entries between RPM 4.0.2 and 4.2.1. Only non-bugfixing entries are shown. See RPM 4.2.1's CHANGES file for complete details. o 4.2 -> 4.2.1: - add version check for package with provides and obsoletes. - build with external beecrypt-3.0.0. - add ":xml" header format modifier. - --queryformat '[%%{*:xml}\n]' to dump header content in XML. o 4.1 -> 4.2: - include file-3.39 (with homebrewed) libfmagic, for now. - add /usr/lib/rpm/rpmdeps. - add /usr/lib/rpm/magic. - use rpmdeps rather than find-{requires,provides}. - move libfmagic to librpmio. - internal automagic dependency generation (disabled for now). - don't generate dependencies unless execute bit is set. - enable internal automagic dependency generation as default. - add --file{class,provide,require} popt aliases and header extensions. - add matching "config(N) = EVR" dependencies iff %%config is found. - add %%pubkey attribute to read armored pubkey files into header. - permit both relative/absolute paths, display 'P' when verifying. - resurrect automagic perl(foo) dependency generation. - add BETA-GPG-KEY (but not in headers using %%pubkey yet). - configurable default query output format. - add file(1) as /usr/lib/rpm/rpmfile. o 4.0.4 -> 4.1: - loosely wire beecrypt library into rpm. - proof-of-concept GPG/DSA verification for legacy signatures. - proof-of-concept PGP/RSA verification for legacy signatures. - add :base64 and :armor format extensions, dump binary tags in hex. - proof-of-concept pubkey retrieval from RPM-{PGP,GPG}-KEY. - stupid macros to configure public key file paths. - 1st crack at making zlib rsync friendly. - add payload uncompressed size to signature to avoid rewriting header. - drill header sha1 into signature parallel to header+payload md5. - mandatory "most effective" signature check on query/verify/install. - add header DSA signature. - add header RSA signature (untested, disabled for now). - wire --nodigest/--nosignature options to checksig/query/verify modes. - add --nosuggests to disable suggested resolutions. - enable CDB by default. - use /etc/rpm/platform (if it exists), rather than uname(2), for arch. - remove build mode compatibility aliases, documented and gone. - add --build/--program-prefix, delete libtoolize, from %configure. - open rpmdb early RDONLY, reopen later RDWR, avoid signed srpm noise. - add preliminary rpmgraph(8) and rpmcache(8) executables to rpm-devel. - display signature details using rpm -qi. - configurable (default off) build failure if missing %%doc files (PLD). - configurable (default off) build failure iff unpackaged files (PLD). - resurrect --rollback. - add --predefine to define macros before reading macro configuration. - add a macro to create a sub-package with debugging symbols. - use %%{_lib} for libraries. o 4.0.3 -> 4.0.4: - permit gpg/pgp/pgp5 execs to be reconfigured. - permit subset installs for %lang colored hardlink file sets. - add :armor and :base64 query format qualifiers for binary signatures. - legacy: configurable whiteout for known Red Hat dependency loops. - macro for %files, always include %defattr(), redhat config only. - transaction rollbacks are functional. - make --addsign and --resign behave exactly the same. - speedup large queries by ~50%. o 4.0.2 -> 4.0.3: - add %exclude support (i.e. "everything but") to %files. - add --with/--without popt glue for conditional builds(Tomasz Kloczko). - add --repackage option to put erased bits back into a package. - regenerate rpm.8 man page from docbook glop (in max-rpm). - add cron/logrotate scripts to save installed package filenames. - fix: popt exec doesn't add '--', --target et al no longer need '='. - document build modes in rpmbuild.8, rpmbuild is born. - permit rpm -qa to take RE args applied to name tag. - rpmdb iterator selectors permit default/strcmp/regex/glob matching. - add --noghost to filter non-payload files from rpm -qlv output. - permit alias/exec description/arg text to be set from popt config. - (--POPTdesc/--POPTargs deletion?) - package version now configureable, default v3. - add linux per-platform macro %_smp_mflags <sopwith@redhat.com>. - add %dev(type,major,minor) directive to permit non-root dev build. - resurrect --specedit for i18n. - verify perms (but not mode) on %ghost files. - fix: ftp remote globs broken (#46892).
