04 March 2009

Oh ... my ... goodness ...

I've been mirroring, rebuilding, patching, filing bugs, and so on, against bleeding edge source package RPMs (SRPMs) out of 'RawHide' for at least a decade. It is one means I have used to push features from later Red Hat releases and hotfixes back into previous releases, to customers in our consulting (and on my own account).

We pushed post-RHL support, and commercial RPM-based support for side architectures including Netwinders [MIPS], PA-RISC, PPC, Sparc hardware, and Alphas from RawHide. We have built FOSS-based 'latest and greatest' LTSP forks of reduced package sets for commercial applications from RawHide

A largely unheralded change to a new RPM package file format for Raw Hide SRPMs coming out to the builder, at Red Hat's rpm-4.6 breaks all that, for the first time in at least a decade ... Jeff Johnson, a former lead developer and maintainer for Red Hat preceding the current incumbent lead, bent over backward and jumped through hoops, to make rpm a lingua Franca. Jeff was followed by a short-time incumbent, Paul Nasrat, who similarly did no harm to rpm

[herrold@centos-5 ctrlproxy]$ cp /mnt/nfs/var/ftp/pub/mirror/redhat/rawhide/SRPMS/ctrlproxy-3.0.8-2.fc11.src.rpm .
[herrold@centos-5 ctrlproxy]$ ls
[herrold@centos-5 ctrlproxy]$ rpmbuild --rebuild \
Installing ctrlproxy-3.0.8-2.fc11.src.rpm
warning: user mockbuild does not exist - using root
warning: group mockbuild does not exist - using root
error: unpacking of archive failed on file /home/herrold/rpmbuild/SOURCES/ctrlproxy-3.0.8.tar.gz;49aeb249: cpio: MD5 sum mismatch
error: ctrlproxy-3.0.8-2.fc11.src.rpm cannot be installed
[herrold@centos-5 ctrlproxy]$ rpm -Vp ctrlproxy-3.0.8-2.fc11.src.rpm
Unsatisfied dependencies for ctrlproxy-3.0.8-2.fc11.src: rpmlib(FileDigests) <= 4.6.0-1
missing ctrlproxy-3.0.8.tar.gz
missing ctrlproxy.config
missing ctrlproxy.init
missing ctrlproxy.spec
[herrold@centos-5 ctrlproxy]$

Amid all that, the part which is important is that: a new rpmlib of at level 4.6.0-1 is needed (marked in red), and that without it, it produces a rather unhelpful cpio md5sum error message (marked in blue)

For the short term, until I get the matter sorted better, I'll set up a domU Raw Hide xen instance (which has the later rpmlib, and so can manipulate the package), such domU will be upgraded just enough to handle rpmlib(FileDigests) <= 4.6.0-1, then frozen against other breakage from later other updates

Further I'll grant that domU RW access to the NFS export that contains my build tree (/home/herrold/rpmbuild/), so that I can position a SRPM into that tree, and unpack it rpm -U packagename.src.rpm

With that unpacked set in the SOURCES subdirectory -- the tarball, patches, and such; and the SPECS subdirectory .spec file, I can then (hopefully) switch into an earlier rpm variant on an older unit, and rebuild the package to write a new SRPM with the older rpmlib form

We'll see. Change requires that the caterpillar moult and break out of the crysalis. It does not require that an angry Mothra result, and destroy the surrounding city in concert with Godzilla.

It might be asserted that this is some sort of performance or speed optimiszation.
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. -- Donald Knuth
rpm has had the instrumentation capabilities in place to see about where performance penalties lie, and has them for some time, so that one can easily test where load is. This is not rocket science; similarly, we drilled in micro-second instrumentation and time-stamping in the shim, as we are accustomed to looking for code bottlenecks, and people using our software, and trading in the financial markets really care about where lags are. I wrote a bit about this last month in the New Future Always Coming piece.

As proof of the pudding, take a moment and run rpm 4.6.0 with --stats on a large package built under the old format, and rebuild and repeat the test with the new 4.6.0 variant. Please feel free to get back to me at: timetrials at owlriver dot com as to where YOU find the true performance issues are, comparing a package build with SHA2, v. MD5. I have my preliminary stats for a later post, but welcome more data

This sure seems like a gratuitous and thoughtless format breakage to me, with no backward compatibility path announced. RHAS 2.1 is about to go out of support, of course (which dates back to a foundation including a fix for the as shipped RHL 8 rpm database locking issues), but RHEL 3, 4 and 5 just lost the ability to use Raw Hide so far as I can currently see

I may be missing something obvious by way of a workaround, and would be glad to be corrected, but ...

I really feel that the current incumbent Panu Matilainen has NOT clearly articulated this on the mailing list and bugs at Red Hat which I read very closely

This WILL cause at a minimum confusion, and also compatibility problems down the road with bi-directional interoperability at the SRPM level between both the LSB, and Red Hat's major enterprise Linux distribution competitor, Novell and its SLES line [seemingly being branded: SUSE Linux Enterprise, presently]. Oracle's UBL and its consumers are sort of off in a 'market niche' world of its own here, and the impact will be less pronounced. It will probably cause such SRPMs to 'just NOT work' in the SuSE buildservice, but I have not tested this yet

Heaven help the users of side distributions -- cAos, Mandriva, PLD, and the non-English fluent RPM file format using -- who try to rebuild and use a random found SRPM. It really seems that Red Hat marginalizes (or forces one to choose a camp to join) other formats with this move. Perhaps that was their intent. Fedora has been used by Red Hat as a 'wedge' for this purpose in the past [consider studied dis-interest in inter-archive compatability], and it may be just more of the same

Grrr ...

Disclaimer: These days, I am aligned with the RPM5 insurgents, and served as the long time maintainer of the old RPM website which Red Hat has since re-claimed