09 March 2009

CentOS 5.3 qa - fresh installs testing on i386

centos-5.3 QA fresh installs
KB released released a twitter note:
ah! the smell of a freshly baked distro.
earlier today, reflecting the presence of updated binaries, and wire-install ready images. Yummy, and I noted them in my twitter feed.

I completed the rsync of the new candidate archive, and have knocked out both GUI and text mode domU wire installs of the i386 candidate. As you may recall, I had done updates testing before, without incident.

The screenshot is fullscreen of 1440x1050 and I am not about to booger the PCO feed again -- direct link is at: http://www.herrold.com/images/blog/c53.-qa.jpg

The test running (side by side in two xen domU test candidates) is a rather hard one, which build graphviz, on the way to the trading shim -- the test script can be retrieved from:
ftp://ftp.owlriver.com/pub/local/COLUG/shim-builder.sh
for those who want to play along.

Something is slightly wrong in the install of the graphviz binary produced, and I'll get that fixed. Also there is some LaTeX issue which I need to run down in the current doco build, generated from a SPOT -- a single point of truth. The -f option to shim-builder.sh may be enough to get past it, regenerating the .dot files, and this the .ps included art.

It succeeded in the GUI based installation. We have some slight warning level noise in today's shim in building today's shim, but it is just c++ being picky (hoo-ray for type checking). The TUI test lags, as I had to build the TUI instance separately, and get the local install archive set up in /etc/yum.repos.d/

This is ready to go as to my needs, but I'll test media installs, and on x86_64, next.

06 March 2009

Wrestling blog markup, Round Three



Lovely ... the large image in the prior post boogered the CentOS aggregator. Posting this to push it further down the page.

Where is that coffee cup?

Wrestling blog markup, Round Two

Perhaps I should have thought through what I was saying in preparing a posting a bit more to avoid getting bitten

This should look better in the CentOS consolidator:

ring girl, round two

PCO nine thumb  Stuck between a rock and a hard place here, it seems. Notice the ocean of whitespace to the left of the picture of the pregnant lady on the left image  OSE nine thumb


than this mess:
PCO eats markup

PCO eats markup diffferently than Blogger. Point taken.

But, dang it: 'wrangling' through blogging software quirks by adding tables with:
<td>&nbsp;<img src="http://www.herrold.com/images/blog/spacer.gif" width="0" height="0">&nbsp;</td>
blocks littered through it sure feels nasty and a lot like fighting browser quirks in a by-gone era. This should be WYSIWYG

 Where did my coffee cup get to? 

Wrestling blog markup

Stuck between a rock and a hard place here, it seems. Notice the ocean of whitespace to the left of the picture of the pregnant lady on the left image


I had previously noticed that the Centos feed was honoring exposing the <hr> 'separator rules' in my markup, where the Blogger was not, on my OMG post earlier this week. Not a big deal, and I did a quick re-edit and report, but I dislike doing so. It exposes a problem in the Blogger preview engine as well, I guess

But darn it -- I should not have to set up a table to box image placement, at this late date. I had to do so to get the 'clown' picture at the bottom of Nine pregnant gals to lay right at the bottom in the Blogger client. Then I see these side-by-sides once it hits the CentOS consolidator:

OSE
PCO


Grrr ....

News recap for Rip van Winkle

Rip van Winkle
This posted, in light of a question seen today:

CentOS' upstream published an announcement, reminding users that its 2.1 line reaches its scheduled end of life soon.

By the way: Just a quick 'shout out' and 'Thank you', by the way to John Newbigin, for being its shepard for the CentOS project.

From memory, substantive bug fixes were turned off, say, a year and a half ago. John has been there for CentOS 2.1 (based on a product itself based on the largely lame and unloved RHL 8). RHL 8 was a sick puppy, but also a gateway to the upstream's re-entry into the Linux market with the 'Enterprise' model, and so to CentOS itself

Reviewing this page, I see: tzdata enhancement update 2008-11-05, but that scarcely counts. Before that, libxml2 bug fix update 2008-09-02 (but that carries security overtones)

Before that, the first 'real' bugfix seems to be: dump bug fix update 2007-02-02 -- So, two years now since a material (non-security) bug fix.

Time to get that conversion to CentOS 3 series done.

Nine pregnant gals in a queue

a pregnant lady
I wrote a bit ago on the fact that the CentOS 5.3 update release was proceeding apace. I may have been insufficiently direct.
If you feel you need the facilities provided by the CentOS project sooner than it is provided, or that you need deterministic releases of support: Please go buy such from our upstream, or from a third party vendor who can sell you the expedited subset of services truly needed.

Almost all of the CentOS team have active consulting practices, or had such before their present $DAYJOB. They had demonstrated the ability to handle the matter.

Similarly, CentOS' upstream has a unit cost for a three year JBOSS and their enterprise distribution product of US $297 -- WITH non-metered support -- This is unbeatable to point your pointy haired boss at when she is badgering you about 'CentOS is late'.

We are not late of course; we will issue the 5.3 respin when it is right, and not before. There was some loose talk speculating that there were insufficient resources, or that QA testing had not begun. Neither is correct.

Consider the well known lesson as to the futility of trying to parallelize a task at a sequential constraint chokepoint. This was pointed out by Fred Brooks in The Mythical Man Month.
Adding manpower to a late software project makes it later.

Brooks constrains his example to late tasks, but it turns out this is a broader rule than that, and has general applicability.

A quick example might be to consider the time it takes to produce one new human baby. For one woman, the time is nine months. No matter how hard they try, nine women working in parallel cannot get that one baby any quicker by adding eight more pregnant women to that queue.

CentOS has some goals in its build process which the upstream does not -- we strive to produce the packages we release on a 'self-hosting' basis, so that anyone who works at it can replicate our work freely. Upstream has never had that goal in their RHL nor now their Enterprise product. We have to identify failures with the tools such as the ones KB talked about recently.

Also, build sequence matters a lot in bootstrapping into a next point release; there are hidden build order dependencies which need to be solved -- sort of like packing a station wagon with furniture and household goods, when moving. The big stuff HAS to go in first, and the little stuff later placed in 'found' gaps. This cannot be well parallelized.

We have the QA team up and primed [there is a non-public webpage, and I see 29 members by a quick count]; the needed ACL's to get at the candidates are in place and tested [I pushed an update earlier this week for one member]; some QA has occurred. I updated some results I had announced a couple weeks ago, in a coordinating mailing list on the QA's notes on this release as well.

This is a little too wide a parallelizing fanout, in terms of coordination of testing, but it happens to be how it turned out this time. The future with a CentOS 4.8 and 6.0 coming will probably be a bit smaller. The QA master (and indeed I hope, each of the 29 individually) will review the participation at the end of this cycle, and some on the list will get dropped from a QA role, and slots freed up for new members to be invited in.

Experience has shown that there is no sense adding 'community' that does not put their shoulder to the wheel and work; We on the CentOS team see the laments from people wanting to join. It seems to me from watching, that they really want to consume and not carry the load of CentOS. Note that 'users' of CentOS are not our target 'community'; they are welcomed, but really, how does having lots of support HELP the project; the main IRC channel #centos is consciously limited to CentOS specific issues, and structured as a learning environment for a reason.

Want to be asked onto CentOS QA? Go for it, there are no barriers to demonstrating competence and interest in any of the following venue: file good bugs; comment bugs with reproducers or better, fixes; participate on a sustained basis in a knowledgeable fashion on any of: mailing list, wiki, forums, and IRC

The CentOS team watches all these venue all the time -- the CentOS 'community' is a meritocracy, and merit will be welcomed in -- but also know, this means there is an implicit 'Bozo filter' as well.clowns

04 March 2009

Oh ... my ... goodness ...

I've been mirroring, rebuilding, patching, filing bugs, and so on, against bleeding edge source package RPMs (SRPMs) out of 'RawHide' for at least a decade. It is one means I have used to push features from later Red Hat releases and hotfixes back into previous releases, to customers in our consulting (and on my own account).

We pushed post-RHL support, and commercial RPM-based support for side architectures including Netwinders [MIPS], PA-RISC, PPC, Sparc hardware, and Alphas from RawHide. We have built FOSS-based 'latest and greatest' LTSP forks of reduced package sets for commercial applications from RawHide

A largely unheralded change to a new RPM package file format for Raw Hide SRPMs coming out to the builder, at Red Hat's rpm-4.6 breaks all that, for the first time in at least a decade ... Jeff Johnson, a former lead developer and maintainer for Red Hat preceding the current incumbent lead, bent over backward and jumped through hoops, to make rpm a lingua Franca. Jeff was followed by a short-time incumbent, Paul Nasrat, who similarly did no harm to rpm

[herrold@centos-5 ctrlproxy]$ cp /mnt/nfs/var/ftp/pub/mirror/redhat/rawhide/SRPMS/ctrlproxy-3.0.8-2.fc11.src.rpm .
[herrold@centos-5 ctrlproxy]$ ls
ctrlproxy-3.0.8-2.fc11.src.rpm
[herrold@centos-5 ctrlproxy]$ rpmbuild --rebuild \
   ctrlproxy-3.0.8-2.fc11.src.rpm
Installing ctrlproxy-3.0.8-2.fc11.src.rpm
warning: user mockbuild does not exist - using root
warning: group mockbuild does not exist - using root
error: unpacking of archive failed on file /home/herrold/rpmbuild/SOURCES/ctrlproxy-3.0.8.tar.gz;49aeb249: cpio: MD5 sum mismatch
error: ctrlproxy-3.0.8-2.fc11.src.rpm cannot be installed
[herrold@centos-5 ctrlproxy]$ rpm -Vp ctrlproxy-3.0.8-2.fc11.src.rpm
Unsatisfied dependencies for ctrlproxy-3.0.8-2.fc11.src: rpmlib(FileDigests) <= 4.6.0-1
missing ctrlproxy-3.0.8.tar.gz
missing ctrlproxy.config
missing ctrlproxy.init
missing ctrlproxy.spec
[herrold@centos-5 ctrlproxy]$


Amid all that, the part which is important is that: a new rpmlib of at level 4.6.0-1 is needed (marked in red), and that without it, it produces a rather unhelpful cpio md5sum error message (marked in blue)

For the short term, until I get the matter sorted better, I'll set up a domU Raw Hide xen instance (which has the later rpmlib, and so can manipulate the package), such domU will be upgraded just enough to handle rpmlib(FileDigests) <= 4.6.0-1, then frozen against other breakage from later other updates

Further I'll grant that domU RW access to the NFS export that contains my build tree (/home/herrold/rpmbuild/), so that I can position a SRPM into that tree, and unpack it rpm -U packagename.src.rpm

With that unpacked set in the SOURCES subdirectory -- the tarball, patches, and such; and the SPECS subdirectory .spec file, I can then (hopefully) switch into an earlier rpm variant on an older unit, and rebuild the package to write a new SRPM with the older rpmlib form


We'll see. Change requires that the caterpillar moult and break out of the crysalis. It does not require that an angry Mothra result, and destroy the surrounding city in concert with Godzilla.

It might be asserted that this is some sort of performance or speed optimiszation.
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. -- Donald Knuth
rpm has had the instrumentation capabilities in place to see about where performance penalties lie, and has them for some time, so that one can easily test where load is. This is not rocket science; similarly, we drilled in micro-second instrumentation and time-stamping in the shim, as we are accustomed to looking for code bottlenecks, and people using our software, and trading in the financial markets really care about where lags are. I wrote a bit about this last month in the New Future Always Coming piece.

As proof of the pudding, take a moment and run rpm 4.6.0 with --stats on a large package built under the old format, and rebuild and repeat the test with the new 4.6.0 variant. Please feel free to get back to me at: timetrials at owlriver dot com as to where YOU find the true performance issues are, comparing a package build with SHA2, v. MD5. I have my preliminary stats for a later post, but welcome more data


This sure seems like a gratuitous and thoughtless format breakage to me, with no backward compatibility path announced. RHAS 2.1 is about to go out of support, of course (which dates back to a foundation including a fix for the as shipped RHL 8 rpm database locking issues), but RHEL 3, 4 and 5 just lost the ability to use Raw Hide so far as I can currently see

I may be missing something obvious by way of a workaround, and would be glad to be corrected, but ...

I really feel that the current incumbent Panu Matilainen has NOT clearly articulated this on the mailing list and bugs at Red Hat which I read very closely

This WILL cause at a minimum confusion, and also compatibility problems down the road with bi-directional interoperability at the SRPM level between both the LSB, and Red Hat's major enterprise Linux distribution competitor, Novell and its SLES line [seemingly being branded: SUSE Linux Enterprise, presently]. Oracle's UBL and its consumers are sort of off in a 'market niche' world of its own here, and the impact will be less pronounced. It will probably cause such SRPMs to 'just NOT work' in the SuSE buildservice, but I have not tested this yet

Heaven help the users of side distributions -- cAos, Mandriva, PLD, and the non-English fluent RPM file format using -- who try to rebuild and use a random found SRPM. It really seems that Red Hat marginalizes (or forces one to choose a camp to join) other formats with this move. Perhaps that was their intent. Fedora has been used by Red Hat as a 'wedge' for this purpose in the past [consider studied dis-interest in inter-archive compatability], and it may be just more of the same

Grrr ...

Disclaimer: These days, I am aligned with the RPM5 insurgents, and served as the long time maintainer of the old RPM website which Red Hat has since re-claimed