... I am the eggman

One recent addition to Python modules packaging at Red Hat in its Fedora project, is carrying along an additional, and optional structured metadata about the contents of that module (package), held outside of the RPM database

This additional information: egg-info came into Python at Python version 2.3 and following. More may be learned about these eggs at: egg-info (optional extra Python metadata about a Python module).
See: Fedora specifics

This new detail is a two edged sword. On one hand, it provides sufficient information that an ad hoc root level process, for when one using native Python tools that it makes for an easy_install [the skeptic in me suggests it might be 'easier', perhaps, along some skewed axis of metric of goodness]. See also the egg superset, Python setuptools which now work well in RPM-mediated space

Sadly, down this path, modules which are outside of the protections and managae-ability of the RPM packaging system may "easily" and inadvertently be introduced by an incautious admin, and thus introduce of Python code into a otherwise controlled system. This is the horror of a mixed RPM and CPAN system, all over again. As I say, one needs to choose a 'metric of goodness' with care

Incautious use of mixed packaging approaches in turn can lead to possible security and updates headaches. Using such non-packaging system tools can break the SPOT -- single point of truth -- to determine to what versions of binaries a given host is using. From sad experience, this way lies additional work, and madness

But on the other edge of that blade, the egg-info adds descriptive narrative, and cautiously used, increases the usability of a system that does not (ab)use those native installation tools

As noted, the FOSS world has faced this problem before with perl and CPAN. Weak and strong 'includes' versioning and security model questionable @INC 'include' path search practices in Python and perl are well known failings in their community archive models. I faced it recently in a packing push of CRAN modules for R -- hmmm, I still need to file a few bugs upstream to solve some problems I saw in some R module packaging choices that I consider poor ones

There is not a single, objectively 'technically right' way to proceed, but rather just one consistent, or not consistent with packaging system design and usage choices

Fedora helpfully offers a sample stanza to use in .spec files. I cruised my archive of .spec files to see what else turned up


# See if there's any egg-info
if [ -f %{buildroot}%{python_sitearch}/Conch*.egg-info ]; then
echo %{buildroot}%{python_sitearch}/Conch*.egg-info |
sed -e "s|^%{buildroot}||"
fi > egg-info


and later then using the %files stanza's -f file list option


%files -f egg-info
%defattr(-,root,root,-)
%doc LICENSE NEWS README doc/*
%{_bindir}/cftp
...


It appears I need to do some work in my local archive of SRPMs. Never enough hours in the day

CentOS 5.3 qa - fresh installs testing on i386

KB released released a twitter note:

ah! the smell of a freshly baked distro.

earlier today, reflecting the presence of updated binaries, and wire-install ready images. Yummy, and I noted them in my twitter feed.

I completed the rsync of the new candidate archive, and have knocked out both GUI and text mode domU wire installs of the i386 candidate. As you may recall, I had done updates testing before, without incident.

The screenshot is fullscreen of 1440x1050 and I am not about to booger the PCO feed again -- direct link is at: http://www.herrold.com/images/blog/c53.-qa.jpg

The test running (side by side in two xen domU test candidates) is a rather hard one, which build graphviz, on the way to the trading shim -- the test script can be retrieved from:
ftp://ftp.owlriver.com/pub/local/COLUG/shim-builder.sh
for those who want to play along.

Something is slightly wrong in the install of the graphviz binary produced, and I'll get that fixed. Also there is some LaTeX issue which I need to run down in the current doco build, generated from a SPOT -- a single point of truth. The -f option to shim-builder.sh may be enough to get past it, regenerating the .dot files, and this the .ps included art.

It succeeded in the GUI based installation. We have some slight warning level noise in today's shim in building today's shim, but it is just c++ being picky (hoo-ray for type checking). The TUI test lags, as I had to build the TUI instance separately, and get the local install archive set up in /etc/yum.repos.d/

This is ready to go as to my needs, but I'll test media installs, and on x86_64, next.

Nine pregnant gals in a queue

I wrote a bit ago on the fact that the CentOS 5.3 update release was proceeding apace. I may have been insufficiently direct.

If you feel you need the facilities provided by the CentOS project sooner than it is provided, or that you need deterministic releases of support: Please go buy such from our upstream, or from a third party vendor who can sell you the expedited subset of services truly needed.

Almost all of the CentOS team have active consulting practices, or had such before their present $DAYJOB. They had demonstrated the ability to handle the matter.

Similarly, CentOS' upstream has a unit cost for a three year JBOSS and their enterprise distribution product of US $297 -- WITH non-metered support -- This is unbeatable to point your pointy haired boss at when she is badgering you about 'CentOS is late'.

We are not late of course; we will issue the 5.3 respin when it is right, and not before. There was some loose talk speculating that there were insufficient resources, or that QA testing had not begun. Neither is correct.

Consider the well known lesson as to the futility of trying to parallelize a task at a sequential constraint chokepoint. This was pointed out by Fred Brooks in The Mythical Man Month.

Adding manpower to a late software project makes it later.

Brooks constrains his example to late tasks, but it turns out this is a broader rule than that, and has general applicability.

A quick example might be to consider the time it takes to produce one new human baby. For one woman, the time is nine months. No matter how hard they try, nine women working in parallel cannot get that one baby any quicker by adding eight more pregnant women to that queue.

CentOS has some goals in its build process which the upstream does not -- we strive to produce the packages we release on a 'self-hosting' basis, so that anyone who works at it can replicate our work freely. Upstream has never had that goal in their RHL nor now their Enterprise product. We have to identify failures with the tools such as the ones KB talked about recently.

Also, build sequence matters a lot in bootstrapping into a next point release; there are hidden build order dependencies which need to be solved -- sort of like packing a station wagon with furniture and household goods, when moving. The big stuff HAS to go in first, and the little stuff later placed in 'found' gaps. This cannot be well parallelized.

We have the QA team up and primed [there is a non-public webpage, and I see 29 members by a quick count]; the needed ACL's to get at the candidates are in place and tested [I pushed an update earlier this week for one member]; some QA has occurred. I updated some results I had announced a couple weeks ago, in a coordinating mailing list on the QA's notes on this release as well.

This is a little too wide a parallelizing fanout, in terms of coordination of testing, but it happens to be how it turned out this time. The future with a CentOS 4.8 and 6.0 coming will probably be a bit smaller. The QA master (and indeed I hope, each of the 29 individually) will review the participation at the end of this cycle, and some on the list will get dropped from a QA role, and slots freed up for new members to be invited in.

Experience has shown that there is no sense adding 'community' that does not put their shoulder to the wheel and work; We on the CentOS team see the laments from people wanting to join. It seems to me from watching, that they really want to consume and not carry the load of CentOS. Note that 'users' of CentOS are not our target 'community'; they are welcomed, but really, how does having lots of support HELP the project; the main IRC channel #centos is consciously limited to CentOS specific issues, and structured as a learning environment for a reason.

Want to be asked onto CentOS QA? Go for it, there are no barriers to demonstrating competence and interest in any of the following venue: file good bugs; comment bugs with reproducers or better, fixes; participate on a sustained basis in a knowledgeable fashion on any of: mailing list, wiki, forums, and IRC

The CentOS team watches all these venue all the time -- the CentOS 'community' is a meritocracy, and merit will be welcomed in -- but also know, this means there is an implicit 'Bozo filter' as well.