Unit test shepards

I read with interest over the weekend this unit testing and TDD blog post from Douglas Hubler. I met him in real life a few weeks up in Chicago at the annual ClueCon, and was very impressed

I tracked down his email address and started to write a private email, but then as I re-read my draft and his piece, I noticed that it was a 'talking draft' by him. As such, I decided to surface my thoughts here

---

Hi -- Russ herrold (ex CentOS) here -- we met at cluecon

You put your finger on the problem well here:

"Project Maintainers" were always in fear of holding the bag on contributions that introduced bugs while not advancing their employer's goals

which is the well known 'capture by the employer problem' in FOSS. I am not saying (and would never suggest) that employer sponsorship of an interested 'Project Maintainer' is undesirable -- just the opposite, as it funds getting SOME motion in some cases (i.e., when it suits the employer's goals, or is not a clear 'CLM' -- career limiting move). Of course this path leads to 'freeze ups' similar to what we see in Debian Stable, where nothing short of dynamite (or a working remote exploit) seems to work to pry some forward progress into the main trunk

I put on my 'agile' thinking cap, to scope out the implications of your post

To work, the "Unit Test Sheppards" need to have global mandate to commit at least unit tests, via a Version Control System, and there needs to be a working Continuous Integration server. If this 'breaks the build' either the test is wrong, or the code is wrong. In the first response to 'breaking the build' the CI server has to revert the test, and file an exception report, to be owned by the UTS in the first instance, with a CC to the PM

This gets a 'heads up' in front of the PM, and a careful UTS will at a minimum either: 1) acknowledge that the test was ill-considered, withdraw it, and close the bug; 2) amend their code to correct misunderstanding that resulted in a broken test and re-attempt the commit [closing the bug, with the possibility of a 're-entry' of a new bug on the revised test], or 3) add documentation to the bug filing that indicates why the test is right [perhaps something as simple as pointing to a release target milestone, or part of the Requirements document] in preparation to handing the bug off to the PM (staying on the bug as a CC), and handing it along to the PM's queue

One problem is that when there is only a single PM, there is also only a single point of blockage, and 'real life' intervening, or a work-plan to do a substantial refactoring (perhaps even already partially working in a private tree), or even a non-public agenda on the part of one's employer may prevent the PM from ** wanting ** to respond 'just now' if a well-form test and bugreport gets dropped on them ...

... but if the unit test is 'right', usually it is proper to add to a test suite. I put to one side whether one should run all unit tests every pass; Tests do rot and one may well need to trim obsolete tests away, or refactor old ones to match code reorganizations; clearly one answer when the suite gets 'too big' is to start prioritizing, adding stochastic selection to generally omit tests related to rarely encountered failure modes and so forth

But a well written test never fully 'goes away' by default. At some predictable interval, of course, the 'full boat' of ALL tests, as well as more rigorous end to end functional tests are needed. Beck's TDD book glosses over this to some extent as his focus was development, but 'testing' means much more than 'unit testing'

One additional avenue toward a solution would be to convert the single PM 'person' into a trellised PM 'role' or 'team' containing two or more non-affiliated project members

By and large, FOSS works better when there is a consensus approach to management of a resource. It is basic group dynamics that achieving consensus is easier in a small team, able to consult in the 'stand up five minute meeting, and to come to a tactical 'what is the simplest thing that we can do' to conform to a well-formed test, write (or adopt) the unit test, apply it, and move on ;) With only two people in the PM trellis, or a senior and a junior relationship, the group dynamics may result in impasse, which is only visible to the 'outsider' UTS as 'nothing is happening on this bug'

Lots of inter-person political approaches exist here, but ultimately and in most projects, there is a agreed-to Release Manager team with global commits, that has to be willing and able to 'take up the reins', intervene to intentionally 'break the build' in HEAD when an impasse continues 'too long' [I assume here a model of a stable release, and a developmental HEAD], and force the PM to respond (perhaps by relinquishing participation as a co-PM)

I don't have an obvious candidate solution to suggest here, as there as many approaches are possible, and I've seen the issue as a project lead, as well as a mere participant, and sometimes simply as a concerned onlooker

Free for some people just means they are not footing the bill ... maybe

I see the following in the New York times today:

Health Plans Must Provide Some Tests at No Cost
By ROBERT PEAR

Published: July 14, 2010

WASHINGTON — The White House on Wednesday issued new rules requiring health insurance companies to provide free coverage for dozens of screenings, laboratory tests and other types of preventive care.

The new requirements promise significant benefits for consumers — if they take advantage of the services that should now be more readily available and affordable.

In general, the government said, Americans use preventive services at about half the rate recommended by doctors and public health experts.

The rules will eliminate co-payments, deductibles and other charges for blood pressure, diabetes and cholesterol tests; many cancer screenings; routine vaccinations; prenatal care; and regular wellness visits for infants and children. ...

I assume that the reporter no longer believes in the tooth fairy. The article is tailored as news, and placed in that section of the paper (Page A16) by the Times editors. It has a laundry list of wonderful tests and services that no 'right thinking' person can deny are useful and desired

But the suggestion is a 'promise [of new] significant benefits for consumers — if [only] they take advantage of the services' without a corresponding cost for getting there. No hint nor argument is made that such 'medical' services are unavailable for private purchase already

Indeed, at the end of the day, there is no support for the headline writers assertion of 'no cost' and the reporter is well willing to disregard the pesky question of how to pay for this largess. Clearly these tests are not free and when accounts are settled; these costs will either pass through in a rate base, or the provider will exit the market it cannot make money in, or the insurance market will wither and die as 'the government' provides an 'option' that picks up the tab ... . But the problem is -- 'the govenment' at whatever level likewise needs to get the money to pay for such happy healthiness, and from the very same pool of people 'benefitted'

It is not at all clear that the transaction friction of a single govenment payer works at all well, or that having no choice but 'insurance' through government once the private insurers die is a good thing at all. In watching the 'response' of the government to the oil spill in the Gulf, it is patently clear that government 'oversight' has slowed the response, as BP has become risk adverse to the (reasonable) prospect of being second-guessed at every turn, and so is seeking prior governmental approval before acting in the remediation. The ccase can be made that playing 'Mother may I?' has harmed the Gulf more than the prior approach

Do we really think that a central government single point of control is going to react as well and quickly as a local doctor on the scene, when Aunt Minnie is lying, dying under an oxygen tent and needs some immediate surgery? Under the current system, the doc knows that he'll get paid, perhaps only in part of what is billed as a 'list price' for a prodedure, but eventually from the present model

But that is the end game, anyway, right? Vote and mandate 'bread and circus entertainment' ... until the producers all surrender and act to stop being charged for 'free' benefits to the consumers

'The problem with socialism is that eventually you run out of the other peoples (willing to be robbed of their) money'

Who is in charge, here?

"When I use a word," Humpty Dumpty said, in a rather scornful tone, "it means just what I choose it to mean - neither more nor less."
"The question is," said Alice, "whether you can make words mean so many different things."
"The question is," said Humpty Dumpty, "which is to be master - that's all."
   -- Through the Looking Glass

A U S Supreme Court memorandum order (called a 'Slip Opinion' here) today said:

PER CURIAM.
The writ of certiorari is dismissed as improvidently
granted.
It is so ordered.

http://www.supremecourtus.gov/opinions/08pdf/07-1216.pdf

Some Latin in there. 'PER CURIAM' is: By the Court as a complete panel and entity, without specific attribution of the action to any particular Justice. A 'writ of certiorari' is: a publicly stated intent of the Court to receive a case for presentation and argument, and possibly (usually) decision.

That a matter is characterized as: "improvidently granted" is not Latin, but is a 'term of art' -- basically: It turns out we (as a decision making body) cannot, will not, or should decide after, and we decline to consider the particular aspect of the case we initially though we should hear for the present.

It happens -- maybe with one member of the Court ill, the Court decided it needed to lighten its load; perhaps some conflict came to light in the investment portfolio of some Justices that the remaining (non-recusing) panel members of the Court felt now that fairly they could not hear and decide the matter, to avoid the appearance of making a biased, short handed, or improper decision.

The functional effect of such a terse statement is to leave intact and in effect the next prior lower court's ruling

---

While a lawyer's craft is well depicted in television crime procedurals, the more cerebral parts of avoiding issues which make one interesting for television, are decidedly more valuable services a lawyer provides to the general society, when advising a client. A lawyer's opinion can help the client see the areas to avoid; how to structure its affairs. The Court speaks in a completely understood fashion here, to communicate just what it intends to say, and nothing else

The recent 'mob rule' in Congress of vilifying and proposing to punishingly tax the folks at AIG who had clear contracts for payment of a 'retention' and performance bonus, in exchange for staying on as the ship of AIG seemed to be sinking (and indeed staying and doing their contractual duty) show how fragile civilization is. All the hollow words about 'fairness' and populist anger cannot mask the fact that Congress successfully pointed the finger of blame away from itself and the government, and toward a tiny minority of our society. If a Constitution, contracts, and rule of law are so readily cast aside, no one is safe. Paris 1789 and following, all over again. Whom shall the mob turn to next?

The use of words is how we explain to another, and sometimes to ourselves, what we are thinking; why we believe what we believe; permit us to reflect and find weak points in our thought processes. Structured words -- Court decisions, Constitutions, laws, opinion letters from lawyers are part of an ongoing societal dialog

We are all diminished when cheap talk trumps reason

Seven layer ISO stack, plus two

The Seven layer OSI model stack -- an ISO standard, palindromically -- characterizes how computer and data network (and thus internet) applications communicate between separate computers in a defined fashion. From the bottom up: Physical, Data Link, Network, Transport, Session, Presentation, and Application layers

Computer networking geeks use that the seven-layer stack, to specify and to analyze matters running from what a given voltage level means at the bottom (Physical layer), to how a email client retrieves email from a mailstore at the top (Application layer).

In preparing this piece, I find see reference some to mnemonic forms to get the sequence of the Layers correct that I did not know before:

The mnemonics "People Design Networks To Send Packets Accurately", "Please Do Not Throw Sausage Pizza Away", and "All People Seem To Need Data Processing" may help you remember the layers.
BambooWeb article

Some wags extend the definition thus:

Of course, above those seven layers are two more: the Financial, and the Political Layers, which also need to be functional.

The dis-functionality and friction of office politics, or perhaps of being unable to get funding needed for a more reliable and functional network infrastructure rather than a Bigger, Newer, Better computer to sit unused on a pointy headed boss' desk, each come to mind as obvious examples. In wider scopes, Financial may encompass Economics and Markets; Political expands to social interaction, policy and polity

It seems in at least one restatement, to also have added an explicit 'religious' layer as well under some restatements. I think one can perhaps subsume Religious into Political, but I can see the genesis of the historical tension to draw a difference between those asserting dominion over a person's earthly presence, and spiritual corpus.

Network technicians will sometimes refer euphemistically to "layer-eight problems," meaning problems with an end user and not with the network.

Carl Malamud, in his book [1991] book"Stacks," defines layers 8, 9, and 10 as "Money", "Politics", and "Religion". The "Religion layer" is used to describe non-rational behavior and/or decision-making that cannot be accounted for within the lower nine levels. (For example, a manager who insists on migrating all systems to a Microsoft platform "because everyone else is doing it" is said to be operating in Layer 10.)
BambooWeb article

I see also that people have adapted use of [pdf] the Political and Finacial Layers. No model is so good it cannot be twisted, abused and extended, it seems

Anyway, I lay this foundation to provide a kick-off target to point back to in future discussions of the Political and Financial layers of the stack

Behind Blue Eyes

Dateline: U.S. Department of Labor

The largest increases in initial claims for the week ending Nov. 1 were in Ohio (+3,885), Michigan (+2,619), Pennsylvania (+2,155), Wisconsin(+2,119) ...
-- UNEMPLOYMENT INSURANCE WEEKLY CLAIMS REPORT (week ending Nov. 8 2008)

A friend wrote:

All this proves is that when someone crosses a state line to register to vote it is just as easy to register for unemployment while you're at it.

I think it is probably much worse than that

It is easy enough for anyone to set up a (several!) new 'employers' and then walk away from them 8 weeks later with no individual financial responsibility for the 'tail' -- after all, we encourages formation of 'small business, the engine of economic growth' and barriers to entry should be small, right?

Unemployment benefits may be had at full rate for 6 months after 6 weeks employment at a given 'employer' if one is otherwise qualified; when an 'employer' goes out of business, the employees are eligible for benefits Several telephone poles in central Ohio had signs, with differing phone numbers, for what appeared to be short term 'jobs' working to elect Obama and 'make Change'. I snapped a picture with my mobile device, and will see if I can find it for the exact text; I recall thinking at the time:

-- Don't the 'employee candidates' KNOW they will be let go the day after the election

Now, I think the answer is:

-- Sure -- indeed they were TOLD by the recruiter at the other end of the phone, that this was a way to get rid of a pesky 'termination for cause' {disqualifying} black mark which was keeping them from what they were 'entitled to'

As the One won, and 'We can do it!' if the system is properly 'gamed', I think there will be no investigations after Jan 20 to 'connect the dots', and the Lame One will just snooze out his term. 'No law will prevent it'

And so the Republic was lost. "Meet the new boss; same as the old boss"

Sledding down the slippery slope

Mr Dooley reads the paper:

08:52 Facing shortfall, Massachusetts inquires about a Federal loan - NY Times

NY Times reports the Massachusetts state treasurer has asked the federal government about lending the state money under the same favorable terms given to banks and investment firms during the financial crisis ...

Call me old fashioned, but wasn't this result perfectly predictable [to the Fed, to Treasury, and to the Joint Economic Committee], once starting down the 'moral hazard' path?

It is too early for strong drink, but ...