19 June 2013

I am not Harry Truman

I received a email from a customer, followed by a phone call, to the effect they had received huge number of email 'return' bounces to a general intake email address.  He and I have had this discussion before

I have written about email sender forgery (There is probably NOT an email account: godzilla@microsoft.com) and its fallout ("Customer: My cousin says that his email to me is not going through") before.  So let's take the time to think it through yet again

Takeaway: He wanted me to stop such pieces from cluttering their email box, but he is unwilling to have 'heavy' spam filtering

As a personal matter, and also wearing my sysadmin hat, I would like to stop seeing this cruft as well

But as a technical matter, it seems that it cannot easily be done without constant 'tuning' of rejection rules or some other rather serious matching of 'Message-ID' of pieces sent against return pieces offered.  An attempt to do so through filtering tools with no prior knowledge of Message-ID's sent, is to always 'play defense' against the spammers, without an ability ever score a 'win'.  The effort to match Message-Id's in offered return pieces is perhaps more promising

But, so far, no-one has been sufficiently vexed by it in the FOSS community to publish such a tool and to commit do doing do the ongoing 'tuning' of message parsers needed.  Perhaps we can design around it with existing tools, and amending our outgoing pieces by adding a certification that a given candidate email is truly from us

As a design matter, building a milter, writing some procmail rules, and parsing sendmail logs, probably into a database backend, as my first thought as to how I would approach the matter.  The database constraint is troubling, though.  I have other work that I need to attend to first, but I went through the thought process.  I memorialize that process in part in case someone is interested.  Even more, I will provide webspace, mailing list support, and a VCS gratis, if someone 'feels the itch'.  It would be useful to have, but is not urgent to attain -- Seven Habits Quadrant 2 or 4 stuff.  Absent such a volunteer effort or a paying customer, for me, Quadrant 4

Or, version two, a trusted cohort of outbound mailservers could build a MAC MIME Multipart attachment for each outbound message, and also a second MIME attachment that is cryptographically validated 'clearsign' of that MAC part.  Possibly bundle this up into a Multipart Related set of structured attachments.  Add these two new MIME attachments to all messages on every outbound piece.  The first part -- the MAC part -- would be based a hash of the message body, plus a timestamp of seconds since Epoch or such, and other optional entropy, to avoid forgery and replay attacks

Later then, when a putative return is offered, only accept for further processing those returns that had a validating pair of MIME attachments, produced based on a re-hash the message body in chief, and that MAC section's timestamp; and  that had previously clearsigned by it. Discard stale stuff, and non-validating content. This gets rid of the need for the database and simplifies the procmail rules.  A well-formed candidate return piece can carry around all that is needed to known to decide if one will pass a mail return message along to human eyes

Not free, as it will burn up compute cycles on every send, and a few more at return time, but also complete and under controllable locally so resistant to spammers.  Avoids the database requirement, so it can scale out. Most of the needed tools already exist as FOSS.  hmmm


The protocols governing what constitutes: email permit a sender to enter whatever 'return address, and 'sender address' they wish on a piece of email.  It is trivial to find a 'open' relay to accept email to send to any third party.  Consider the analogy:
  • All the while being careful to not leave a fingerprint or other biometric, I use cash to purchase a post card at the corner store, along with a stamp
  • I address it to someone of tender sensibilities, and assert that I noticed that their car was parked outside the local 'adult entertainment' establishment
  • I sign it: Harry S Truman
  • I enter a 'return address' of:

  Harry S Truman
  President Emeritus
  1600 Pennsylvania Avenue
  Washington DC  20500
  • I mail it


  • The recipient is outraged to find such a libelous assertion, visible for their letter carrier to see, and demands that the person who did so be identified, and stopped.  Also, for good measure, they want the Postal Service Inspectors to get on the matter to prevent such heartbreaking assertions to never happen again

    About all the Postal Service will offer to do in the usual case is to return the piece to its nominal sender. And he no longer receives mail at that address

    (I note parenthetically that the Postal Service DOES seem to scan images of ALL paper mail passing through their system)


    Stopping spam (here: bounce backsplatter and 'joe jobs') is just not going to turn out to have a durable, easy, and comprehensive solution, without re-thinking what we send looks like.  Spammers and legitimate receivers are in a 'arms race' and today's fix will rot if senders can re-engineer around the fixes.  If this state of affairs distresses a person greatly and until I can get that MIME solution going to test my hypothesis: stop reading email; hire a full time, 24x7 secretary to pre-read all email and toss the junk.; turn up the filtering and accept the false positives; grow a thick skin

    Or, of course, start coding and beat me to it

    13 June 2013

    Phone call: 'I've got this sick machine ...'

    me:  well, why it is sick?

    them:  yum complains about a missing signing key

    me: so install the key; it is down in /etc/pki/rpm-gpg/, and rpm --import ... will do the trick

    them: that directory is not there

    me: who set up the machine?

    them:  well, I was handed it, and ...

    me: so, take a level zero backup and then clean up the machine before trying to work on it, or deploy a new one

    them: well, I can't

    I just got off that call from a friend in a new employment situation

    The technical fix was outlined by me long ago, and I sent an email with the link along to the person calling

    BUT: Fixing the mindset inside the caller's head: do not try to work in a undefined (here: broken) environment is harder

    But the caller has a problem in their work-flow process; a fix has to be done; sooner is probably better than later; a broken machine in production is 'technical debt', pure and simple.  Fundamental expectations are not met; binary partition will not work well to isolate problems, as more than one piece is probably broken.  It will break again, and a perception may well form that the caller may be the problem, rather than the broken environment they were handed

    Be sure to make a note to yourself to also address the broken process that permitted that machine to escape into production.  Dollars to doughnuts, there is 'more than one roach' lurking.  I'll cover a bet that there are not tested backups in that shop