Generating a valid Date:-header when using the Zope-sendmail tag

by Frank Tegtmeyer <fte@fte.to>

Daniel J. Bernsteins tools save the world again: this time the ofmipd daemon from the mess822 package is used to provide functionality that is missed from the sendmail tag in Zope.


Note: The problem went away with Zope version 2.6. This version generates the Date:-header itself, so there is no need for any workaround. Still it's nice to have the message-id generated, so I will keep the document here.
The text will stay in it's current form, so don't wonder why 2.3.1 is called the actual Zope version.


The problem

In April 2001 I recognized that the handy sendmail tag in Zope doesn't create a Date: header when transmitting a mail message. This is mostly no problem but this header is a required header for generated mails.

I observed this in Zope 2.3.0, but a quick check of the CHANGES files showed that this was not corrected in the current version 2.3.1 and beta versions.
In the meantime I filed a bug report in the collector to make DC aware of the issue. The collector number is #2179. The current status is "postponed". Have a look at the section "status of the bug report" what happened after filing the bug report.

There is another thing missing from many mail sending scripts (and also at the sendmail tag): generation of message-id's. This is not required by any standard but it is a good thing to have most of the time. Every good mail user agent (MUA) supports threading (Mutt, Gnus, ...) and so do most newsreaders.
Reliable threading is based on message-id's and it is implemented using them also in Web archives of mailingslists and so on.

Solutions

The obvious solution is to generate the Date:-header when preparing the message text. Unfortunately it is very common to make mistakes regarding syntax and semantics of this header. Therefor I don't recommend this except there is a simple way to use a well done Python script for this task.
Update: Pablo Ziliani from Argentina recommended the use of the rfc822() method of the DateTime class. This doesnt solve the Message-Id problem but it is a simple way to generate the Date header. Thanks Pablo!
Here is the call of rfc822() inside the sendmail tag provided by Pablo:


 Date: <dtml-var "_.DateTime().rfc822()"> 

My solution:
For servers I always use qmail as mail transport agent. The author, Daniel J. Bernstein, is of course aware of dumb clients that use SMTP for the delivery of their mails.
To correct problems created by such clients he wrote the programs contained in the mess822-package. I run the contained ofmipd-daemon on port 26 of 127.0.0.1. The mailhost objects and the sendmail tags are pointed to this server. This ensures that headers are correct and a message-id is inserted.

Status of the bug report

One day after filing the bug report I got a response from Andreas (I don't know his last name):


 To: fte@lightwerk.com, support@digicool.com
 CC: support@digicool.com
 From: support@digicool.com
 Subject: Collector #2179: Zope, sendmail tag doesn't create Date: header 
 Date: Mon, 23 Apr 2001 10:02:05 -0400

 Frank Tegtmeyer reported:

 > RFC 822 requires that a dates specification is included in
 > the headers of a message. The sendmail tag doesn't enforce this.
 > Because most programmers do not think about/are not able to
 > produce syntactically and semantically correct Date: headers for
 > mails :) the sendmail tag should do this automatically.
 > It would be nice to be able to choose the timezone and the
 > presentation of the timezone (name/numeric). I for example would
 > prefer UTC with timezone -0000 (what qmail generates).
 >
 > Additionally the generation of a message-id would be nice although
 > it is not required for mail. Many web archivers and mail/news
 > gateways and even MUAs work much better with message-id's in
 > email (for threading for example).


 RFC 822 does not mention that the date and the message-id
 *MUST* be present. So this not really a bug.

 Usually most MTA add a both header fields when they
 are not present.

 Because dtml-sendmail uses the smtplib module from Python
 the generation of these headers should be fixed there.

 Andreas

Unfortunately this answer is completely wrong regarding the Date:-header and the responsibility of the smtplib module.

See my answer:


 Date: Mon, 23 Apr 2001 16:38:31 +0200
 From: Frank Tegtmeyer <fte@lightwerk.com>
 To: support@digicool.com
 Cc: fte@lightwerk.com
 Subject: Re: Collector #2179: Zope, sendmail tag doesn't create Date: header 

 On Mon, Apr 23, 2001 at 10:01:43AM -0400, support@digicool.com wrote:
 > RFC 822 does not mention that the date and the message-id
 > *MUST* be present. So this not really a bug.

 I don't think so. How do you interpret:

      fields      =    dates                      ; Creation time,
                       source                     ;  author id & one
                     1*destination                ;  address required
                      *optional-field             ;  others optional

 It's not only obvious from the syntax that "dates" is required, the
 comment clearly states that it is required.

 > Usually most MTA add a both header fields when they
 > are not present.

 "Most" is the keyword here. See http://cr.yp.to/proto/ofmip.html
 Regarding this terminology the sendmail tag is a ofmip client.

 > Because dtml-sendmail uses the smtplib module from Python
 > the generation of these headers should be fixed there.

 Wrong. The Date: header has nothing to do with SMTP. SMTP is a
 transport mechanism.
 SMTP is meant to be used for well formatted RFC 822 messages. Sloppy
 programming lead to the ofmip situation.
 The smtplib module is clearly not the place to generate missing
 required headers. It's also not an MTA's task. You may check that
 RFC 822 is not even mentioned in the smtplib documentation.

 So I still see it as a bug of the sendmail tag.

 Regards, Frank

Because I still got no answer to that I doubt that DC will do anything about it. If there is time I will have a look at it and send a patch to DC that makes the necessary corrections.
Note: I never found the time and Zope 2.6 now generates the Date header.

Document history

2002-11-06
Martijn Pieters from Zope Corporation told me that Zope 2.6 now generates the Date header. Thanks Martijn!
2002-01-16
added Pablo Zilianis DateTime.rfc822() tip
2001-05-06
status of the bug report added
2001-04-22
first version