Re: flexfax: Problems with german umlaute

[David Woolley <david@djwhome.demon.co.uk>]

Note I haven't bothered to preserve 8 bit characters here; they are in
MIME QP format.

N.B. Although both mail examples are broken, there is also a bug in
Hylafax (char assumed unsigned).

> X-Mailer: Mozilla 3.01Gold (Win95; I)
> MIME-Version: 1.0
> To: ndista
> Subject: test with =E4=F6=FC=C4=D6=DC                <<<<<<<<<<< six umla=

The above line is illegal as it contains non-ASCII characters.  Headers must
be ASCII as you need the headers to be processed before you can declare the
choice of character set and how 8 bit characters are being processed.

> ute without
> linefeeds
> Content-Type: text/plain; charset=3Diso-8859-1
> Content-Transfer-Encoding: 8bit

This is good for the content, and your mail gateway actually converted your
message to quoted printable based on this sort of information.

> 
> -- fax output --
> Subject: test with                      <<<<<<< why is here a linefeed?
>          =E4=F6=FC=C4                           <<<<<<< and here?
>          =D6=DC
> 
> test with                               <<<<<<< it's a kind of magic
> =E4                                       <<<<<<<=20
> =F6                                       <<<<<<<
> =FC=C4=D6=DC                                    <<<<<<<
> 

I think your problem is in the following:

djwhome:/usr/src/hylafax-v4.0pl1/util$ grep -n charwidth TextFmt.h
62:    TextCoord charwidth(const char c) const;
68:inline TextCoord TextFont::charwidth(const char c) const { return widths[c];
}
djwhome:/usr/src/hylafax-v4.0pl1/util$

On a machine that has signed chars, the array index will index before the
start of the array for non-ASCII characters.  That means you will get bogus
widths for the characters and it will think the line has overflowed. I'd
guess that unsigned char would work, and int might also be OK.

You didn't say what machine you were using though!

> When sending a faxmail from Lotus Notes, it gets even worse. See the
> following:
> 
> -- ugly header start --
> Subject: test with =3D?ISO-8859-1?Q?=3DE4=3DF6=3DFC=3DC4=3DD6=3DDC?=3D   =
> <<<<<< looks
> strange

But it is perfectly correct.

> 
> --Boundary_(ID_lzvW7tb5QgZ1xzKVc5pyIQ)
> Content-type: TEXT/PLAIN; CHARSET=3DISO-8859-1
> 

That was illegal.  You've declared an 8 bit character set, but are
only sending 7 bits.  Content-Transfer-Encoding is mandatory for 
8 bit character sets.

> Subject: test with =3D?ISO-8859-1?Q?=3DE4=3DF6=3DFC=3DC4=3DD6=3DDC?=3D  <=
> <<<<< ISO
> strings not interpreted?

There doesn't seem to be any support for MIME encoding of headers.  I don't
think this is part of the compliance requirements for basic MIME, although
it would probably be desirable.

> 
> test with dv|DV\                                        <<<<<<
> completely out of bounds

The correct action here was for the mail transport software to bounce
the message.  With no MIME headers, the action on out of range characters
is undefined (typically partly local policy).  However, I believe that
out of bounds characters with MIME headers are considered a clear protocol
violation and the message should be rejected.  You have an implied
Content-Transfer-Encoding: 7bit.

In fact Hylafax does this when it has a part which is in C-T-E: 7bit:

c &= 0x7f;

I.E. it strips the top bit.