Mime-proxy

Version 2.3-5Y, November 10, 2006 Patrick Lamaizière <patrick(dot)softs(at)lamaiziere(dot)net>

WARNING.
    Sending me a correction in good english of this documentation will not be considered an affront!

Purpose.

Mime-proxy is a program to convert characters sets in news articles. It adds a support of characters sets to newsreaders (as Xnews or Gravity) that doesn't support them.

Outgoing articles ( from the client to the server ) are encoded to the appropriate characters set. Incoming articles are decoded to display them. The application is a proxy server.

Features.

    Support characters sets via the library libiconv
    Encode / decode quoted-printable encoding.
    Encode / decode base 64 encoding.
    Encode / decode encoded words (RFC2047).
    Manage multipart MIME articles.
    Support yEnc encoding (I mean : Yencoded articles are not broken by Mime-proxy)
    Multi servers, one configuration for each.
    Possibility to set configuration via article's header.
    Protocols : NNTP, POP, SMTP.
    Operating Systems : Windows ( 98, Me, NT4.0, 2000, XP ) / Unix

Encoding .

For outgoing articles (from the client to the server), Mime-proxy first decodes quoted-printable or base 64 encoding to a 8 bits article. Then it uses the characters set specified in the parameter "client_charset" to translate the article to Unicode.

After, it looks for a suitable characters set for encoding by trying each characters set specified in the parameter "charsets". If a characters set was found, MP uses it to translate the article from Unicode to this characters set.
If it can't find any character sets, the article is rejected.

Mime-proxy can also encode the body to base64 or quoted-printable and headers to encoded-words (RFC2047).

Mime-proxy adds the following headers:
Mime-Version: 1.0
Content-Type: text/plain; charset='used charset'.
Content-Transfer-Encoding: 7bit, 8bit, quoted-printable or base64

Then Mime-proxy sends the article to the server.

Decoding.

Posts with MIME headers are decoded and translated to the characters set used by the newsreader. If MIME headers are missing, Mime-proxy uses the parameter "default_charset" as character set. It also decodes encoded-words and XOVER.

Installation.

Mime-proxy is a stand alone program, it only needs the library libiconv (I provide a DLL for Windows on my web site).
Extract the zip archive to a directory and edit the file "sample_mproxy.ini" (see configuration).

Under Windows, put the DLL iconv.dll into the Mime-proxy's directory.

Uninstall

Delete previous files :-)

Running

From the Windows shell :

mproxy.exe configuration_file [Options]

where :
    configuration_file : a ".ini" like file for settings (see configuration for more)

Options :

-silent : Do not display anything, use the task manager to watch the application activity.
-free_console : Don't use any console, use the task manager to watch the application activity.
-low_memory : Garbage collector management : safe memory regardless to CPU activity.
-default_memory : Default Garbage collector management : somewhere between low_memory and high_memory
-high_memory : Garbage collector management : not safe memory.
-lint   : Display the configuration and exit.
-user username : Username to switch after the installation of the servers; to avoid running MP as root (under Unix only).

example :

mproxy.exe mproxy.ini -low_memory -free_console
# mime-proxy mproxy.ini -user patrick

The simple way to launch the soft is to create a shortcut under Windows.
configuration_file can use a path, If no path is specified, Mime-proxy looks for the file into the current working directory and after into the Mime-proxy's directory.

Configuration.

Mime-proxy is configured via a text file, with a "Windows .ini file" syntax. Use the file "sample_mproxy.ini" as a sample.
You can check the configuration with the "-lint" option, in this case Mime-proxy displays the configuration and exit :

mproxy.exe mproxy.ini -lint

Settings into the sample_mproxy.ini file are for a newsreader that uses windows-1252 characters set and for the fr.* hierachy (encoded-words, 8 bits encoding) .

So for fr.*, just change the settings for the connections in the section [proxies]

Proxies configuration.

This section describes connection parameters for each proxy :

[proxies]
server_alias = local_port_number, remote_host, remote_port_number [,protocol]

Where server_alias is a name for identify this proxy into the configuration file.
local_port_number : the local port to listen.
remote_host : the remote host to connect to.
remote_port_number : the port on the remote host to connect to.
protocol : specify the protocol (nntp, pop, smtp or none). This parameter is optional. Default is nntp.

The "none" protocol does nothing, in this case MP acts as a simple proxy.

Example :

[proxies]
hamster = 6000, localhost, 119
free = 6001, news.free.fr, 119
wanadoo = 6002, news.wanadoo.fr, 119, nntp
smtp_wanadoo = 25, smtp.wanadoo.fr, 25, smtp
pop_wanadoo = 110, pop.wanadoo.fr, 110, pop

The proxy listening on port 6000 connects to the Hamster server, the one listening on port 6001 connects to the Free news server, the one listening on port 6002 connects to the Wanadoo news server. The one listening on port 25 is a smtp proxy. It connects to the Wanadoo smtp mail server. The last one is a pop proxy.

Advanced configuration.

The [default] section provides (default) parameters values for all the proxies. To override some parameters, add a news section named as the server_alias used into the [proxies] section.

Example :

[proxies]
hamster = 6000, localhost, 119
free = 6001, news.free.fr, 119

[default]
encoded_word = 1
charsets = us-ascii, iso-8859-1, iso-8859-15, utf-8

[hamster]
encoded_word = 0

[free]
encoded_word = 1

With this configuration, the proxy uses encoded-words on the "free" server but not on the "hamster" server.

Characters sets.

Mime-proxy handles many charsets (see http://www.gnu.org/software/libiconv ). Mime-proxy has also support for transliteration, i.e. when a character cannot be represented in the target charset, it can be approximated through one or several similarly looking characters. Transliteration is activated when "//TRANSLIT" is appended to the charset. By sample, the euro symbol can be approximated by "EUR" in the iso-8859-1//TRANSLIT charset.

Parameters.

o allow_mp_control_header = 0 | 1

Allow (1) / disallow (0) the use of Mime-proxy's control headers. (see control headers)

o charsets = charset1, charset2, ..., charsetn

The list of charsets to use for encoding. Mime-proxy uses the first suitable charset for the article.

Example :

charsets = us-ascii, iso-8859-1, iso-8859-15, utf-8

o client_charset = charset

The charset used by the client (the newsreader) for outgoing articles. MP translates outgoing articles (from the client to the server) from this charset.

Example :

client_charset = windows-1252

o client_decoding_charset = charset

The charset used by the client (the newsreader) for incoming articles. MP translates incoming articles (from the server to the client) to this charset. If this parameter is not set, Mime-proxy uses the client_charset parameter.

Example :

client_decoding_charset = windows-1252

o client_socket_rcvbuf = n

Buffer size for receives, for the socket connected to the client.

o client_socket_sndbuf = n

Buffer size for sends, for the socket connected to the client.

o client_socket_sndlowat = n
    Send low water mark, for the socket connected to the client (under Unix only)

o date = 0 | 1

If set, Mime-proxy adds a Date: header into outgoing articles. Default values 0.

o decode_header = 0 | 1

Enable (1) / disable (0) the decoding of the headers in incoming articles. Default value : 1.

o decoding = 0 | 1

Enable (1) / disable (0) all decoding features for incoming articles. Default value 1

o default_charset = charset

Default charset to use when charset indication is missing. This parameter applies to the headers and the body of incoming articles. It is also used with the XOVER when the headers are not in an "encoded-word" format.

sample :
    default_charset = iso-8859-15

o enable_incoming_yEncode = 0 | 1

Enable (1) / Enable (0) support for yEnc in incoming articles. If set, incoming yencoded articles are not broken by
Mime-proxy.

o enable_mine_proxy_header = 0 | 1

Enable (1) / disable (0) the "X-Mime-proxy:" header. Default value : 1

o enable_outgoing_yEncode = 0 | 1

Enable (1) / disable (0) support for yEnc in outgoing articles. If set, outgoing yencoded articles are not broken by
Mime-proxy.

o encoded_word = 0 | 1

Enable (1) / disable (0) encoded words (RFC2047) in outgoing articles. Default value 1

o encode_to_b64 = 0 | 1

Enable (1) / disable (0) the base 64 encoding for the body of the outgoing articles. Default value 0.

o encode_to_qp = 0 | 1

Enable (1) / disable (0) the quoted-printable encoding for the body of the outgoing articles. Default value 0

o encoding = 0 | 1

Enable (1) / disable (0) all encoding features for the outgoing articles. Default value : 1

o header_charset = charset

Specify the charset to use with "raw 8 bits headers" encoding. If this parameter is not set, MP uses the "client_charset" parameter instead.
In case of RFC2047 encoding (parameter encoded_word=1), this parameter is not used.

Example :

header_charset = iso-8859-1

o incoming_encoded_word = 0 | 1

Enable (1) / disable (0) the use of encoded words (RFC2047) into incoming articles. Default value 0. If set, Mime-proxy encodes XOVER and incoming articles' headers to encoded-word.

o ip= ip | ip1-ip2 [,ip | ip1-ip2]

Check of the client's IP address : use ip to allow a single ip (as 127.0.0.1) or ip1-ip2 to allow an ip range :
sample :
    ip = 127.0.0.1, 192.168.0.1-192.168.0.16

Default value : 127.0.0.1

o logfile = path_to_a_file or 'syslog'

Mime-proxy can log to a file, or to syslog (only under Unix) with the priority LOG_INFO

example:

[default]

logfile = c:\log.txt

[default]

logfile = syslog

log_facility = LOG_NEWS

o log_facility = facility

Facility to use with syslog.

o max_connections = number

     Limit of the number of connections from a host, per proxy. By default, the value is '0' : no limit.

o message_id =  format_string

MP can generate a message-id field. It uses the format_string, any character '#' followed by two digits will be replaced by a random hexadecimal number with the number of digits.

sample:

    message_id: Xns#13totoplam#02@news.free.fr
    Should generate a Message-ID: field like :
    Message-ID: <Xns0123456789ABCtotoplam01@news.free.fr>

Warning: do no append '<' and '>' in the format string.
Default value : none

o original_ip_header = 0 | 1

If set, MP adds a X-Original-IP: header filled with the address IP of the client. Default value 0

o original_ip_reverse = 0 | 1

If set, the "X-Original-IP:" header is filled with the reverse dns of the address IP of the client. Default value 0

o replace_characters = character_to_search - character_to_put

"search and replace characters" feature for outgoing articles. Characters are specified by the unicode value in hexadecimal.

By sample to replace the character RIGHT SINGLE QUOTATION MARK (Unicode 0x2019) by the character APOSTROPHE (Unicode 0x27) :

    replace_characters = 0x2019 - 0x27

You can make more than one replacement by separate them with a coma :

    replace_characters = 0x2019 - 0x27, 0x.... - 0x ....

Replacements can also be specified with the header 'X-Mp-Replace:'
X-Mp-Replace: 0x2019 - 0x27

o server_socket_rcvbuf = n

Buffer size for receives, for the socket connected to the server.

o server_socket_sndbuf = n

Buffer size for sends, for the socket connected to the server.

o server_socket_sndlowat = n
    Send low water mark, for the socket connected to the server (under Unix only)

o smtp_helo_to_ehlo = 0 | 1

If set, Mime-proxy translates the 'HELO' command (SMTP) by 'EHLO'. This is a hack for Xnews. Default value 0.

o trust_client_charset = 0 | 1

If set, Mime-proxy uses the characters sets of outgoing articles (the charset into the Content-Type: or the charsets into encoded-words) instead of the "client_charset" parameter. Default value 0.

o unknown_character : one_character

The character to use to replace any non recognised character. Default value ?
( sample : unknown_character = % )

o user_agent = 0 | 1

If set, Mime-proxy adds itself to your User-Agent: header (or X-Mailer, X-Newsreader). Defaut value 1

o xover_decoding = 0 | 1

Enable (1) / disable (0) the decoding of the headers in XOVER command. Default value 1

Other parameters.

The [misc] section allows to tune other Mime-proxy's parameters :

o fork_connection = 0 | 1

Under *nix, Mime-proxy can fork each connection.
    fork_connection = 1 (default 0)

If fork() fails, MP rejects the connection with an error "No more ressources available ! Try later"

o max_child = integer

Check the number of forked childs. If the number of childs is greater than this number, the connection is rejected with an error "No more ressources available ! Try later"

    max_child = 25 (default 50)

o max_managed_sockets = integer

Check the number of managed sockets. MP rejects the connection if the number of managed sockets is greater than this parameter with an error "No more ressources available ! Try later".

    max_managed_sockets = 200 (default 128)

MP needs one socket by local server and two sockets by connection to handle the connection between the client and the remote server. If the connections are forked, only local server sockets are counted.

Control headers.

Outgoing article can override some parameters with some "control headers" :

Encode outgoing article ( from client to server )
    X-Mp-Encoding: no/yes

Append to User-Agent:
    X-Mp-User-Agent: no/yes

Use encoded words :
    X-Mp-Encoded-Word: no/yes

Encode to quoted-printable:
    X-Mp-Encode-To-Qp: no/yes

Encode to base 64:
    X-Mp-Encode-To-B64: no/yes

Encoding Charset :
   X-Mp-Charsets: First,Next,...Last charset

Encoding charset for raw 8 bits header.
    X-Mp-Header-Charset: charset

Support for yEnc encoding :
    X-Mp-yEncode: no/yes

Search an replacement feature :
    X-Mp-Replace: 0x2019 - 0x27

Trust client charset:
    X-Mp-Trust-Client: no/yes

Those fields will be removed from the post before sending to the server.

Those headers could be disallowed by setting the option:
    allow_mp_control_header = 0 | 1 (default : 1 )

Language.

You can change application's user messages, see mproxy_en.dat file for a translation in english. Edit this file and rename it in mproxy.dat if you want to use it. Default language is french.

Newsreader settings.

You must change the settings of the connection in your newsreader. The newsreader must connect to Mime-proyy, change the server hostname to the host where Mime-proxy is runing. If Mime-proxy runs on your machine, use 'localhost' as server name. Change the port number to the port that MP is listening on. Keep other parameters as password, login, ...

Thanks.

David.

David Epelbaum.
Piotrek.
Luca.

SmartEiffel : http://SmartEiffel.loria.fr
Elj : http://www.elj.com/elj-win32/
Eposix : http://berend.gameren.nl/eposix/
Gobo : http://elj.sourceforge.net/projects/other/gobo/
Nenie : http://sourceforge.net/projects/nenie/

Ucstring : http://sourceforge.net/projects/ucstring/
Yaesockets : http://sourceforge.net/projects/yaesockets/
Libiconv : http://www.gnu.org/software/libiconv/

Licence (file forum.txt).

The library Libiconv is available under the LGPL license, see COPYING.LIB.

-----------------------------------------------------------------------------------------------
Mime-proxy.

Copyright (c) 2002-2006 Patrick Lamaizière <patrick(dot).softs(at)lamaiziere(dot)net>
-----------------------------------------------------------------------------------------------

Eiffel Forum Freeware License, version 1

Permission is hereby granted, without written agreement and without
license or royalty fees, to use, copy, modify and/or distribute this
package, provided that:

- copyright notices are retained unchanged

- any distribution of this package, whether modified or not,
    includes this file

Permission is hereby also granted, without written agreement and
without license or royalty fees, to distribute binary programs which
depend on this package, provided that:

- if the binary program depends on a modified version of this
    package, you must publicly release the modified version of this
    package - for example by submitting it to the Eiffel Forum archive
    (http://www.eiffel-forum.org/archive/ )

THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT WARRANTY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS BE LIABLE TO ANY PARTY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THIS PACKAGE