[Vmail-discuss] RELAY question(s)

Jose Luis Martin jlmartin@biocora.com
Tue, 23 Jul 2002 00:22:30 +0200


markus wrote:

> Greetings,
>
> does anyone know how to enable pop before smtp to allow costumers to
> send mail trough the mail server?
> Or other ways to let my costumers use my mail server even if they dont
> have static ips/hostnames,
> I saw in tpop3d`s README.POP-before-SMTP:
> "First, a caveat: I do not use POP-before-SMTP relaying, and I think
> that it's a ghastly hack."
> other ideas?
>
> -cheerio Markus
>

 Create  a table in the vmail mysql database named relay: [ip|time]

 then use this table to hold the ips that can relay in the
pop-before-smtp
 script. (at botton)

 In exim.conf use some like:

 host_accept_relay = 127.0.0.1 : otherhost.com : net-mysql;select ip from
relay
 where ip='$key';

You can also do auth for relay:

plain & login for outlook and netscape in exim.conf:
(this is for md5 password only)
######################################################################
#                   AUTHENTICATION CONFIGURATION                     #
######################################################################

 plain:
   driver = plaintext
   public_name = PLAIN
   server_condition = "${if eq{${md5:$3}}{${substr_5:${lookup
mysql{select password_hash from popbox where
local_part='${local_part:$2}' and
domain_name='${domain:$2}'}{$value}}}}{1}{0}}"
   server_set_id = $2

 login:
   driver = plaintext
   public_name = LOGIN
   server_prompts = "Username:: : Password::"
   server_condition = "${if eq{${md5:$2}}{${substr_5:${lookup
mysql{select password_hash from popbox where
local_part='${local_part:$1}' and
domain_name='${domain:$1}'}{$value}}}}{1}{0}}"
   server_set_id = $1


**POP-BeFore-SMTP

** Note that you need to set #!/usr/bin/perl -wT -I/usr/local/perl/vmail/

to your vmail perl directory.
--------------- snip -----------------------
#!/usr/bin/perl -wT -I/usr/local/perl/vmail/
#use strict;

# Check the website <URL:http://people.oven.com/bet/pop-before-smtp/> for

# the latest version, and the mailing list for discussing this program
# and asking for help.

# Changes for Debian:
#
# Use AppConfig instead of Getopt::Long to support a configfile in
addition
# to command line options.
#
# Stig Hackvan <stig@hackvan.com> added support for syslog, set to not
# read entire logfile upon startup...just wipe the relay database and
# look for pop connections.
#
# Switch to using BerkeleyDB instead of DB_File - to make the database
# compatible with postfix on Debian.
#
# Disable flock locking, as it is not recommended (read DB_File 1.75
# documentation!), and I don't know how to get the file descriptor with
# BerkelyDB.

# pop-before-smtp 1.28 Bennett Todd <bet@rahul.net> Freely
Redistributable
# 1.29 2001-12-4  Modified to work whith vmail & use the mysql database
#                 of vmail, store relay ips in relay use, ub exim:
#                 host_acept_relay = mysql;select ip from relay
# 1.28 2001-01-06 Dropped now-bogus ref to perlmod2rpm from spec file
# 1.27 2001-01-06 Added note in README for Outlook Express config, no
#                 send immediately, from Henk Kuipers <henk@bb36.nl>
#                 Added another fix from Alexander Burke
#                 <alex@pdqsolutions.com> for his Red Hat init script,
#                 moved it into contrib, moved generic up as the main
init
#                 script.
#                 Added COPYING, stating the license terms under which
this
#                 software is made available.
# 1.26 2000-11-24 Updated getfromcpan to 1.1 from Alexander Burke
#                 <alex@pdqsolutions.com>
#                 Moved perlmod2rpm into contrib/, required for
#                 getfromcpan and appropriate anyway, reported by
#                 both Alex Burke and also Stig Hackvan.
#                 Detabbed this file.
#                 Refolded the changelog to get lines under 75 cols.
#                 Replaced init script with vastly simpler, cleaner
#                 contrib from alex@pdqsolutions.com, moved my old
#                 bulky one to contrib/redhat-init.orig
#                 Added Stig Hackvan's qmail patches in contrib/qmail.
#                 Added Stig Hackvan's syslog patches in contrib/syslog.
#                 contrib/pop-before-smtp.init-Solaris-7 renamed to
#                 contrib/pop-before-smtp.init-generic, after applying
#                 a minor tweak from Mike Saunders <method@method.cx>
#                 that sufficed to let it be called from rc.local on
#                 FreeBSD.
# 1.25 2000-11-15 Added commentary about DB_File prereq, from
#                 Mike Saunders <method@method.cx>, as well as the
#                 Solaris init script he fixed and tested.
#                 Justin <justin@inwa.com.au> contributed Courier
#                 support.
#                 Added PLATFORM-SPECIFIC NOTES to the top of README
#                 as a quick guide to help point people at bits they
#                 might find interesting, as the number of platform-
#                 and application-specific bits is climbing quick.
#                 Alexander Burke <alex@pdqsolutions.com> contributed the

#                 getfromcpan script, to pkg and install all the needed
#                 prerequisite perl modules in one go.
#                 Added references to the website in the above comment
#                 and DOWNLOAD and AUTHOR sections to the pod.
#                 Added popa3d support contributed by bartek
#                 marcinkiewicz <jr@rzeznia.eu.org>
#                 Separated out the contrib stuff into a contrib/
#                 directory, as the single flat directory was
#                 getting crowded
# 1.24 2000-11-01 Sanitize the environment a little harder
# 1.23 2000-10-30 Fixed GetOptions to match docs and handle --logfile
# 1.22 2000-10-11 Jeremy Shaffner <jer@jorsm.com> pointed out that
#                 several of the example $pats didn't have the needed
#                 terminal semicolons; fixed that.
# 1.21 2000-09-20 Olivier Castan <ocastan@cybercable.fr> reported a bug
#                 in the debugging code, it reported purges it wasn't
#                 acting on because the grace period hadn't expired
# 1.20 2000-09-18 Robert L Mathews pointed out that I'd neglected to
#                 skip the update of the db file if the client was
already
#                 authorized; this update adds that optimization.
#                 And Robert L Mathews also gave another optimization,
#                 hoisting the syncs out of the purge loop and ensuring
#                 that all updates are covered by a single sync.
# 1.19 2000-09-18 Robert L Mathews performed benchmarks demonstrating
that
#                 the performance cost of the flocks is negligible, so
#                 I switched them to default to on.
# 1.18 2000-09-17 added flocks from Robert L Mathews <rob@tigertech.com>
# 1.17 2000-09-17 Brian Duggan <bduggan@oven.com> and Jeff Moore
#                 <jbm@oven.com> worked out a fix for the
#                 sometimes-present nmsgs= at the end of the UW daemons'
#                 log entries.
#                 Kevin Lynn <klynn@santacruz.org> suggested I include
#                 instructions for running as a non-root user.
# 1.16 2000-08-10 William Yodlowsky <wyodlows@andromeda.rutgers.edu>
#                 provided the beautifully trivial patch for Taint
#                 support
#                 Darron Froese provided README.QUICKSTART.
#                 Added pattern for gnu-pop3d, as $pat2, for
#                 coexisting with UW (or some other) imapd;
#                 made $debug default in init script
# 1.15 2000-07-31 Changes requested by Daniel Roesen
#                 <dr@bofh.de>: renamed init script
#                 to a nice lengthy pop-before-smtp; yanked daemon
#                 start/stop on rpm -i/rpm -e; added a
#                 commented-out pattern for cucipop; added documentation
#                 for --nowrite and --debug; fixed init script to clean
#                 up msgs for restart, added status option to init
script,
#                 added logic to remove the pidfile when the daemons is
#                 successfully killed.
#                 Also adopted regex tweak from
#                 wyodlows@andromeda.rutgers.edu so
#                 that Cyrus users can pop from unresolvable ip addrs
#                 (common w/ dialups).
#                 Back to Daniel Roesen, after a request from him,
#                 documented the trick of using a logger daemon to prod
#                 cleanups. Also from him, fixed typo "reset" ->
"restart"
#                 in syntax msg in pop-before-smtp.init
#                 Attempted yet another desperate effort to perhaps
#                 accomodate all the different varients of qpopper in one

#                 $pat
# 1.14 2000-06-19 will the varient qpopper logfile formats never cease?
#                 this one from Nick Bauer <nickb@inc.net>
# 1.13 2000-06-13 yet another qpopper entry, this time from Chris
#                 D.Halverson <cdh@CompleteIS.com>; matches a
#                 logfile format he collected on a Solaris 2.6
#                 system w/ Qpopper 3.0b29.
# 1.12 2000-06-09 added popper entry to match logfile rec from Alex
#                 Burke <alex@pdqsolutions.com>
# 1.11 2000-06-08 added qpopper support thanks to Daniel Meredith
#                 <dman@madcat.investimg.com>
# 1.10 2000-04-05 added regexp for qmail's pop3d thanks to Frank Auciello

#                 <frank@torontowired.com>
# 1.9 2000-03-21 added support for files in mynetworks, tested by Andy
#                Dills
# 1.8 2000-03-21 tweaked UW regexp based on further feedback from Andy
#                Dills
# 1.7 2000-03-20 added regexp for courier-imap
# 1.6 2000-03-13 added installation notes on prerequisites to the readme
# 1.5 2000-02-21 added comment pointing to File::Tail for description of
#                the options with which I'm initializing it
# 1.4 2000-02-21 added comment describing pattern for Cyrus logfile
#                entries, from Kenn Martin <kmartin@infoteam.com>.
# 1.3 2000-02-07 fixed log-watching pattern to correctly recognize imap
#                authentications.  -- Stig Hackvan <stig@hackvan.com>
# 1.2 2000-01-25 added discussion of possible problems with File::Tail
#                using tail => -1, thanks to Andy Dills <andy@xecu.net>.
#                Also thanks to Andy, fixed regexp to work right with
#                clients who don't have reverse DNS set up properly.
# 1.1 2000-01-21 added mention of Time::HiRes as prereq for File::Tail,
#                thanks to Stig Hackvan <stig@hackvan.com>
# 1.0 2000-01-04 first public release

use File::Tail;
# use BerkeleyDB;
use Net::Netmask;
use Date::Parse;
use AppConfig;
use Sys::Syslog;
#use Fcntl ':flock';

# Support for mysql & vmail
use DomainAdminDB;



##################################
#                                #
# Tuneable parameters start here #
#                                #
##################################

# Flags
my $write = 1;
my $flock = 1;
my $debug = 1;

# File to watch for pop3d/imapd records
my $logfile = '/var/log/mail.log';

my $me = getpwuid($<); # real me
# Build complete sanitary environment
# If postconf isn't somewhere on this PATH, do fix the PATH so it is
%ENV = (
    PATH =>
'/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/local/bin',
    HOME => '/tmp',
    SHELL => '/bin/sh',
    LOGNAME => $me,
);

# This regex pull the lines I'm interested in out of $logfile, and yanks
out
# the timestamp and IP address

# For UW ipop3d/imapd, pattern tweaked by Stig Hackvan <stig@hackvan.com>

#my $pat = '^(... .. ..:..:..) \S+ (?:ipop3d|imapd)\[\d+\]: ' .
#          '(?:Login|Authenticated|Auth) user=\S+ host=(?:\S+
)?\[(\d+\.\d+\.\d+\.\d+)\](?: nmsgs=\d+/\d+)?$';

# Bennett Todd to add support for GNU pop3d
my $pat2 = '^(... .. ..:..:..) \S+ gnu-pop3d\[\d+\]: ' .
      'User .* logged in with mailbox .* from (\d+\.\d+\.\d+\.\d+)$';

# There are many, many different logfile formats emitted by various
# qpoppers. Here's an attempt to match any of them, but for all
# I know it might also match failed logins, or something else.
# my $pat = '^(... .. ..:..:..) \S+ q?popper\S+\[\d+\]:
.*\s(\d+.\d+.\d+.\d+)$';

# For Cyrus, Kenn Martin <kmartin@infoteam.com>, with tweak
# from William Yodlowsky for IP addrs that don't resolve:
# my $pat = '^(... .. ..:..:..) \S+ (?:pop3d|imapd)\[\d+\]: ' .
#           'login: \S*\[(\d+\.\d+\.\d+\.\d+)\] \S+ \S+';

# For Courier-IMAP:
#my $pat = '^(... .. ..:..:..) \S+ imaplogin: ' .
#          'LOGIN, user=\S+, ip=\[(\d+\.\d+\.\d+\.\d+)\]$';

# For qmail's pop3d:
#my $pat = '^(... .. ..:..:..) \S+ vpopmail\[\d+\]: ' .
#          'vchkpw: login \[\S+\] from (\d+\.\d+\.\d+\.\d+)$';

# For Qpopper POP/APOP Server
# my $pat = '^(... .. ..:..:..) \S+ (?:qpopper)\[\d+\]: Stats: \S+ ' .
#           '(?:\d+ ){4}(\d+.\d+.\d+.\d+)';

# Alex Burke's popper install
# my $pat = '^(... .. ..:..:..) \S+ popper\[\d+\]: Stats: \S+ ' .
#           '(?:\d+ ){4}(?:\S+ )?(\d+.\d+.\d+.\d+)$';

# Chris D.Halverson's pattern for Qpopper 3.0b29 on Solaris 2.6
# my $pat = '^(\w{3} \w{3} \d{2} \d{2}:\d{2}:\d{2} \d{4}) \[\d+\] ' .
#           ' Stats:\s+\w+ \d \d \d \d [\w\.]+ (\d+\.\d+\.\d+\.\d+)';

# Nick Bauer <nickb@inc.net> has something completely different as
# a qpopper logfile format
# my $pat = '^(... .. ..:..:..) \S+ qpopper\S+\[\d+\]: \([^)]*\) POP
login ' .
#           'by user "[^"]+" at \([^)]+\) (\d+.\d+.\d+.\d+)$';

# For cucipop, matching a sample from Daniel Roesen:
# my $pat = '^(... .. ..:..:..) \S+ cucipop\[\d+\]: \S+ ' .
#           '(\d+\.\d+\.\d+\.\d+) \d+, \d+ \(\d+\), \d+ \(\d+\)';

# For popa3d with the patch from bartek marcinkiewicz <jr@rzeznia.eu.org>

# (available in contrib/popa3d/):
# my $pat = '^(... .. ..:..:..) \S+ popa3d\[\d+\]: Authentication passed
for \S+ -- \[(\d+.\d+.\d+.\d+)\]$';

# For tpop3d <tale@biocora.com>
my $pat = '^(... .. ..:..:..) \S+ tpop3d\[\d+\]: fork_child:
\[\d+\](.*)\((\S+)\): successfully authenticated with mysql';

my $dbfile = '/etc/exim/pop-before-smtp'; # DB hash to write
my $grace = 1800; # 30 minutes --- grace period

# Initialize config
my $config = AppConfig->new();
$config->define("write!", { DEFAULT => $write } );
$config->define("debug!", { DEFAULT => $debug } );
#$config->define("flock!", { DEFAULT => $flock } );
$config->define("logfile=s", { DEFAULT => $logfile } );
$config->define("dbfile=s", { DEFAULT => $dbfile } );
$config->define("grace=i", { DEFAULT => $grace } );
$config->define("pat=s", { DEFAULT => $pat } );
$config->define("pat2=s", { DEFAULT => $pat2 } );

# Read config from file
my $configfile = '/etc/pop-before-smtp/pop-before-smtp.conf';
$config->file($configfile);

# Read config from command line
#$config->getopt() or die "syntax: $0 [--[no]write] [--[no]debug]
[--[no]flock] " .
#    "[--logfile=filename] [--dbfile=filename] [--grace=seconds]\n";
$config->getopt() or die "syntax: $0 [--[no]write] [--[no]debug] " .
    "[--logfile=filename] [--dbfile=filename] [--grace=seconds]\n";

#$config->set("flock", 0) unless $config->write(); # flocking makes no
sense if you're not writing

# These parameters control how closely the watcher tries to follow the
# logfile, which affects how much resources it consumes, and how quickly
# people can smtp after they have popped.
# They are documented in the File::Tail pod; run "perldoc File::Tail" to
# find out details. I guessed at these to try and get this daemon to
follow
# the logfile pretty closely (to avoid users having to wait too long
# after a pop before they can relay) without wasting too much CPU
# needlessly.
my $fi = File::Tail->new(
    name => $config->logfile(),
    maxinterval => 10,
    interval => 5,
    adjustafter => 3,
    tail => 0,
);

# Daniel Roesen prefers this one; he feels that it does a better job
# of being suitably prompt about noticing new logins, and that the
# tuning that I chose is too likely to cause users to fail to be
# able to send email.
#
# my $fi = File::Tail->new(
#         name => $config->logfile(),
#         maxinterval => 2,
#         interval => 1,
#         adjustafter => 3,
#         resetafter => 30,
#         tail => -1,
# );

################################
#                              #
# Tuneable parameters end here #
#                              #
################################

openlog ('pop-before-smtp', 'pid', 'mail');

sub say_goodbye {
  syslog('crit', "exiting on signal %s", $_[0]);
  closelog();
  exit(1);
}
$SIG{'INT'} = sub { say_goodbye('INT'); };
$SIG{'TERM'} = sub { say_goodbye('TERM'); };

$SIG{__DIE__} = sub {
  syslog('crit', "fatal error %s (%m)", $_[0]);
  closelog();
  # perl will perform the exit...
};
syslog('info','starting...');

# Show running configuration
if ($config->debug()) {
    my ($configname, $configvalue);
    my %config = $config->varlist(".*");
    while (($configname, $configvalue) = each (%config)) {
 syslog('debug', "Config: $configname: $configvalue");
    }
}

sub cleanup_nets {
    my @r;
    for (@_) {
        s/^\s+//;
        s/\s+$//;
        s/\s+/ /g;
        s/^mynetworks\s*=\s*//;
        push @r, split /[,\s]+/, $_;
    }
    return @r;
}
# my @mynets = cleanup_nets("1.1.1.1");
#while (my @tmp = grep { $mynets[$_] =~ m#^/# } 0..$#mynets) {
#    for (reverse @tmp) {
#        splice @mynets, $_, 1, cleanup_nets(`cat $mynets[$_]`);
#    }
#}

# Net::Netmask->new($_)->storeNetblock() for @mynets;

my (%t, @q);

#use vars qw(%db);
#unlink $config->dbfile() . ".db";
#my $dbh = tie %db, 'BerkeleyDB::Hash',
#    -Filename => $config->dbfile() . ".db",
#    -Flags => DB_CREATE or
#    die "$0: cannot dbopen " . $config->dbfile() . ": $!\n" if
$config->write(#);
#my $fd = $dbh->fd;
#open(DB_FH, "+<&=$fd") or die "$0: cannot open " . $config->dbfile() . "
filehandle: $!\n" if $config->write();
#flock(DB_FH, LOCK_EX) or die "$0: flock LOCK_EX failed: $!\n" if
$config->flock();
#delete $db{$_} for keys %db;
#flock(DB_FH, LOCK_UN) or die "$0: flock LOCK_UN failed: $!\n" if
$config->flock();

$| = 1 if $config->debug();

my $pattern = $config->pat();
my $pattern2 = $config->pat2();

# mysql support

use DBI;
use DBD::mysql;

$db = new DomainAdminDB();
if(!$db->connect()) {
    Error($db->error());
}

$dbh = $db->dbh();

# borro las ips que caducan en intervalos de una hora mediante un query
# el metodo que usa el script lo dejo, pero no parece funcionar.

$expira  = time() + 60;


# mysql end

while (1) {
    $_ = $fi->read;
    m/$pattern/o or m/$pattern2/o or next;
    my ($timestamp, $ipaddr) = ($1, $3);
    my $ts = str2time($timestamp) or next;
    $ts += $grace;
    next if $ts < time;
    syslog('debug', "read ts=$timestamp ip=$ipaddr") if $config->debug();

    next if findNetblock($ipaddr);
    syslog('debug', "accepted $ipaddr --- not in mynetworks") if
$config->debug();
    push @q, [$ipaddr, $ts];
    my $already_enabled = exists($t{$ipaddr});
    $t{$ipaddr} = $ts;
    next if $already_enabled;
    syslog('info', "opening relay for $ipaddr --- not in mynetworks");
#    flock(DB_FH, LOCK_EX) or die "$0: flock LOCK_EX failed: $!\n" if
$config->flock();
#    $db{$ipaddr} = "ok" if $config->write();
$dbh->do("insert into relay (ip,time) values ('$ipaddr','$ts');") or
Error("could not update popbox record", $dbh->errstr());
    syslog('debug', "written ok") if $config->write() and
$config->debug();
    while ($q[0][1] < time) {
        if ($q[0][1] == $t{$q[0][0]}) {
            syslog('info', "closing relay for $q[0][0]".
                    ($config->debug()?
                    " (ts=".localtime($q[0][1]).")" :
                "")
            );
            delete $t{$q[0][0]};
            $dbh->do("delete from relay where ip='$q[0][0]';");
        }
        shift @q;
    }
#    $dbh->db_sync and die "$0: sync " . $config->dbfile() . ": $!\n" if
$config->write();
#    flock(DB_FH, LOCK_UN) or die "$0: flock LOCK_UN failed: $!\n" if
$config->flock();
    if (time() > $expira) {
 $dbh->do("delete from relay where time < '$expira';");
 $expira = time() + 60;
 syslog('debug',"borrando las ip que han caducado.");
    }
}
---------------------------------- snip
------------------------------------------------------


--------------------------------------------------------------------
Jose Luis Martin
Dpto. Informatica Biocora Consultores S.L.
http://www.biocora.com/linux