Index About us
Index Setup & configLogs & bandwidth usage SPAMDownloadDevelopers  

Bandwidth: Getting numbers.

Selecting a logtype

You can extract a lot of mail bandwidth usage info with Moxy, here's how.

You first need to decide what type of information you wish to extract and configure Moxy to log appropriately. You can have a look at the Setup and config section for details but it boils down to 3 options:

  • Use syslog or filelog logging if you're only interested in keeping track of general system use. You can still extract lots of info (using, see below) from these logs and they behave in a manner similar to regular MTA logging.
  • Use gdbmlog logs to have up to the minute bandwidth usage statistics on a per destination basis (using moxystats, see below).
  • Get the best of both worlds, using sys_and_gdbmlog which logs to both a gdbm file and the system's syslog facility.

After you've set your selection using the config file's logtype attribute, start Moxy and send a few emails

Extracting the data

When it comes time to retrieve meaningful data from the Moxy logs, the method used will depend on the logtype you are using.

Syslog and filelog

The syslog and filelog log types log information in plaintext, through the system syslog facility (in which case the data will go wherever syslog.conf is set to send mail.*, usually /var/log/maillog) or to a regular text file, respectively.

You may open these files in any editor and peruse them to your hearts content. When you want some hard numbers from these logs, use moxylogalyze. moxylogalyze is a simple Perl script that can parse Moxy logs and calculate total bandwidth usage based on a number of criteria. Here is the output generated for --help:


	  	LOGFILE	The log file to parse
		-l,--listby P	List all totals, sorted by P (from, ip or dest).
		-v,--verbose	Verbose output to STDERR
		-V,--version	Version info
		-h,--help		This message.
		--from EMAIL		from this sender.
		-i,--ip IP			from this IP addr.
		-d,--dest HOST:PORT	mail proxied to this host and port.
		--for EMAIL			for this user.

		Multiple MATCHES may be used (they will be ANDed together). 
		Note that you can use regular expressions in place of string literals for EMAIL,
		Examples: --dest /path/to/logfile --listby from /var/log/maillog -v --dest ':\d+25' --for 'pat(rick)?' /var/log/maillog


As you can see, you can get data for many many different scenarios with moxylogalyze. It's important to understand the exact meaning of the 4 MATCHES arguments.

  • --from matches the sender's from email address as it was stated during the MAIL FROM: phase of the SMTP transaction. E.g.
  • --ip matches the sender's source Internet Protocol (IP) address at the time the message was sent. E.g.
  • --dest is the host and port combination that this email was proxied to. If for instance an email comes in for two domains - and - a both theses domains are set to be proxied to port 1025, then a single connection will be made and this will be reflected by a single entry in the logs indicating a connection to with email for the users ','.
  • --for matches a recipient's email address. The log entry in the example used above would match both --for and --for even though only a single connection was used (and thus, the data was sent only once).
The three example uses from the --help output are pretty self explanatory but here's a little more detailed description: --dest /path/to/logfile
    This outputs the number of bytes transfered from any source to any destination containing the sequence of letters ''. Remember that the destination may have no relation to the recipient email addresses - it is simply the machine host:port to which this email was proxied.
 --listby from /var/log/maillog
    The --listby option allows you to output all the totals found in the log file. These may be calculated and sorted by one of from, ip or dest, as defined above. In this example, the number of bytes sent to domains Moxy is responsible for will be displayed on a per sender email address basis.
 -v --dest ':\d+25' --for 'pat(rick)?' /var/log/maillog
    This is a search that shows that you can combine MATCH requirements - these are logically ANDed - and use Perl regular expressions in you arguments (the literal strings used in the examples to date are but a degenerate type of regex). This example matches all log entries for mail proxied to a machine on anyport which contains at least one digit followed by the sequence '25' (eg 12325 but not 25) and destined for at least one user with an email address that contains 'pat' or 'patrick' (yeah, the (rick)? can be eliminated without affecting the number of matches but I'm just trying to demo the use of regular expressions ;) ). assumes you are using the default log_format (see the /usr/local/moxy/moxy.conf file for details). If you've modified the configuration, you can still use by modifying a single line in the Perl script. Change the line:

my $Regexp = q|\s*(\d+)\s+bytes\s+from\s+<([^>]+)>\s+\[([^\]]+)\]\s+to\s+\[([^\]]+)\]\s+for\s+'([^']+)'|; 

so that the regex matches your output format. You may also need to change the ordering of the

my ($size, $lfrom,  $lip,  $ldest, $lfor) = ($1, $2, $3, $4, $5);

lines if you've changed the order of the size, from, ip, dest and for in the log output format.


The gdbmlog log types keeps data in, oddly enough, a gdbm (GNU database) file. The disadvantage of using gdbmlog is that log files are no longer human readable. The advantage, however, is having rapidly accessible and up to the minute bandwidth usage statistics on a per destination basis. If having a running total of the number of bytes sent to each destination host:port combination is what you're intersted in, gdbmlog is the way to go but how do you extract information from this database?

The moxystats utility may be used to get current bandwidth usage for a given host:port combination when using gdbmlog. The output of moxystats --help is :

Usage: moxystats -f INFILE [DOMAIN:PORT[S]]
   Get bandwidth statistics from moxy gdbm log file INFILE.
      -f,--file INFILE	Specify the moxy gdbm log file to open.
      -a,--all			Dump entire contents of gdbm file.
      -v,--verbose		Print extra info (dates). 
      -V,--version		Output version info.
      -h, --help		This message.

Thus, to extract bandwidth usage for mail proxied to on port 25 and to on port 1234 you would use:
moxystats -f /var/log/moxy.log
and to output statistics on all host:port combinations present in the gdbm file, use
moxystats -f /var/log/moxy.log --all


If you choose to use the sys_and_gdbm_log log type, which logs to both syslog and a gdbm db, you can use either or moxystats as described above.

© 2001-2002 Psychogenic inc.
SourceForge Logo