Monitoring Bandwidth Usage for a Xen node

After my last post on limiting bandwidth usage on a Xen node I thought I would follow up with how to monitor the bandwidth usage of a Xen node.

I chose to use netacct-mysql to monitor bandwidth and store the results into a mysql database. This isn’t the only way of monitoring the bandwidth but it was easy to set up and stores the information directly into mysql without needing any extra scripts.

On the Xen setup I’m using for an example there are 3 nodes and a bucket for each node will be created in the mysql database by netacct. Setting up netacct can be tricky so you will want to check to make sure traffic for each node is going into the correct bucket.

Your naccttab file will look something like:

niff 0
database mysql
mysql_user acct
mysql_password acct_password
mysql_host localhost
mysql_port 0
mysql_database netacct
pidfile /var/run/nacctd.pid

compactnet 192.168.1.1 255.255.255.255
compactnet 192.168.1.2 255.255.255.255
compactnet 192.168.1.3 255.255.255.255

ournet 192.168.1.1 255.255.255.255
ournet 192.168.1.2 255.255.255.255
ournet 192.168.1.3 255.255.255.255

flush 30

errdelay 2
fdelay 60

device peth0

ignorenet 127.0.0.0 255.0.0.0

debug 0
debugfile /tmp/nacctd.debuz

A couple notes on setting up netacct:

1. nacctd runs on Xen0 so it can capture all the traffic from the XenU nodes.
2. The naccttab config file needs to go into /usr/local/etc in case it isn’t obvious from the README.
3. You need to make sure you have the correct Xen0 device set up as the listen device in the naccttab config file.
4. If the mysql database is on the Xen0 node then you probably don’t need a username or password in the config file.

After running netacct for some amount of time you can query the database it is logging to and get a count of all the different nodes and their bandwidth usage. Try this query out:

select traffic.ip, sum(input)/1073741824 inp, sum(output)/1073741824 outp from traffic group by ip;

That will show a total sum of bandwidth in gigabytes broken down by ip.

To get just the current month you would want this query:

select traffic.ip, sum(input)/1073741824 inp, sum(output)/1073741824 outp from traffic where time between concat(EXTRACT(YEAR_MONTH FROM now()), ’01’) and now() group by ip;

To display text with totals if fairly easy since all we need to do is run the above queries and then display the output. Here is an example of doing just that with PHP:

<?php
$query_ip = ‘192.168.1.2’;

mysql_connect( ‘192.168.1.1’, ‘user’, ‘pass’ ) or die ( ‘Unable to connect to server.’ );
mysql_select_db( ‘netacct’ ) or die ( ‘Unable to select database.’ );

$sql = “select sum(input) inp, sum(output) outp from traffic where ip = ‘$query_ip’ and time between concat(EXTRACT(YEAR_MONTH FROM now()), ’01’) and now()”;
$result = mysql_query( $sql ) or die ( ‘Unable to execute query.’ );
$row = mysql_fetch_row($result);

$current_inpgigs = round($row[0]/1073741824, 3);
$current_outgigs = round($row[1]/1073741824, 3);
$current_total = round(($row[1]+$row[0])/1073741824, 3);

$sql = “select sum(input) inp, sum(output) outp from traffic where ip = ‘$query_ip’ and time between concat(EXTRACT(YEAR_MONTH FROM date_sub(now(), interval 1 month)), ’01’) and concat(EXTRACT(YEAR_MONTH FROM now()), ’01’)”;
$result = mysql_query( $sql ) or die ( ‘Unable to execute query.’ );
$row = mysql_fetch_row($result);

$last_inpgigs = round($row[0]/1073741824, 3);
$last_outgigs = round($row[1]/1073741824, 3);
$last_total = round(($row[1]+$row[0])/1073741824, 3);

echo “Current: $current_inpgigs GB, $current_outgigs GB, $current_total GB\n”;
echo “Last: $last_inpgigs GB, $last_outgigs GB, $last_total GB\n”;
?>

You will want to put a command in cron to run the above script every 5 minutes to update the database.

The following example PHP script will display a graph of an IP generated from the given database:

<?php
$query_ip = ‘192.168.1.2’;

$opts = array( “–start”, “-1d”, “–vertical-label=B/s”,
“DEF:inoctets=/tmp/rrd/$query_ip.rrd:input:AVERAGE”,
“DEF:outoctets=/tmp/rrd/$query_ip.rrd:output:AVERAGE”,
“AREA:inoctets#00FF00:In traffic”,
“LINE1:outoctets#0000FF:Out traffic\\r”,
“CDEF:inbits=inoctets,8,*”,
“CDEF:outbits=outoctets,8,*”,
“COMMENT:\\n”,
“GPRINT:inbits:AVERAGE:Avg In traffic\: %6.2lf %Sbps”,
“COMMENT: “,
“GPRINT:inbits:MAX:Max In traffic\: %6.2lf %Sbps\\r”,
“GPRINT:outbits:AVERAGE:Avg Out traffic\: %6.2lf %Sbps”,
“COMMENT: “,
“GPRINT:outbits:MAX:Max Out traffic\: %6.2lf %Sbps\\r”
);

$ret = rrd_graph(“/tmp/images/${query_ip}_1d.gif”, $opts, count($opts));

if( !is_array($ret) )
{
$err = rrd_error();
echo “rrd_graph() ERROR: $err\n”;
}
?>