Posted at 2009-10-01 11:26 BST by RevK
Update #0: 2009-10-01 11:33 BST

Our RADIUS system tracks the usage on lines every hour and records this
for usage metering that is ultimately used for billing.

The system detects of something has somehow gone wrong whereby the
usage stats do not relate to a known session in the database. If/when
this ever happens then we cannot actually track the usage as we do not
have the reference (of the previous hour). If that happens we kill the
session so that the line reconnects and we have usage stats again.

Obviously this should never happen, and normally it does not.

However, something has messed up in our database this morning. It is
likely it caused both the usage stats error and caused sessions for
which we received RADIUS updates at 11:00 to be unknown and hence
killed a lot of lines.

In an ideal world all lines would have reconnected withing seconds, but
BT still have issues so the impact is much more serious.

So, we have changed the logic now. In future, if a session is not known
we do not kill the session unless it is in the middle of the night.
This means if ever this happens (and it should not) it simply means
people get free un-metered usage until the end of they day and only
then are the lines cleared.

This should help matters which we find the underlying cause.[IMAGE]

URL: http://aaisp.blogspot.com/2009/10/info-radius-issues-change.html

-- 
AAISP Status Blog
URL:http://aaisp.blogspot.com/

Posted at 2009-10-01 11:26 BST by RevK
Update #2: 2009-10-01 14:26 BST

  Our RADIUS system tracks the usage on lines every hour and records this
  for usage metering that is ultimately used for billing.
  
  The system detects if something has somehow gone wrong whereby the
  usage stats do not relate to a known session in the database. If/when
  this ever happens then we cannot actually track the usage as we do not
  have the reference (of the previous hour). If that happens we kill the
  session so that the line reconnects and we have usage stats again.
  
  Obviously this should never happen, and normally it does not.
  
  However, something has messed up in our database this morning. It is
  likely it caused both the usage stats error and caused sessions for
  which we received RADIUS updates at 11:00 to be unknown and hence
  killed a lot of lines.
  
| In an ideal world all lines would have reconnected within seconds, but
  BT still have issues so the impact is much more serious.
  
  So, we have changed the logic now. In future, if a session is not known
  we do not kill the session unless it is in the middle of the night.
  This means if ever this happens (and it should not) it simply means
  people get free un-metered usage until the end of they day and only
  then are the lines cleared.
  
| This should help matters while we find the underlying cause.[IMAGE]

URL: http://aaisp.blogspot.com/2009/10/info-radius-issues-change.html

-- 
AAISP Status Blog
URL:http://aaisp.blogspot.com/