Statistics Reporting and Collection

Introduction

Mantissa includes a way to report simple string data to external processes roughly in real time. This system is used internally to expose certain statistics which may be relevant for performance and optimization investigations. Applications can contribute similar statistics to be reported in the same way. Custom observers for this information can also be written.

This document will explain how to report such information. It will also explain how to write a client which can be connected to a Mantissa server to receive this information.

Readers should familiarize themselves with the following concepts in order to understand all sections of this document:

  • Zope Interfaces
  • Twisted Logging
  • Twisted TCP clients
  • AMP Authentication
  • AMP Routes
  • Axiom Powerups
  • Mantissa Port Configuration

Reporting Events

twisted.python.log.msg is used to report events which Mantissa will publish to clients. The only requirement is that the "interface" key have a value of axiom.iaxiom.IStatEvent . For example, a blogging application might want to report each new blog post. To do this, in the blog post handling code, this code would be included:

from twisted.python.log import msg
from axiom.iaxiom import IStatEvent

msg(interface=IStatEvent, new_blog_post=1)

It is left up to observers how this data is interpreted. Almost any keys or values are allowed. The primary restriction on keys is that they are not already used by the logging system itself. For example, "message" is reserved for the value of the first positional argument to twisted.python.log.msg , so a stat event cannot use it. Values are restricted to objects which can be represented as unicode strings.

Stat Observers

Server Configuration

Observation of reported statistics is done using an AMP client. The client connects to and authenticates with a Mantissa server. The server must have an AMP port configured and the user must have the xmantissa.stats.RemoteStatsCollectorFactory powerup installed in order to observe stats. The Mantissa administrative offering (not installed by default) provides this powerup.

Stats Route and StatUpdate Command

A stats route is made accessible via the u"http://divmod.org/ns/mantissa-stats" protocol. Once the route is set up, the server will send StatUpdate commands to the client until it disconnects. The StatUpdate command has one parameter, data . The value for the parameter is an encoded list of AMP boxes. Each box has two keys, key and value , each of which is associated with one part of keyword argument to the corresponding msg call. Additionally, there is an element including a timestamp for the event. For example, the msg call in the blog post example above would results in [{u'key': u'time', u'value': u'1224183617.916481'}, {u'key': u'new_blog_post', u'value': u'1'}] . The ordering of the list is arbitrary.