EMT
From Proven Scaling Wiki
Contents |
EMT is a system to gather, aggregate, and store information collected from existing status tools. EMT normally runs from cron and executes plugins. Plugins define commands to execute and fields to be parsed from the output of those commands. EMT then takes the parsed output and passes it to output handlers which can store the data locally, or send it off to centralized monitoring servers for aggregation and graphing.
Plugins
Plugins are divided into two sections, fields, and commands. Commands are executed once per instance. Command instances are for running the same command for multiple different named instances. For example with two instances of mysqld running on a single machine emt can gather the same set of stats for each instance and track them separately. For more information on instances see the configuration section.
For an example of how plugins are created let's take a look at the mysql plugin. The mysql plugin consists of five parts, the command object for execution, the command object for output, the command being executed that connects to mysql and the field objects that define the data being collected.
Let's start with the defining the command object. emt/plugins/commands/emt_mysql.php contains the source code for the two classes that handle the emt_mysql command. class emt_mysql is responsible for defining the shell command that emt will execute. The emt_mysql_parse class contains the parsing logic for taking the output of the emt_mysql shell command and creating field objects out of it. The emt_mysql command can be replaced by any other cli program for you own plugins. Configration parameters from the .cnf file for your plugin are parsed and set in the config object.
class emt_mysql extends gather_general
{
function command()
{
$this->cmd = INSTALL_PATH . 'bin/emt_mysql ' .
"-t " . $this->gather_time . " " .
"-h " . $this->config['host'] . " " .
"-u " . $this->config['user'] . " " .
"-p " . $this->config['password'];
}
}
class emt_mysql_parse extends field
{
function parse($str)
{
$fields = preg_split('/,\s*/', $str);
t_used
foreach ($fields as $field)
{
list($key, $value) = split('=', $field);
if ($key == $this->name)
return parent::parse($value);
}
}
}
This example class parses the emt_mysql program output (which is already csv) into values. Calling parent::parse on a value will tell the parent field class to create an emt object field out of the value.
After the command and parse classes have been defined it's time to create some field objects. Field objects define the structure of a single value gathered by emt. Looking at emt/plugins/fields/mysql.php there are several field classes defined for the emt_mysql plugin. Each value gathered by emt needs a field class associated with it. This gives information about decimal precision, namespace, and description which are used by emt_view and will eventually used by an associated graphing application.
IMPORATION! Do not forget to register your fields with the register_field function. This adds the field name to a global array that emt uses to know which fields have been created. Emt will not gather data for a field if the field is not registered.
The last step is to create a configuration file for your plugin.
Configuration
Overview
Configuration of emt is very similar to mysql and other ini based configuration applications with one small extention for instances which I'll cover later. By default configuration files are loaded in two parts. The first part is the main emt configuration file located at /etc/emt.cnf This file contains options for the base emt programs such as emt_gather and emt_view. Plugin configuration files are loaded after the main configuration and can be placed in /etc/emt.d/
Plugin Specific
When creating a plugin an appropriately named cnf file should be placed in /etc/emt.d/ These files need to hold the fields to be gathered under an [emt_gather] heading as well as any plugin specific values under [plugin_name] heading. For example the emt_mysql.cnf file looks like:
[emt_mysql] host=:/home/ebergen/mysql/mysql-5.0.51a/mysql.sock user=root password= [emt_gather] field=mysql_bytes_received field=mysql_bytes_sent field=mysql_com_commit field=mysql_com_delete field=mysql_com_insert field=mysql_com_rollback field=mysql_com_select field=mysql_com_update field=mysql_open_tables field=mysql_opened_tables field=mysql_threads_connected
The [emt_mysql] section values are passed into the config property of the command object for emt_mysql. The [emt_gather] section needs to be a part of every plugin configuration file because it specifies the active fields to gather. I will likely be changing this before the first release as it's duplicated in the plugin php code and doesn't support instances well.
Instances
Emt can gather data for several instances of a program on the same host. These named instances are defined via the configuration files as part of the section heading. For example to gather data from two running instances of mysql on the same machine you define two emt_mysql headings with colon and the instance name.
[emt_mysql:foo] host=:/home/ebergen/mysql/mysql-5.0.51a/mysql_foo.sock user=foo-root password= [emt_mysql:bar] host=:/home/ebergen/mysql/mysql-5.0.51a/mysql_bar.sock user=bar_root password=
For each instance defined in the configuration emt will execute a command with the specific config object for that instance. When the output handler is called the instance name is set in the field object so output handlers know which instance they're working with.
