


 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



 NNNNAAAAMMMMEEEE
      rrd-beginners - Beginners guide

 SSSSYYYYNNNNOOOOPPPPSSSSIIIISSSS
      Helping new RRDTool users to understand the basics of RRDTool

 DDDDEEEESSSSCCCCRRRRIIIIPPPPTTTTIIIIOOOONNNN
      This manual is an attempt to assist beginners in understanding the
      concepts of RRDTool. It sheds a light on differences between RRDTool
      and other databases. With help of an example, it explains structure of
      RRDTool database. This is followed by an overview of the "graph"
      feature of RRDTool.  At the end, it has sample scripts that
      illustrates the usage/wrapping of RRDTool within Shell/Perl.

      WWWWhhhhaaaatttt mmmmaaaakkkkeeeessss RRRRRRRRDDDDTTTToooooooollll ssssoooo ssssppppeeeecccciiiiaaaallll????

      RDtool is GNU licensed software developed by Tobies Oetiker, a system
      manager at the Swiss Federal Institute of Technology. Though it is a
      database, there are distinct differences between RRDtool database and
      other databases as listed below:

      o+   RRDTool stores data; that makes it a back end tool. The RRDtool
          command set allows the creation of graphs; that makes it a front
          end tool as well. Other databases just stores data and can not
          create graphs.

      o+   In case of linear databases, new data gets appended at the bottom
          of the database table. Thus its size keeps on increasing, whereas
          size of an RRDtool database is determined at creation time.
          Imagine an RRDtool database as the perimeter of a circle. Data is
          added along the perimeter. When new data reaches the starting
          point, it overwrites existing data. This way, the size of an
          RRDtool database always remains constant. The name "Round Robin"
          stems from this attribute.

      o+   Other databases store the values as supplied. RRDtool can be
          configured to calculate the rate of change from the previous to
          the current value and store this information instead.

      o+   Other databases get updated when values are supplied. The RRDTool
          database is structured in such a way that it needs data at
          predefined time intervals. If it does not get a new value during
          the interval, it stores an UNKNOWN value for that interval. So,
          when using the RRDTool database, it is imperative to use scripts
          that runs at regular intervals to ensure a constant data flow to
          update the RRDTool database.

      RRDtool has a lot to do with time. With every data update, it also
      needs to know the time when that update occurred. Time is always
      expressed in seconds passed since epoch (01-01-1971). RRDTool can be



                                    - 1 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



      installed on Unix as well as Windows. It has command set to carry out
      various operations on rrd database. This command set can be accessed
      from the command line, and from shell or perl scripts. The scripts act
      as wrappers for accessing data stored in RRDtool database.

      UUUUnnnnddddeeeerrrrssssttttaaaannnnddddiiiinnnngggg bbbbyyyy aaaannnn eeeexxxxaaaammmmpppplllleeee

      The structure of an RRD database is different than other linear
      databases.  Other databases define tables with columns, and many other
      parameters. These definitions sometime are very complex, especially in
      large databases.  RRDTool databases are primarily used for monitoring
      purposes and hence are very simple in structure. The parameters that
      need to be defined are variables that hold values and archives of
      those values. Being time sensitive, a couple of time related
      parameters are also defined. Because of its structure, the definition
      of an RRDTool database also includes a provision to specify specific
      actions to take in the absence of update values. Data Source (DS),
      heartbeat, Date Source Type (DST), Round Robin Archive (RRA), and
      Consolidation Function (CF) are some of the terminologies related to
      RRDTool databases.

      The structure of a database and the terminology associated with it can
      be best explained with an example.

       rrdtool create target.rrd
               --start 1023654125
               --step 300
               DS:mem:GAUGE:600:0:671744
               RRA:AVERAGE:0.5:12:24
               RRA:AVERAGE:0.5:288:31

      This example creates a database named _t_a_r_g_e_t._r_r_d. Start time
      (1023654125) is specified in total number of seconds since epoch (time
      in seconds since 01-01-1970). While updating the database, update time
      is also specified.  This update time MUST occur after start time and
      MUST be in seconds since epoch.

      The step of 300 seconds indicates that database expects new values
      every 300 seconds. The wrapper script should be scheduled to run every
      sssstttteeeepppp seconds so that it updates the database every sssstttteeeepppp seconds.

      DS (Data Source) is the actual variable which relates to the parameter
      on the device that has to be monitored. Its syntax is

       DS:variable_name:DST:heartbeat:min:max

      DDDDSSSS is a key word. "variable_name" is a name under which the parameter
      is saved in database. There can be as many DSs in a database as
      needed. After every step interval, a new value of DS is supplied to
      update the database.  This value is also called as Primary Data Point



                                    - 2 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



      ((((PPPPDDDDPPPP)))). In our example mentioned above, a new PDP is generated every
      300 seconds.

      Note, that if you do NOT supply new datapoints exaclty every 300
      seconds, this is not problem, RRDTool will interpolate the data
      accordingly.

      DDDDSSSSTTTT (Data Source Type) defines type of DS. It can be COUNTER, DERIVE,
      ABSOLUTE, GAUGE. A DS declared as COUNTER will save the rate of change
      of the value over a step period. This assumes that the value is always
      increasing (difference between last two values is more than 0).
      Traffic counters on a router is an ideal candidate for using COUNTER
      as DST. DERIVE is same as COUNTER but it allows negative values as
      well. If you want to see the rate of _c_h_a_n_g_e in free diskspace on your
      server, then you might want to use the DERIVE data type. ABSOULTE also
      saves the rate of change but it assumes that previous value is set to
      0. The difference between current and previous value is always equal
      to the current value. So, it stores the current value divided by step
      interval (300 seconds in our example). GAUGE does not save the rate of
      change. It saves the actual value itself. There are no
      divisions/calculations. Memory consumption in a server is an ideal
      example of gauge. Difference among different types DSTs can be
      explained better with following example:

       Values       = 300, 600, 900, 1200
       Step         = 300 seconds
       COUNTER DS   =    1,  1,   1,    1
       DERIVE DS    =    1,  1,   1,    1
       ABSOLUTE DS  =    1,  2,   3,    4
       GAUGE DS     = 300, 600, 900, 1200

      The next parameter is hhhheeeeaaaarrrrttttbbbbeeeeaaaatttt. In our example, heartbeat is 600
      seconds. If database doesn not get a new PDP within 300 seconds, it
      will wait for another 300 seconds (total 600 seconds).  If it doesnt
      receive any PDP with in 600 seconds, it will save an UNKNOWN value
      into database. This UNKOWN value is a special feature of RRDTool - it
      is much better than to assume a missing value was 0 (zero).  For
      example, the traffic flow counter on a router keeps on increasing.
      Lets say, a value is missed for an interval and 0 is stored instead of
      UNKNOWN. Now when next value becomes available, it will calculate
      difference between current value and previous value (0) which is not
      correct. So, inserting value UNKNOWN makes much more sense here.

      The next two parameters are the minimum and maximum value
      respectively. If variable to be stored has predictable maximum and
      minimum value, this should be specified here. Any update value falling
      out of this range will be saved as UNKNOWN.

      The next line declares a round robin archive (RRA). The syntax for
      declaring an RRA is



                                    - 3 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



       RRA:CF:xff:step:rows

      RRA is the keyword to declare RRAs. The consolidation function (CF)
      can be AVERAGE, MINIMUM, MAXIMUM, and LAST. The concept of the
      consolidated data point (CDP) comes into the picture here. A CDP is
      CFed (averaged, maximum/minimum value or last value) from _s_t_e_p number
      of PDPs. This RRA will hold _r_o_w_s CDPs.

      Lets have a look at the example above. For the first RRA, 12 (steps)
      PDPs (DS variables) are AVERAGEed (CF) to form one CDP. 24 (rows) of
      theses CDPs are archived. Each PDP occurs at 300 seconds. 12 PDPs
      represent 12 times 300 seconds which is 1 hour. It means 1 CDP (which
      is equal to 12 PDPs) represents data worth 1 hour. 24 such CDPs
      represent 1 day (1 hour times 24 CDPs). It means, this RRA is an
      archive for one day. After 24 CDPs, CDP number 25 will replace the 1st
      CDP. Second RRA saves 31 CDPs; each CPD represents an AVERAGE value
      for a day (288 PDPs, each covering 300 seconds = 24 hours). Therefore
      this RRA is an archive for one month. A single database can have many
      RRAs. If there are multiple DSs, each individual RRA will save data
      for all the DSes in the database. For example, if a database has 3
      DSs; and daily, weekly, monthly, and yearly RRAs are declared, then
      each RRA will hold data from all 3 data sources.

      GGGGrrrraaaapppphhhhiiiiccccaaaallll MMMMaaaaggggiiiicccc

      Another important feature of RRDTool is its ability to create graphs.
      The "graph" command uses "fetch" command internally to retrieve values
      from the database. With the retrieved values, it draws graphs as
      defined by the parameters supplied on the command line. A single graph
      can show different DS (Data Sources0) from a database. It is also
      possible to show the values from more than one databases into a single
      graph. Often, it is necessary to perform some math on the values
      retrieved from database, before plotting them. For example, in SNMP
      replies, memory consumption values are usually specified in KBytes and
      traffic flow on interfaces is specified in Bytes. Graphs for these
      values will be more senseful if values are represented in MBytes and
      mbps. the RRDTool graph command allows to define such conversions.
      Apart from mathematical calculations, it is also possible to peform
      logical operations such as greater than, less than, and if then else.
      If a database contains more than one RRA archive, then a question may
      arise - how does RRDTool decide which RRA archive to use for
      retrieving the values? RRDtool takes looks at several things when
      making its choice. First it makes sure that the RRA covers as much of
      the graphing time frame as possible. Second it looks at the resolution
      of the RRA compared to the resolution of the graph. It tries to find
      one which has the same or better resolution. With the "-r" option you
      can force RRDtool to assume a different resolution than the one
      calculated from the pixel width of the graph.

      Values of different variables can be presented in 5 different shapes



                                    - 4 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



      in a graph - AREA, LINE1, LINE2, LINE3, and STACK. AREA is represented
      by a solid colored area with values as the boundary of this area.
      LINE1/2/3 (increasing width) are just plain lines representing the
      values. STACK is also an area but it is "stack"ed on AREA or
      LINE1/2/3. Another important thing to note, is that variables are
      plotted in the order they are defined in graph command. So, care must
      be taken to define STACK only after defining AREA/LINE. It is also
      possible to put formatted comments within the graph.  Detailed
      instructions be found under graph manual.

      WWWWrrrraaaappppppppiiiinnnngggg RRRRRRRRDDDDTTTToooooooollll wwwwiiiitttthhhhiiiinnnn SSSShhhheeeellllllll////PPPPeeeerrrrllll ssssccccrrrriiiipppptttt

      After understanding RRDTool, it is now a time to actually use RRDTool
      in scripts. Tasks involved in network management are data collection,
      data storage, and data retrieval. In the following example, the
      previously created target.rrd database is used. Data collection and
      data storage is done using Shell scrip. Data retrieval and report
      generation is done using Perl script. These scripts are as shown
      below:

      _S_h_e_l_l _s_c_r_i_p_t (_c_o_l_l_e_c_t_s _d_a_t_a, _u_p_d_a_t_e_s _d_a_t_a_b_a_s_e)

       #!/bin/sh
       a=0
       while [ "$a" == 0 ]; do
       snmpwalk -c public 192.168.1.250 hrSWRunPerfMem > snmp_reply
           total_mem=`awk 'BEGIN {tot_mem=0}
                                 { if ($NF == "KBytes")
                                   {tot_mem=tot_mem+$(NF-1)}
                                 }
                           END {print tot_mem}' snmp_reply`
           # I can use N as a replacement for the current time
           rrdtool update target.rrd N:$total_mem
           # sleep until the next 300 seconds are full
           perl -e 'sleep 300 - time % 300'
       done # end of while loop

      _P_e_r_l _s_c_r_i_p_t (_r_e_t_r_i_e_v_e_s _d_a_t_a _f_r_o_m _d_a_t_a_b_a_s_e _a_n_d _g_e_n_e_r_a_t_e_s _g_r_a_p_h_s _a_n_d
      _s_t_a_t_i_s_t_i_c_s)

       #!/usr/bin/perl -w
       #This script fetch data from target.rrd, creates graph of memory consumption
       on target (Dual P3 Processor 1 GHz, 656 MB RAM)

       #calling RRD perl module
       use lib qw( /usr/local/rrdtool-1.0.41/lib/perl ../lib/perl );
       use RRDs;
       my $cur_time = time();                # setting current time
       my $end_time = $cur_time - 86400;     # setting end time to 24 hours behind current time
       my $start_time = $end_time - 2592000; # setting start time to 30 days from end time



                                    - 5 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111
























































                                    - 6 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



       #fetching average values from rrd database between start and end time
       my ($start,$step,$ds_names,$data) =
           RRDs::fetch("target.rrd", "AVERAGE",
                       "-r", "600", "-s", "$start_time", "-e", "$end_time");
       #saving fetched values in 2-dimensional array
       my $rows = 0;
       my $columns = 0;
       my $time_variable = $start;
       foreach $line (@$data) {
         $vals[$rows][$columns] = $time_variable;
         $time_variable = $time_variable + $step;
         foreach $val (@$line) {
                 $vals[$rows][++$columns] = $val;}
         $rows++;
         $columns = 0;
       }
       my $tot_time = 0;
       my $count = 0;
       #saving values from 2-dimensional into 1-dimensional array
       for $i ( 0 .. $#vals )
           $tot_mem[$count] = $vals[$i][1];
           $count++;
       }
       my $tot_mem_sum = 0;
       #calculating total of all values
       for $i ( 0 .. ($count-1) ) {
           $tot_mem_sum = $tot_mem_sum + $tot_mem[$i];
       }
       #calculating average of array
       my $tot_mem_ave = $tot_mem_sum/($count);
       #creating graph
       RRDs::graph ("/images/mem_$count.gif",   \
                   "--title= Memory Usage",    \
                   "--vertical-label=Memory Consumption (MB)", \
                   "--start=$start_time",      \
                   "--end=$end_time",          \
                   "--color=BACK#CCCCCC",      \
                   "--color=CANVAS#CCFFFF",    \
                   "--color=SHADEB#9999CC",    \
                   "--height=125",             \
                   "--upper-limit=656",        \
                   "--lower-limit=0",          \
                   "--rigid",                  \
                   "--base=1024",              \
                   "DEF:tot_mem=target.rrd:mem:AVERAGE", \
                   "CDEF:correct_tot_mem=tot_mem,0,671744,LIMIT,UN,0,tot_mem,IF,1024,/",\
                   "CDEF:machine_mem=tot_mem,656,+,tot_mem,-",\
                   "COMMENT:Memory Consumption between $start_time",\
                   "COMMENT:    and $end_time                     ",\
                   "HRULE:656#000000:Maximum Available Memory - 656 MB",\



                                    - 7 -        Formatted:  August 20, 2003






 RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))                  1111....0000....44445555                   RRRRRRRRDDDD----BBBBEEEEGGGGIIIINNNNNNNNEEEERRRRSSSS((((1111))))
 rrrrrrrrddddttttoooooooollll                                                             rrrrrrrrddddttttoooooooollll
                                 2222000000003333----00007777----22221111



                   "AREA:machine_mem#CCFFFF:Memory Unused",   \
                   "AREA:correct_tot_mem#6699CC:Total memory consumed in MB");
       my $err=RRDs::error;
       if ($err) {print "problem generating the graph: $err\n";}
       #printing the output
       print "Average memory consumption is ";
       printf "%5.2f",$tot_mem_ave/1024;
       print " MB. Graphical representation can be found at /images/mem_$count.gif.";

 AAAAUUUUTTTTHHHHOOOORRRR
      Ketan Patel <k2pattu@yahoo.com>










































                                    - 8 -        Formatted:  August 20, 2003



