Viewing metric data

Metric data created by the controller and applications can assist you when you are troubleshooting issues with the controller or network.

The cURL commands in this section interact with the metrics/apps REST API to display information about metric data collected by the controller. These commands must be used on individual controllers; they do not return information from controller teams.


NOTE: For all of the command descriptions and examples in this section:

  • The backslash (\) character at the end of the line indicates that the command continues on the next line. In the Bash shell, which you use to enter cURL commands, a backslash character that is followed by the newline character is removed from the input stream automatically such that the command is processed as if it were entered on a single line.

  • Items in italic are items you replace with your own values. For example, when you enter a command shown in this section:

    • Replace controller_ip with the IP address of the controller, for example 10.1.1.9

    • Replace token with the authorization token for your session (For information about using authorization tokens, see “API access requires authentication”). Examples in this section use the following token: 3d61f0d3e61349359e6dbd82ec02c113

  • The question mark (?) at the end of the command indicates the beginning of parameters and the ampersand (&) character separates parameters.

  • The output returned by the command is formatted to make it easier for you to read.

    The output of cURL commands is in JSON (JavaScript Object Notation) format. The JSON output can be difficult for human beings to read when it is not formatted for display. To display output that is in a format that is easier to read, you can use the pipe command to send the output to a JSON formatter, or you can interact with the metrics/apps REST API using the RSdoc web interface instead of cURL. For information about using RSdoc, see the HP VAN SDN Controller Programming Guide.


About metrics

Metrics are created by controller components or applications and can be of several different types, such as timers, counters, gauges, and histograms.

About metric values

When the controller component or application creates a metric, it specifies what to do with the resulting metric values. Metric values can be processed in one or more of the following ways:

  • They can be persisted to disk as a time-series of values.

  • They can be exposed for inspection by the user via JMX.

  • They can be retained in memory for internal use by the component or application.

For metric values that are persisted to disk:

  • The time-series values for each individual metric can be persisted at intervals of 1, 5, or 15 minutes. Controller administrators configure the default persistence interval using the MetricManagerComponent controller component. A controller component or application that creates a metric can override the default persistence interval for that metric.

  • Each persisted value represents the value of the metric over the elapsed persistence interval.  For example, for a counter that increments from 100 to 145 during a 5-minute interval, the persisted value for the interval is 45, which is the value of the counter at the start of the interval subtracted from the value of the counter at the end of the interval. For each type of metric, there is a different method of computing the persisted value.

  • Because some metric values are not updated at a steady rate, the persisted information for a metric value includes the raw value and the time over which the value accrued. Providing this information enables the metric viewer to normalize data. For example if the counter value was updated only during the last 3 minutes of the 5-minute interval for which it was persisted:

    • The raw value accrued over the interval is 45, which is the value at the start of the interval subtracted from the value at the end of the interval.

    • The persisted value includes both the raw value and the time over which the data was accrued, which is 3 minutes.

    • The persistence interval configured for the metric is 5 minutes.

    • A normalized rate of accrual over the interval is 15 increments per minute (45 increments divided by 3 minutes) or 75 for the entire 5-minute interval.

  • Controller administrators use MetricManagerComponent controller component to configure the length of time the controller retains persisted metric values on disk. Controller components and applications cannot override this retention period.

For metric values that are exposed via JMX:

  • The metric value reported by JMX is the most recently-reported raw value. The value does not include time interval information or any normalization data such as accrual rate. For example, for a counter that increments from 100 to 145 during a 5-minute interval, but only increments during the last 3 minutes of that 5 minute interval, the metric value reported by JMX depends on when you query JMX for the most recent value. For example:

    • The value JMX reports for the metric at the beginning of the interval is 100.

    • The value JMX reports for the metric at the end of the interval is 145.

    • The value JMX reports for the metric at any time during the first 2 minutes of the interval is 100, because, in this example, the counter does not start incrementing until the last 3 minutes of the interval.

    • The value JMX reports for the metric at minute 4 could be any value from 100 through 145 depending on how frequently the counter value is updated by the creating application or component between minute 2 and the end of the interval.

For metric values that are retained in memory without being persisted or exposed via JMX:

  • Controller components can use these values internally.

  • The component or application can expose this metric data in other ways, such as by including the values in the appropriate section of a controller support report . In such cases, the values exposed are the most recent, raw values (as would be shown if they were exposed via JMX) instead of values for a specific time interval (such as values that are persisted to disk).

Metric identifiers

When a metric is created, it is associated with the following text strings that, taken as a combination, uniquely identify the metric:

Application ID

(REQUIRED) Identifies the application creating the metric. For example, the application ID for the controller is:

com.hp.sdn

metric name

(REQUIRED) Describes the metric. This name is provided by the application that creates the metric. For example, an application that creates metrics to represent the characteristics of traffic on device ports might use names such as:

framesTransmitted
TxFrames
framesReceived
RcvFrames
primary tag, secondary tag

Additional description of the metric, such as a category or a device. Primary and secondary tags are optional, but if a secondary tag is provided, a primary tag must be provided. These tags are provided by the application or component that creates the metric. For example, an application that creates metrics to represent the characteristics of traffic on device ports might use a primary tag to identify a device and a secondary tag to identify specific ports on the device. The resulting metrics are be structured so that for each device (primary tag) there are multiple ports (secondary tags) that each have several metrics associated with them.

Listing the application IDs for applications that have persisted metrics to disk

The metrics/apps command displays the application IDs for applications—including the controller itself—that have persisted metric data to disk.

You use the application ID as input for metrics commands that require an application ID in the URL.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/apps”

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/apps”

Example output

In this example, only the base controller itself has persisted metric data to disk. The application id is: com.hp.sdn

This example shows the JSON output as returned by the cURL command:

{"apps":[{"app_id":"com.hp.sdn","app_name":"HP VAN SDN Controller"}]}

This example shows the JSON output formatted for readability:

{
   "apps":[
      {
         "app_id":"com.hp.sdn",
         "app_name":"HP VAN SDN Controller"
      }
   ]
}

Listing the metrics persisted by a specific application

The metrics/apps/app_id command lists all of the metrics available for a specific application. The list includes all metrics that are currently configured to persist or are no longer configured to persist but have not yet been removed from disk by the metering framework.

name

The name of the metric. The controller lists only the metrics that have a name that matches the specified value. If you do not specify a name, the controller lists all metrics.

primary_tag

The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value.

secondary_tag

The name of the primary tag. The controller lists only the metrics that have a secondary tag that matches the specified value.

If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id \
?name=name&primary_tag=primary_tag&secondary_tag=secondary_tag

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn”

Example output

This example shows a partial listing of the output from the example command.

The uid for a metric is the unique identifier assigned to the metric on the controller.

{
  "metrics":[
    {
       "app_id":"com.hp.sdn",
       "type":"RATIO_GAUGE",
       "name":"cpuLoadSystem",
       "description":"The recent CPU usage of the system.",
       "primary_tag":"jvm",
       "secondary_tag":"operatingSystem",
       "jmx":false,
       "persistence":true,
       "summary_interval":"ONE",
       "uid":"42f65cd8-03c3-4cad-9788-012d513e3c0f"
     },
     {
       "app_id":"com.hp.sdn",
       "type":"GAUGE","name":"committedBytes",
       "description":"The amount of non-heap memory in bytes committed for the JVM to use.",
       "primary_tag":"jvm",
       "secondary_tag":"memoryNonHeap",
       "jmx":false,
       "persistence":true,
       "summary_interval":"ONE",
       "uid":"b82f5b00-0373-4a23-b5a8-bbda7eec44cb"
     },
     {
       "app_id":"com.hp.sdn",
       "type":"GAUGE","name":"countTerminated",
       "description":"Number of JVM threads that had exited.",
       "primary_tag":"jvm",
       "secondary_tag":"threads",
       "jmx":false,
       "persistence":true,
       "summary_interval":"ONE",
       "uid":"0e9fe62e-01fd-42e9-88a6-f92021a5e786"
     },

.
.
.       
     {
       "app_id":"com.hp.sdn",
       "type":"ROLLING_COUNTER",
       "name":"count",
       "description":"The number of JVM garbage collections actions during the sampling interval.",
       "primary_tag":"jvm",
       "secondary_tag":"garbageCollection",
       "jmx":false,
       "persistence":true,
       "summary_interval":"ONE",
       "uid":"d62c49d4-46b3-4c2c-be60-24f6fa6c6bf6"
      },
.
.
.

Listing the primary tags for metrics persisted by an application

The metrics/apps/app_id/primaries command lists all of the primary tags associated with the metrics persisted by a specific application.

Optionally you can specify the following parameter:

name

The name of the metric.

If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/primaries\
?name=metric_name

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/primaries”

Example output

The only primary tag associated with the controller itself in this example is jvm

{
  "primaries":[
     "jvm"
  ]
}

Listing the secondary tags for metrics persisted by an application

The metrics/apps/app_id/secondaries command lists all of the secondary tags associated with the metrics persisted by a specific application.

Optionally you can specify one or more of the following parameters:

name

The name of the metric. The controller lists only the metrics that have a name that matches the specified value.

primary_tag

The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value.

If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/secondaries\
?name=name&primary_tag=primary_tag

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/secondaries”

Example output

{
   "secondaries":[
      "nioDirectMemory",
      "operatingSystem",
      "threads",
      "garbageCollection",
      "memoryNonHeap",
      "memoryHeap",
      "memoryTotal",
      "nioMappedMemory"
   ]
}

Listing the names of metrics persisted by an application

The metrics/apps/app_id/names command lists names of the metrics persisted by a specific application.

Optionally you can specify one or more of the following parameters:

primary_tag

The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value.

secondary_tag

The name of the primary tag. The controller lists only the metrics that have a secondary tag that matches the specified value.

If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/names\
?primary_tag=primary_tag&secondary_tag=secondary_tag

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/names”

Example output

This example shows a partial listing of the output from the example command.

{
   "names":[
      "averageBufferUsedBytes",
      "countDeadlocked",
      "bufferCapacityBytes",
      "count",
      "countNew",
      "bufferCount",
      "countWaiting",
      "fileDescriptorsOpen",
      "uptimeMs",
      "countTerminated",
      "elapsedMs",
      "countTimedWaiting",
      "countDaemon",
.
.
.
      "countBlocked",
      "committedBytes"
   ]
}

Listing information about a persisted metric identified by its UID

The metrics/metric_uid command lists information about a specific metric, which is identified by its UID (unique ID).

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/metric_uid

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/431b746e-e62e-4874-a801-b1438eaac635”

Example output

{
   "metric":{
      "app_id":"com.hp.sdn",
      "type":"GAUGE",
      "name":"usedBytes",
      "description":"The amount of heap memory currently being used by the JVM in bytes.",
      "primary_tag":"jvm",
      "secondary_tag":"memoryHeap",
      "jmx":false,
      "persistence":true,
      "summary_interval":"ONE",
      "uid":"431b746e-e62e-4874-a801-b1438eaac635"
   }
}

Listing the time-series values for a persisted metric identified by its UID

The metrics/metric_uid/values command lists information about a specific metric, which is identified by its UID (unique ID).

Optionally you can specify one or more of the following parameters:

start

The earliest time to query for time-series data. The date and time in must be in the format yyyy-mm-dd+hh:mm. If you specify an end but do not specify a start, the value used for start is the time of the oldest instance of the metric that is within the configured age-out time.

end

The most recent time to query for time-series data. The date and time in must be in the format yyyy-mm-dd+hh:mm. If you specify a start but do not specify an end, the value used for end is the time you enter the command.

interval

The interval each returned data point is to represent. This parameter is required if you specify the start or the end parameter. The interval you specify must be equal to or greater than the interval at which the metric’s time-series data is persisted. Choose one of the following values: 1, 5, 15, 30, 60, day, and all, where the numeric values represent the number of minutes, day represents a 24-hour period, and all represents the summarized values over the lifetime of the metric data collection.

If you do not specify parameters, only the last persisted time-series value for the metric is returned.

Specifying an interval summarizes the data, and enables you to quickly look for anomalous data. For example, instead of viewing 1440 discrete minute-by-minute values for a specific metric over the course of a day, you can request that each returned value represent 60 minutes so that the number of returned values is more manageable and can be more easily inspected.

Summarizing the returned data for a period typically results in larger values for the metrics themselves and larger values for the milliseconds spanned. The computation of the values returned over longer intervals depends upon the type of metric. For example, gauge values are averaged over the data points encompassed in the summary, but counter values are summed over the summary interval.

Command form


curl --noproxy controller_ip -X GET \
--header "X-Auth-Token:token" --fail -ksSfL \
--url “https://controller_ip:8443/sdn/v2.0/metrics/metric_uid/values\
?start=start&end=end&interval=interval

Command example


curl --noproxy 10.1.1.9 -X GET \
--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \
--url “https://10.1.1.9:8443/sdn/v2.0/metrics/431b746e-e62e-4874-a801-b1438eaac635?start=2014-09-23+18:00&interval=5”

Example output

{
   "metric_values":{
      "uid":"431b746e-e62e-4874-a801-b1438eaac635",
      "type":"GAUGE",
      "datapoint_count":6,
      "datapoints":[
         {
            "update_time":"Tue Sep 23 18:03:55 PDT 2014",
            "milliseconds_span":300000,
            "last":3.274097568E8
         },
         {
            "update_time":"Tue Sep 23 18:08:55 PDT 2014",
            "milliseconds_span":300000,
            "last":3.133927072E8
         },
         {
            "update_time":"Tue Sep 23 18:13:55 PDT 2014",
            "milliseconds_span":300000,
            "last":2.154562624E8
         },
         {
            "update_time":"Tue Sep 23 18:18:55 PDT 2014",
            "milliseconds_span":300000,
            "last":4.192128832E8
         },
         {
             "update_time":"Tue Sep 23 18:23:55 PDT 2014",
             "milliseconds_span":300000,
             "last":3.864813136E8
         },
         {
             "update_time":"Tue Sep 23 18:27:55 PDT 2014",
             "milliseconds_span":240000,
             "last":3.847236E8
         }
      ]
   }
}