High switch CPU and memory usage are affecting switch performance

Symptom

A switch that has Network Analytics Engine (NAE) agents installed is experiencing high CPU usage, high memory usage, or both, and overall performance is affected.

Cause

The NAE is attempting to monitor too many switch resources. For example, a script uses wildcard characters in a URI to monitor all interfaces, ACLs, or VLANs, and the switch has hundreds or thousands of those items.

Action
  1. Verify that the resources associated with the NAE are consuming the memory and CPU resources by entering the following commands in the switch CLI:
    • show system resource-utilization daemon hpe-tsdbd

    • show system resource-utilization daemon hpe-policyd

    • show system resource-utilization daemon prometheus

    The response to the show system resource-utilization daemon <daemon> command shows the CPU and memory usage the specified daemon.

  2. Identify installed scripts that include the wildcard character (*) in the URIs of monitors.

    For example, the following scripts written by Aruba contain wildcard characters to specify resources that can exist in large numbers on a switch:

    • tx_rx_stats_monitor

    • fault_finder

    In the Web UI, the Script Details page shows the contents of the script.

    Look for monitor URIs that specify the wildcard character for resources that your switch has in large numbers. For example:

    • interfaces/*

    • acls/*

    • vlans/*

  3. Delete agents associated with the scripts you identified.

    Disabling the agent is not sufficient because the switch does not delete time series data when an agent is disabled.