Using the Event Log for troubleshooting switch problems

The Event Log records operating events in single- or double-line entries and serves as a tool to isolate and troubleshoot problems.

Once the log has received 2000 entries, it discards the oldest message each time a new message is received. The Event Log window contains 14 log entry lines. You can scroll through it to view any part of the log.

Once the log has received 2000 entries, it discards the oldest message each time a new message is received. The Event Log window contains 14 log-entry lines. You can scroll through it to view any part of the log.


[NOTE: ]

NOTE: The Event Log is erased if power to the switch is interruptedor if you enter the boot system command. The contents of the Event Log are not erased if you:

  • Reboot the switch by choosing the Reboot Switch option from the menu interface.

  • Enter the reload command from the CLI.


Event Log entries

As shown in Format of an event log entry, each Event Log entry is composed of six or seven fields, depending on whether numbering is turned on or not:

Format of an event log entry

Format of an event log entry
Item Description

Severity

One of the followingcodes (from highest to lowest severity):

M—(major) indicates that a fatal switch error has occurred.

E—(error) indicates that an error condition occurred on the switch.

W—(warning) indicates that a switch service has behaved unexpectedly.

I—(information) provides information on normal switch operation.

D—(debug) is reserved for HP internal diagnostic information.

Date

Thedate in the format mm/dd/yy when an entry is recorded in the log.

Time

The time in the format hh:mm:ss when an entry is recorded in the log.

Event number

The number assigned to an event. You can turn event numbering on and off with the [no] log-number command.

System module

The internal module (such as "ports:" for port manager) that generated a log entry.If VLANs are configured, a VLAN name also appears for an event that is specific to an individual VLAN.

Event message

A brief description of the operating event.

Event Log system modules

System module Description Documented in HP Switch hardware/software guide

802.1x

802.1X authentication: Provides access control on a per-client or per-port basis:

  • Client-level security that allows LAN access to 802.1X clients (up to 32 per port) with valid user credentials

  • Port-level security that allows LAN access only on ports on which a single 802.1X-capable client (supplicant) has entered valid RADIUS user credentials

Access Security Guide

acl

ACLs: Filter layer-3 IP traffic to or from a host to block unwanted IP traffic and block or limit other protocol traffic such as TCP, UDP, IGMP, and ICMP. ACEs specify the filter criteria and an action (permit or deny) to take on a packet if it meets the criteria.

Advanced Traffic Management Guide

addrmgr

Address Table Manager: Manages MAC addresses that the switch has learned and are stored in the switch's address table.

Management and Configuration Guide

arp-protect

Dynamic ARP Protection: Protects the network from ARP cache poisoning. Only valid ARP requests and responses are relayed or used to update the local ARP cache. ARP packets with invalid IP-to-MAC address bindings advertised in the source protocol address and source physical address fields are discarded.

Access Security Guide

auth

Authorization: A connected client must receive authorization through web, AMC, RADIUS-based, TACACS+-based, or 802.1X authentication before it can send traffic to the switch.

Access Security Guide

cdp

Cisco Discovery Protocol: Supports reading CDP packets received from neighbor devices, enabling a switch to learn about adjacent CDP devices. HP does not support the transmission of CDP packets to neighbor devices.

Management and Configuration Guide

chassis

Hardware operation, including modules and ports, power supply, fans, transceivers, CPU interrupt errors, switch temperature, and so on. Chassis messages include events on Power Over Ethernet (POE) operation.

Installations Guides

Management and Configuration Guide

connfilt

Connection-rate filtering: Used on the network edge to protect the network from attack by worm-like malicious code by detecting hosts that are generating IP traffic that exhibits this behavior and (optionally) either throttling or dropping all IP traffic from the offending hosts.

Connection-rate filtering messages include events on virus throttling. Virus throttling uses connection-rate filtering to stop the propagation of malicious agents.

Access Security Guide

console

Console interface used to monitor switch and port status, reconfigure the switch, and read the event log through an in-band Telnet or out-of-band connection.

Installation and Getting Started Guide

cos

Class of Service (CoS): Provides priority handling of packets traversing the switch, based on the IEEE 802.1p priority carried by each packet.

CoS messages also include QoS events. The QoS feature classifies and prioritizes traffic throughout a network, establishing an end-to-end traffic priority policy to manage available bandwidth and improve throughput of important data.

Advanced Traffic Management Guide

dca

Dynamic Configuration Arbiter (DCA) determines the client-specific parameters that are assigned in an authentication session.

Access Security Guide

dhcp

Dynamic Host Configuration Protocol (DHCP) server configuration: Switch is automatically configured from a DHCP (Bootp) server, including IP address, subnet mask, default gateway, Timep Server address, and TFTP server address.

Management and Configuration Guide

dhcp v6c

DHCP for IPv6 prefix assignment

IPv6 Configuration Guide

dhcpr

DHCP relay: Forwards client-originated DHCP packets to a DHCP network server.

Advanced Traffic Management Guide

download

Download operation for copying a software version or files to the switch.

Management and Configuration Guide

dhcp-snoop

DHCP snooping: Protects your network from common DHCP attacks, such as address spoofing and repeated address requests.

Access Security Guide

dma

Direct Access Memory (DMA): Transmits and receives packets between the CPU and the switch.

fault

Fault Detection facility, including response policy and the sensitivity level at which a network problem should generate an alert.

Management and Configuration Guide

fdr-log

FDR collects information that is “interesting” at the time of the crash, as well as when the switch is misbehaving, but has not crashed. Runtime logs are written to FDR memory while the switch is running, and crashtime logs are collected and stored in the FDR buffer during a switch crash.

Management and Configuration Guide

ffi

Find, Fix, and Inform: Event or alert log messages indicating a possible topology loop that causes excessive network activity and results in the network running slow. FFI messages include events on transceiver connections with other network devices.

Installation and Getting Started Guide Management and Configuration Guide

garp

Generic Attribute Registration Protocol (GARP), defined in the IEEE 802.1D-1998 standard.

Advanced Traffic Management Guide

gvrp

GARP VLAN Registration Protocol (GVRP): Manages dynamic 802.1Q VLAN operations, in which the switch creates temporary VLAN membership on a port to provide a link to another port in the same VLAN on another device.

Advanced Traffic Management Guide

hpesp

Management module that maintains communication between switch ports.

Installation and Getting Started Guide

idm

Identity-driven Management: Optional management application used to monitor and control access to switch.

Advanced Traffic Management Guide

igmp

Internet Group Management Protocol: Reduces unnecessary bandwidth usage for multicast traffic transmitted from multimedia applications on a per-port basis.

Multicast and Routing Guide

inst-mon

Instrumentation Monitor: Identifies attacks on the switch by generating alerts for detected anomalies.

Access Security Guide

ip

IP addressing: Configures the switch with an IP address and subnet mask to communicate on the network and support remote management access; configures multiple IP addresses on a VLAN; enables IP routing on the switch.

Management and Configuration Guide

Multicast and Routing Guide

ipaddrmgr

IP Address Manager: Programs IP routing information in switch hardware.

Multicast and Routing Guide

iplock

IP Lockdown: Prevents IP source address spoofing on a per-port and per-VLAN basis by forwarding only the IP packets in VLAN traffic that contain a known source IP address and MAC address binding for the port.

Access Security Guide

ipx

Novell Netware protocol filtering: On the basis of protocol type, the switch can forward or drop traffic to a specific set of destination ports on the switch.

Access Security Guide

kms

Key Management System: Configures and maintains security information (keys) for all routing protocols, including a timing mechanism for activating and deactivating an individual protocol.

Access Security Guide

lacp

LACP trunks: The switch can either automatically establish an 802.3ad-compliant trunk group or provide a manually configured, static LACP trunk.

Management and Configuration Guide

ldbal

Load balancing in LACP port trunks or 802.1s Multiple Spanning Tree protocol (MSTP) that uses VLANs in a network to improve network resource utilization and maintain a loop-free environment.

Load-balancing messages also include switch meshing events. The switch meshing feature provides redundant links, improved bandwidth use, and support for different port types and speeds.

Management and Configuration Guide Advanced Traffic Management Guide

lldp

Link-Layer Discovery Protocol: Supports transmitting LLDP packets to neighbor devices and reading LLDP packets received from neighbor devices, enabling a switch to advertise itself to adjacent devices and to learn about adjacent LLDP devices.

Management and Configuration Guide

loop_protect

Loop protection: Detects the formation of loops when an unmanaged device on the network drops spanning tree packets and provides protection by transmitting loop protocol packets out ports on which loop protection has been enabled.

Advanced Traffic Management Guide

macauth

Web and MAC authentication: Port-based security employed on the network edge to protect private networks and the switch itself from unauthorized access using one of the following interfaces:

  • Web page login to authenticate users for access to the network

  • RADIUS server that uses a device's MAC address for authentication

Access Security Guide

maclock

MAC lockdown and MAC lockout

  • MAC lockdown prevents station movement and MAC address "hijacking" by requiring a MAC address to be used only on an assigned port on the switch. MAC Lockdown also restricts the client device to a specific VLAN.

  • MAC lockout blocks a specific MAC address so that the switch drops all traffic to or from the specified address.

Access Security Guide

mgr

HP PCM and PCM+: Windows-based network management solutions for managing and monitoring performance of HP switches. PCM messages also include events for configuration operations.

Management and Configuration Guide

mld

Multicast Listener Discovery (MLD): IPv6 protocol used by a router to discover the presence of multicast listeners. MLD can also optimize IPv6 multicast traffic flow with the snooping feature.

Multicast and Routing Guide

mtm

Multicast Traffic Manager (MTM): Controls and coordinates L3 multicast traffic for upper layer protocols.

Multicast and Routing Guide

netinet

Network Internet: Monitors the creation of a route or an Address Resolution Protocol (ARP) entry and sends a log message in case of failure.

Advanced Traffic Management Guide

pagp

Ports Aggregation Protocol (PAgP): Obsolete. Replaced by LACP (802.3ad).

ports

Port status and port configuration features, including mode (speed and duplex), flow control, broadcast limit, jumbo packets, and security settings.

Port messages include events on POE operation and transceiver connections with other network devices.

Installation and Getting Started Guide Management and Configuration Guide

Access Security Guide

radius

RADIUS (Remote Authentication Dial-In User Service) authentication and accounting: A network server is used to authenticate user-connection requests on the switch and collect accounting information to track network resource usage.

Access Security Guide

ratelim

Rate-limiting: Enables a port to limit the amount of bandwidth a user or device may utilize for inbound traffic on the switch.

Management and Configuration Guide

sflow

Flow sampling: sFlow is an industry standard sampling technology, defined by RFC 3176, used to continuously monitor traffic flows on all ports providing network-wide visibility into the use of the network.

Management and Configuration Guide

snmp

Simple Network Management Protocol: Allows you to manage the switch from a network management station, including support for security features, event reporting, flow sampling, and standard MIBs.

Management and Configuration Guide

sntp

Simple Network Time Protocol: Synchronizes and ensures a uniform time among interoperating devices.

Management and Configuration Guide

ssh

Secure Shell version 2 (SSHv2): Provides remote access to management functions on a switch via encrypted paths between the switch and management station clients capable of SSH operation.

SSH messages also include events from the Secure File Transfer Protocol (SFTP) feature. SFTP provides a secure alternative to TFTP for transferring sensitive information, such as switch configuration files, to and from the switch in an SSH session.

Access Security Guide

ssl

Secure Socket Layer Version 3 (SSLv3), including Transport Layer Security (TLSv1) support: Provides remote web access to a switch via encrypted paths between the switch and management station clients capable of SSL/TLS operation.

Access Security Guide

stack

Stack management: Uses a single IP address and standard network cabling to manage a group (up to 16) of switches in the same IP subnet (broadcast domain), resulting in a reduced number of IP addresses and simplified management of small workgroups for scaling your network to handle increased bandwidth demand.

Advanced Traffic Management Guide

stp

Multiple-instance spanning tree protocol/MSTP (802.1s): Ensures that only one active path exists between any two nodes in a group of VLANs in the network. MSTP operation is designed to avoid loops and broadcast storms of duplicate messages that can bring down the network.

Advanced Traffic Management Guide

system

Switch management, including system configuration, switch bootup, activation of boot ROM image, memory buffers, traffic and security filters.

System messages also include events from management interfaces (menu, CLI, and HP PCM+) used to reconfigure the switch and monitor switch status and performance.

Basic Operation Guide

Access Security Guide

tacacs

TACACS+ authentication: A central server is used to control access to the switches (and other TACACS-aware devices) in the network through a switch's console port (local access) or Telnet (remote access).

Access Security Guide

tcp

Transmission Control Protocol: A transport protocol that runs on IP and is used to set up connections.

Advanced Traffic Management Guide

telnet

Session established on the switch from a remote device through the Telnet virtual terminal protocol.

Basic Operation Guide

tftp

Trivial File Transfer Protocol: Supports the download of files to the switch from a TFTP network server.

Basic Operation Guide

timep

Time Protocol: Synchronizes and ensures a uniform time among interoperating devices.

Management and Configuration Guide

udld

Uni-directional Link Detection: Monitors a link between two switches and blocks the ports on both ends of the link if the link fails at any point between the two devices.

Access Security Guide

udpf

UDP broadcast forwarding: Supports the forwarding of client requests sent as limited IP broadcasts addressed to a UDP application port on a network server.

Multicast and Routing Guide

update

Updates (TFTP or serial) to HP switch software and updates to running-config and start-up config files

Basic Operation Guide

usb

Auxiliary port that allows you to connect external devices to the switch.

Installation and Getting Started Guide

vlan

Static 802.1Q VLAN operations, including port-and protocol-based configurations that group users by logical function instead of physical location

  • A port-based VLAN creates a layer-2 broadcast domain comprising member ports that bridge IPv4 traffic among themselves.

  • A protocol-based VLAN creates a layer-3 broadcast domain for traffic of a particular routing protocol, and comprises member ports that bridge traffic of the specified protocol type among themselves.

VLAN messages include events from management interfaces (menu, CLI, and HP PCM+) used to reconfigure the switch and monitor switch status and performance.

Advanced Traffic Management Guide

xmodem

Xmodem: Binary transfer feature that supports the download of software files from a PC or UNIX workstation.

Basic Operation Guide

Using the Menu

To display the Event Log from the Main Menu, select Event Log. An event log display shows a sample event log display.

An event log display

HP Switch 5406zl                                     25-Oct-2013  18:02:52
==========================-CONSOLE - MANAGER MODE -
=============================
M 10/25/13 16:30:02 sys: 'Operator cold reboot from CONSOLE session.'
I 10/25/13 17:42:51 00061 system: ------------------------------------------
-
I 10/25/13 17:42:51 00063 system: System went down : 10/25/13 16:30:02
I 10/25/13 17:42:51 00064 system: Operator cold reboot from CONSOLE session.
W 10/25/13 17:42:51 00374 chassis: WARNING: SSC is out of Date: Load 8.2 or
newer
I 10/25/13 17:42:51 00068 chassis: Slot D Inserted
I 10/25/13 17:42:51 00068 chassis: Slot E Inserted
I 10/25/13 17:42:51 00068 chassis: Slot F Inserted
I 10/25/13 17:42:51 00690 udpf: DHCP relay agent feature enabled
I 10/25/13 17:42:51 00433 ssh: Ssh server enabled
I 10/25/13 17:42:51 00400 stack: Stack Protocol disabled
I 10/25/13 17:42:51 00128 tftp: Enable succeeded
I 10/25/13 17:42:51 00417 cdp: CDP enabled

----  Log events stored in memory 1-751. Log events on screen 690-704.

  Actions->    Back    Next page    Prev page    End    Help

Return to previous screen.
Use up/down arrow to scroll one line, left/right arrow keys to
change action selection, and <Enter> to execute action.

The log status line below the recorded entries states the total number of events stored in the event log and which logged events are currently displayed.

To scroll to other entries in the Event Log, either preceding or following the currently visible portion, press the keys indicated at the bottom of the display (Back,Nextpage, Prev page, or End) or the keys described in TableTable 3-3.

Event Log control keys

Key

Action

[N]

Advances the display by one page (next page).

[P]

Rolls back the display by one page (previous page).

[v]

Advances display by one event (down one line).

[ˆ]

Rolls back display by one event (up one line).

[E]

Advances to the end of the log.

[H]

Displays Help for the Event Log.

Using the CLI

Syntax:

show logging [-a, -b, -r, -s, -t, -m, -p, -w, -i, -d] [<option-str>]

By default, the show logging command displays the log messages recorded since the last reboot in chronological order:

-a

Displays all recorded log messages, including those before the last reboot.

-b

Displays log events as the time since the last reboot instead of in a date/time format.

-r

Displays all recorded log messages, with themost recent entries listed first (reverse order).

-s

Displays the active management module (AM) and standby management module (SM) log events.

-t

Displays the log events with a granularity of 10 milliseconds.

-m

Displays only major log events.

-p

Displays only performance log events.

-w

Displays only warning log events.

-i

Displays only informational log events.

-d

Displays only debug log events.

<option-str>

Displays all Event Log entries that contain the specified text. Use an <option-str> value with -a or -r to further filter show logging command output.

Example:

To display all Event Log messages that have "system" in the message text or module name, enter the following command:

HP Switch# show logging -a system

To display all Event Log messages recorded since the last reboot that have the word "system" in the message text or module name, enter:

HP Switch# show logging system

Clearing Event Log entries

Syntax:

clear logging

Removes all entries from the event log display output.

Use the clear logging command to hide, but not erase, Event Log entries displayed in show logging command output. Only new entries generated after you enter the command will be displayed.

To redisplay all hidden entries, including Event Log entries recorded prior to the last reboot, enter the show logging -a command.

Turning event numbering on

Syntax:

[no] log-numbers

Turns event numbering on and off

Using log throttling to reduce duplicate Event Log and SNMP messages

A recurring event can generate a series of duplicate Event Log messages and SNMP traps in a relatively short time. As a result, the Event Log and any configured SNMP trap receivers may be flooded with excessive, exactly identical messages. To help reduce this problem, the switch uses log throttle periods to regulate (throttle) duplicate messages for recurring events, and maintains a counter to record how many times it detects duplicates of a particular event since the last system reboot.

When the first instance of a particular event or condition generates a message, the switch initiates a log throttle period that applies to all recurrences of that event. If the logged event recurs during the log throttle period, the switch increments the counter initiated by the first instance of the event, but does not generate a new message.

If the logged event repeats again after the log throttle period expires, the switch generates a duplicate of the first message, increments the counter, and starts a new log throttle period during which any additional instances of the event are counted, but not logged. Thus, for a particular recurring event, the switch displays only one message in the Event Log for each log throttle period in which the event reoccurs. Also, each logged instance of the event message includes counter data showing how many times the event has occurred since the last reboot. The switch manages messages to SNMP trap receivers in the same way.

Log throttle periods

The length of the log throttle period differs according to an event's severity level:

Severity level

Log throttle period

I (Information)

6000 Seconds

W (Warning)

600 Seconds

D (Debug)

60 Seconds

M (Major)

6 Seconds

Example:

Suppose that you configure VLAN 100 on the switch to support PIM operation, but do not configure an IP address. If PIM attempts to use VLAN 100, the switch generates the first instance of the following Event Log message and counter.


[NOTE: ]

NOTE: In The first instance of an event message and counter the counter (1) indicates that this is the first instance of this event since the switch last rebooted.


The first instance of an event message and counter

W 10/01/12 09:00:33 PIM:No IP address configured on VID 100 (1)

If PIM operation causes the same event to occur six more times during the initial log throttle period, there are no further entries in the Event Log. However, if the event occurs again after the log throttle period has expired, the switch repeats the message (with an updated counter) and starts a new log throttle period.

Duplicate messages over multiple log throttling periods

Duplicate messages over multiple log throttling periods

Note that if the same type of event occurs under different circumstances, the switch handles these as unrelated events for the purpose of Event Log messages. For example, if PIM operation simultaneously detects that VLANs 100 and 205 are configured without IP addresses, you see log messages similar to the following:

Example: of log messages generated by unrelated events of the same type

Example: of log messages generated by unrelated events of the same type

Example: of event counter operation

Suppose the switch detects the following after a reboot:

  • Three duplicate instances of the PIM "Send error" during the first log throttle period for this event

  • Five more instances of the same Send error during the second log throttle period for this event

  • Four instances of the same Send error during the third log throttle period for this event

In this case, the duplicate message appears three times in the Event Log (once for each log throttle period for the event being described), and the duplicate message counter increments as shown in Table 3-4. (The same operation applies for messages sent to any configured SNMP trap receivers.)

How the duplicate message counter increments

Instances during 1st log throttle period

Instances during 2nd log throttle period

Instances during 3rd log throttle period

Duplicate message counter[*]

3

   

1

 

5

 

4

   

4

9

[*] This value always comprises the first instance of the duplicate message in the current log throttle period plus all previous occurrences of the duplicate message occurring since the switch last rebooted.

Reporting information about changes to the running configuration

Syslog can be used for sending notifications to a remote syslog server about changes made to the running configuration. The notifications in the syslog messages are sent in ASCII format and contain this information:

  • Notice-Type: Describes the syslog notification as a “running config change”.

  • Event-ID: Identifier for the running config change event that occurred on the switch.

  • Config-Method: The source for the running config change.

  • Device-Name: The managed device.

  • User-Name: User who made the running config change.

  • Remote-IP-Address: IP address of a remote host from which the user is connected.

Syntax:

[no] logging notify <running-config-change> [transmission-interval <0-4294967295>

Enables sending the running configuration change notifications to the syslog server.

The no form of the command disables sending the running configuration changes to the syslog server.

Default: Disabled

<running-config-change> Mandatory option for the notify parameter. Specifies the type of notification to send.
transmission-interval <0-4294967295> Specifies the time interval (in seconds) between the transmission of two consecutive notifications. Running config changes occurring within the specified interval will not generate syslog notifications.

A value of zero means there is no limit; a notification is sent for every running config change.

Default: Zero

Sending running config changes to the syslog server

HP Switch(config)# logging notify running-config-change
transmission-interval 10