What's being sent to my Cisco devices?

This “new” world of network automation requires us, more and more, to trust external scripts and controllers to make changes on our network devices for us. This might be fine for our in-house Ansible scripts. However, whether we are troubleshooting or just curious, sometimes we want to see what an external controller, such as Cisco DNA Center, is actually doing to the device.

Even with our own scripts, we might want to see what is happening from the device's perspective so that we can debug or optimize them.

Enter Cisco IOS Embedded Event Manager (EEM), which has been around for many years.  EEM provides a way to setup scripts inside of your Cisco router or switch configurations that can be triggered by various events. In addition, EEM can be used to automate tasks on a Cisco device -- these include configuration or logging tasks that are started based on events such as timers, Syslog messages, or even CLI commands.

For our monitoring purposes, we'll examine two very simple EEM scripts to help you see what is happening on your devices. One script will display the output to Syslog and subsequently to your console; the second will write the output to both Syslog and to a file in flash.

Note: These basic scripts will only catch CLI commands entered either 1) via tty with SSH or Telnet or 2) on the console.  They will not capture commands entered via an API interface such as NETCONF or other management interfaces such as HTTP or SNMP.

Also: I do not take credit for writing these scripts - They have been floating around in the wild for years, so I really have no idea who originally wrote them.

Simple EEM Script to Output to Syslog/Console

This first script will capture any commands entered at the CLI and then write them to Syslog (and the console by default).  It can be entered in configuration mode:

event manager applet catchall
 event cli pattern ".*" sync no skip no
 action 1.0 syslog msg "$_cli_msg"
end

Plenty of documentation exists about EEM, so I won't dive too deeply into the syntax. However, the flow of this one is pretty straightforward.

1. Creates the EEM applet named catchall
2. Triggers (event) the applet based on any CLI input
3. Writes (action) the input to Syslog, which by default will also display it on the device console

The result on the device console looks like this: This is an example of some of the commands that Cisco DNA Center will send to a device during its "resync" process:

To quickly remove or disable the applet, just enter no event manager applet catchall in configuration mode.  For example:

Note: Depending on how many tools and scripts you have talking to your device, it's probably not a good idea to leave these applets running on the device indefinitely.  They could easily overwhelm both your Syslog servers and the device itself because the applet will trigger with each CLI command that is detected.

EEM Script to Output to Syslog/Console & Flash

So, the above script is great when 1) there are only a handful of commands that you need to capture or monitor, and 2) you can quickly copy & paste the output somewhere else to analyze. However, what if there are a batch of commands that you want to capture?  Or, what if you want to capture several commands over a period of time instead of in realtime?

We can use this modified version of the EEM script that will help us.  As before, the script should be entered in configuration mode:

event manager applet catchall_flash
 event cli pattern ".*" sync no skip no
 action 1.0 syslog msg "$_cli_msg"
 action 2.0 file open FH flash:eem_logall.txt a+
 action 2.1 file puts FH "$_event_pub_time $_cli_msg"
 action 2.2 file close FH
end

You'll note that the first three lines of the script are identical to the first script, except for the applet being named catchall_flash, however we are also adding some further actions that will also create a file in flash named eem_logall.txt and write any CLI input to that file.

Now, if we jump to the console, we can see that a file is created on the flash filesystem.  We can view the file with the more eem_logall.txt command to see its contents:

You'll see here that a bunch of commands are being sent to our device, including the commands that we entered on the console.  Once you're finished collecting these commands, you can move this file off the device to analyze it.

Note: As with the first script, you probably don't want to leave this script running indefinitely as it will fill your Syslog/console logs (and potentially your flash filesystem) by continually writing output to the file.

The script can be removed using the no event manager applet catchall_flash command.

As I mentioned at the top, EEM on a Cisco device can be used for many different things. This is just a simple example of a script that can be extremely useful when testing and monitoring any kind of automation that uses the CLI.

If you have any issues with the scripts or have any questions, please feel free to let me know in the comments below or Tweet at me @eiddor.