For investigating a problem in client to server communication, I need to capture packets for analysis. However it's not allowed to install a packet analyzer, such as Wireshark or tcpdump, on client or server. They client is connected to a Catalyst 3560 and the server to a Catalyst 3750 switch.
Can I plugin my laptop to a switchport for capturing traffic with my laptop's packet analyzer, and how?
The client switchport or the server switchport can be monitored. A third switchport can be configured as a mirror port. This means that this mirror port will receive copies of all packets on the corresponding original port, while the original traffic won't be affected.
For example, on the Catalyst 3560:
Enter configuration mode:
conf t
Define the source and set the session number:
monitor session 1 source interface fa 0/24
Here, the session number can be from 1 to 66, you could also specify a VLAN or an ethernet channel. Also, interface ranges such as fa 0/25 - 26 are possible, and interface list, such as fa 0/24,fa 0/26, if you would like to monitor several clients at the same time. Also by repeating the command you can add ports, or remove using no. Mixing ports and VLANs is not possible in the same session, another restriction is that you cannot use a destination port as a source port.
Define the destination port:
monitor session 1 destination interface gi 0/1
You can use a normal port, but not a VLAN. Similarly to above, a destination port cannot be a source port: a port used here can either be a source or a destination port, and only of one session. Again, you can specify multiple ports like above.
You may want to exit configiration mode and save the config.
You may have a look at your defined session - here multiple ports, tried like above:
#show monitor session 1
Session 1
---------
Type : Local Session
Source Ports :
Both : Fa0/24,Fa0/25-26
Destination Ports : Fa0/48,Gi0/1
Encapsulation : Native
Ingress : Disabled
You can see an encapsulation here - optionally you can set it to replicate for replicating the source interface encapsulation method, such as by adding encapsulation replicate after the source interface. Furthermore, you can specify a direction (tx, rx, both), filter VLANs and more. The Ingress: Disabled line means that the switch will not accept any frames presented to it by your capture device on a destination port. For such finer details and for further restrictions and default settings have a look at the command reference of the IOS version of your switch.
Once you configured source and destination port, you can capture the traffic using your laptop connected to the destination port, for example with Wireshark.
The number of source sessions can be limited, for example the 3560 supports a maximum of 2.
After the capturing, don't forget to remove this session configuration.
If your traffic happened to be passing through a router running Cisco IOS 12.4(20)T or greater, another possibility is to use the Embedded Packet Capture feature.
This feature is NOT available on switch platforms like the 3560 or 3750.
What this feature does is capture and save a small PCAP file on the router that you can download and analyze with Wireshark.
I'd like to add ELAM in the mix. ELAM is supported on PFC3 (6500, 7600).
You need to have 'service internal' enabled, but it's quite safe feature to run, I've ran it good deal in production networks and not yet once experienced negative impact.
Essentially what ELAM does is it shows you what was send for lookup processing to PFC via DBUS (Data BUS) and what did the PFC give as lookup result in RBUS (Result BUS).
For the triggers there is online help, IP_SA == IP Source Address, IP_DA == IP Destination Address, lot of others are available. IF what you want to check isn't available you can do data + mask match for arbitrary data on the first 64B.
The arbitrary trigger is bit awkward but can be lifesafer, you'll use it like this:
show platform capture elam trigger dbus others if data = DATA1 DATA2 DATAn [MASK1 MASK2 MASKn ]
Data starts from DMAC. So say we want to catch incoming MPLS stack of [0 1951], but we don't care about MAC addresses, we could do this:
show platform capture elam trigger dbus others if data = 0 0 0 0x88470000 0x00000079 0xF0000000 [ 0 0 0 0xffffffff 0xf000ffff 0xf0000000 ]
Example output might be:
7600#show platform capture elam data
DBUS data:
SEQ_NUM [5] = 0x1D
QOS [3] = 1
QOS_TYPE [1] = 0
TYPE [4] = 0 [ETHERNET]
STATUS_BPDU [1] = 0
IPO [1] = 1
NO_ESTBLS [1] = 0
RBH [3] = b000 ! port-channel hash
CR [1] = 1 ! recirculated
TRUSTED [1] = 1
NOTIFY_IL [1] = 0
NOTIFY_NL [1] = 0
DISABLE_NL [1] = 0
DISABLE_IL [1] = 0
DONT_FWD [1] = 0
INDEX_DIRECT [1] = 0
DONT_LEARN [1] = 0
COND_LEARN [1] = 0
BUNDLE_BYPASS [1] = 0
QOS_TIC [1] = 1
INBAND [1] = 0
IGNORE_QOSO [1] = 0
IGNORE_QOSI [1] = 0
IGNORE_ACLO [1] = 0
IGNORE_ACLI [1] = 0
PORT_QOS [1] = 0
CACHE_CNTRL [2] = 0 [NORMAL]
VLAN [12] = 4086
SRC_FLOOD [1] = 0
SRC_INDEX [19] = 0xC0 ! divmod64(0xc0) = 3,0, add 1 to each, 4/1 == our physical port
LEN [16] = 102
FORMAT [2] = 0 [IP]
MPLS_EXP [3] = 0x0
REC [1] = 0
NO_STATS [1] = 0
VPN_INDEX [10] = 0x7F
PACKET_TYPE [3] = 0 [ETHERNET]
L3_PROTOCOL [4] = 0 [IPV4]
L3_PT [8] = 1 [ICMP]
MPLS_TTL [8] = 0
SRC_XTAG [4] = 0xF
DEST_XTAG [4] = 0xA
FF [1] = 0
MN [1] = 0
RF [1] = 1
SC [1] = 0
CARD_TYPE [4] = 0x0
DMAC = 8843.e1de.22c0
SMAC = 0000.0000.0000
IPVER [1] = 0 [IPV4]
IP_DF [1] = 1
IP_MF [1] = 0
IP_HDR_LEN [4] = 5
IP_TOS [8] = 0x0
IP_LEN [16] = 84
IP_HDR_VALID [1] = 1
IP_CHKSUM_VALID [1] = 1
IP_L4HDR_VALID [1] = 1
IP_OFFSET [13] = 0
IP_TTL [8] = 63
IP_CHKSUM [16] = 0xBCF1
IP_SA = x.x.x ! to protect the guilty
IP_DA = y.y.y.y ! to protect the guilty
ICMP_TYPE [8] = 0x8
ICMP_CODE [8] = 0x0
ICMP_DATA [104]
0000: A0 8B 18 A5 00 39 46 35 BF 51 00 6F 3C ".....9F5.Q.o<"
CRC [16] = 0x71B3
RBUS data:
SEQ_NUM [5] = 0x1D
CCC [3] = b100 [L3_RW] ! normal L3_RW, we know it was not dropped, L2/mls policed etc
CAP1 [1] = 0
CAP2 [1] = 0
QOS [3] = 0
EGRESS [1] = 0
DT [1] = 0 [IP]
TL [1] = 0 [B32]
FLOOD [1] = 1
DEST_INDEX [19] = 0x3E8 ! same as VLAN, but not always
VLAN [12] = 1000 ! you may need to check internal vlan
RBH [3] = b111 ! again, port-channel hash
RDT [1] = 0
GENERIC [1] = 0
EXTRA_CICLE [1] = 0
FABRIC_PRIO [1] = 0
L2 [1] = 0
FCS1 [8] = 0x1
IP_TOS_VALID [1] = 1
IP_TOS_OFS [7] = 15
IP_TOS [8] = 0x0
IP_TTL_VALID [1] = 1
IP_TTL_OFS [7] = 22
IP_TTL [8] = 62
IP_CSUM_VALID [1] = 1
IP_CSUM_OFS [7] = 24
IP_CSUM [16] = 0xBDF1
DELTA_LEN [8] = 0
REWRITE_INFO
i0 - replace bytes from ofs 0 to ofs 11 with seq 'D0 D0 FD 09 34 2D 88 43 E1 DE 22 C0'. ! this is the actual L2 rewrite data, so you should obviously see DMAC and SMAC here
FCS2 [8] = 0x0
7600#
Pretty much all bigger platforms have this type of low-level captures for transit packets, which are exceptionally useful when you need to verify HW is doing what configuration says, sometimes there are software defects and it does something else than expected.
I know that in GSR you can see transit in memory, in Juniper Trio there is quite nice tool for it as well. Brocade can do it. It's quite baffling they are not documented in vendor pages.
I have asked some similar questions on serverfault, and the answers may be of use here.
