1

In an ECMP path (consider 2 path ECMP - path1 and path2), if one of the path (say path-1) goes down, all the flows will be mapped the remaining one path (say path2).

Now, after a while, if the path1 comes back up, how will the load balancing happen? Will some the flows will be forcefully re-mapped to the recovered path-1?

For example lets take legacy Cisco routers running IOS.

Ron Maupin
  • 98,218
  • 26
  • 115
  • 191
Hemanth
  • 1,114
  • 11
  • 22

2 Answers2

2

For Cisco, as you have mentioned in your comment, EIGRP can do ECMP (it can also do unequal load balancing). It has a couple of algorithms that it uses: Per-Destination and Per-Packet.

When one route goes down in EIGRP load balancing, all the traffic switches over to the remaining path(s). When the down route comes back up, the algorithm in use will again send packets to the path that the algorithm has chosen for them.

Remember that packets are routed individually, one at a time.

For most routing protocols on Cisco IOS (even for static routing), ECMP is really handled by CEF, and you can see which method is being used with the show ip cef command. The default is Per-Destination load balancing that uses a hash to determine which path a packet should take, and each packet is forwarded independently, so when a new path comes up, that can cause packets that would have gone on an existing path to now take the new path because the hashing of the destination addressing now goes to the new path.

The other method is Per-Packet load balancing, and it simply alternates packets between the available path, so when a new path comes up, packets will also be forwarded using the new path. Per-Packet load balancing is not normally recommended because it can cause out-of-order packet delivery. TCP in the destination can compensate for that at some resource cost, but UDP cannot, and it can completely disrupt real-time protocols that use UDP.

Ron Maupin
  • 98,218
  • 26
  • 115
  • 191
  • What am curious about is, when the path1 comes back up, and if the hashing algo decides to switch certain flows back on path1, will there not be a transient period when the traffic for the same flow is distributed in two interfaces/paths. And, will it not lead to reordering issues ? – Hemanth Jun 01 '22 at 07:17
  • Second, question is, if the algo decides not the interrupt the existing flows and if the router already reached its typical max scale and doesn't expect any new flows. Then, indirectly ECMP is not being used at all after the first failure. – Hemanth Jun 01 '22 at 07:20
  • Packets are routed one at a time. Each packet received is individually evaluated by the algorithm and forwarded to the interface dictated by that. CEF is not looking at a flow in the sense you think. It is just that all packets with the same source/destination in a per-destination method will end up going to the same interface by the hash algorithm, and that is considered a flow. CEF does not care what happened to any previous packet, only the packet it is currently evaluating. – Ron Maupin Jun 01 '22 at 13:57
  • @Hemanth, see what Cisco say about [CEF Load-Balancing here](https://www.cisco.com/c/en/us/td/docs/ios/12_4t/ip_switch/configuration/guide/tceflbs.html#wp1046288). – Ron Maupin Jun 01 '22 at 14:00
0

Assuming flow-based balancing, No. Flows will not be "re-mapped" to path 1. The flows will continue to use path 2 until they complete. New flows will be distributed between path 1 and path 2.

When path 1 is restored, all existing flows will continue to use path 2. But over time, flows will be roughly distributed evenly between the two paths.

Ron Trunk
  • 66,852
  • 5
  • 65
  • 126
  • ECMP traffic distribution *may* be flow-based, but they may also be purely packet-based, depending on the exact device and its config. – Zac67 May 31 '22 at 16:02
  • True, but on Cisco devices, flow-based is the default. Updated my answer. – Ron Trunk May 31 '22 at 16:05
  • Experience tells otherwise. See final paragraphs of https://networkengineering.stackexchange.com/questions/69201/nx-os-9-3-ecmp-polarization-and-ip-load-sharing-rotate . There were 16 distinct, uninterrupted flows changing from one to two and back to one egress interface when an ECMP config parameter was changed and reverted to default. – Marc 'netztier' Luethi May 31 '22 at 20:03
  • @Marc'netztier'Luethi while I haven't compared them in a lab, I suspect there's a difference between a link coming back up and changing the configuration. The later probably resets the hash table. – Ron Trunk Jun 01 '22 at 12:09