6

I have a SAN environment built on FC (Fibre Channel), which is a standalone network from the rest of the DC Ethernet network, and have only Cisco 6509s with Sup720s available for this project.

Most SAN traffic would be from VMWare with web server VMs connecting to their back-end DB.

iSCSI traffic would be kept within a given module as the SAN (HP 3Par and HP EVA) best practice is to have isolated "switches" for the redundant paths, so I don't need to worry about traffic hitting the SUP (at least not yet)-- so I guess DFC is needed -- or traffic needing to flow off the switch and over any uplinks.

My questions are as follows:

  1. Which modules (1 or 10Gb) are preferred to handle iSCSI if we migrate away from FC?
  2. Are Jumbo Frames an unquestionable requirement?
  3. Is it better for these modules to sit in the aggregation layer or access layer in case there's a future need to tie them together (as we have an L2 access layer not directly tied to one another with L2 adjacencies between pairs of access switches that traverse the agg switches).
  4. Is iSCSI the better choice over FCoE?

The most important question is the first: Which modules are preferred? Please give specific Cisco modules or model numbers in answers.

generalnetworkerror
  • 7,094
  • 6
  • 32
  • 65
  • 1
    Avoid FCoE unless someone is forcing you to use it... it's just a terrible thing to try to debug and integrate with IP services – Mike Pennington May 22 '13 at 10:09
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can post and accept your own answer. – Ron Maupin Jan 04 '21 at 22:32

2 Answers2

5

In order to run FCoE you will need something that supports it in order for it to run in a truly (or as close to as possible) lossless environment (which is required from FC/FCoE).

In terms of iSCSI requirements its a lot less demanding but you will want a line card with the deepest buffers you can depending on your oversubscription between hosts and NAS.

If for instance your SAN is connected at 10gbit and your hosts at 1gbit you will need buffers big enough to cope with the mismatch.

David Rothera
  • 2,788
  • 15
  • 20
  • Assuming all links are 1Gb, which modules are appropriate for a short list? Is there much benefit going fibre over copper such as with switching or queueing latency? – generalnetworkerror May 22 '13 at 17:33
  • As to which modules are best it depends on what chassis/sup you are running as to what you can support. In terms of copper over fibre then once you get to the nitty gritty then yes there is a difference between the speed of light and the speed of electricity however when we are talking about lengths of cable < 100M the difference is negligible. – David Rothera May 22 '13 at 22:14
  • 6509 with SUP720 (PFC3B, MSFC3). I've numbered all the subquestions I had to see if I can get answers to all of them. – generalnetworkerror May 23 '13 at 07:45
  • iSCSI still require plenty of buffer space if all ports are on the same speed. I doubt you'll have any problems on a 6500.. – pauska May 23 '13 at 22:09
5

Answering some of the generic questions:

Are Jumbo Frames an unquestionable requirement?

For FCoE it's somewhat irrelevant, as the FCoE frames are handled differently, their own default MTU is 2148 (frame size). You can do larger, but as with jumbo frames in general there's minimal benefit. An FCoE capable switch will just handle this.

Is it better for these modules to sit in the aggregation layer or access layer in case there's a future need to tie them together (as we have an L2 access layer not directly tied to one another with L2 adjacencies between pairs of access switches that traverse the agg switches).

Access really is the main place to use FCoE, replacing HBA's and edge FC switches with CNA's and 10g "converged ethernet" switches. Replacing distribution and core with FCoE can make sense when bandwidth needs justify it (or new build that has to run FC).

This is probably your sticking point actually, if you're not doing 10g to the machine you're almost certainly going to be very underwhelmed with the performance you'll get.

(Extra credit) Is FCoE the better choice over iSCSI?

Depends on what you're doing, almost certainly not, and there may be better options than either.

At a previous job we had a nice design using NFS (storage was NetApp's) where OS volumes were stored on a shared volume between the VM hosts (as normal), but data was individual NFS volumes, shared or separate as appropriate for the host, worked well in our environment as it was one group of admins managing ops from the app down to the hardware (well, its config, rack-n-stack, swaps, etc. was smarthands). Combined with NetApp's snapshots & snapmirror this made backup and restores trivial.

LapTop006
  • 1,743
  • 11
  • 24
  • +1, but the answer to the question about jumbo frames is yes... by IEEE standards anything over a 1500 byte Ethernet payload is a jumbo frame – Mike Pennington May 23 '13 at 17:03