This week at CiscoLive 2011 in Las Vegas, Cisco announced new additions to the Cisco UCS fabric architecture. In addition to the existing UCS fabric hardware, UCS customers now have a choice of a new Fabric Interconnect, a new chassis I/O module, and a new Virtual Interface Card.  The 6248UP Fabric Interconnect delivers double the throughput, almost half the latency, and more than quadruple the virtual interfaces per downlink, while the new 2208XP chassis I/O module delivers double the chassis uplink bandwidth and quadruple the server downlinks.  Last but not least, the 1280 Virtual Interface Card (VIC) server adapter provides quadruple the fabric bandwidth for UCS blade servers by delivering two active 40 Gbps paths per server.

Did I mention these new announcements were additions to the UCS product portfolio, not replacements? I’m not sure I did, so I’ll repeat it… UCS customers now have three Fabric Interconnects, two chassis I/O modules, two Virtual Interface Cards, and multiple traditional network adapters to choose from – and they’re all interoperable.

In addition to the new fabric devices, the soon-to-be-released UCS 2.0 firmware adds several features for existing and future UCS customers: Support for disjoint Layer 2 networks, UCS Service Profile support for iSCSI boot, and support for VM-FEX on RedHat KVM.

Additions to the UCS Fabric Portfolio

The UCS 6248UP Fabric Interconnect

The UCS 6248UP Fabric Interconnect, similar to the Nexus 5548 platform, provides up to 48 Unified Ports in a single Rack Unit (1 RU). Unified Ports are ports that accept either Ethernet and Fibre Channel transceiver (SFP+/SFP) modules. As such, the 6248UP can provide practically any distribution of Ethernet or Fibre Channel uplinks need to meet a customer’s design and bandwidth needs.

Don’t let the tiny package fool you… While the 6248UP is the same size as the UCS 6120 Fabric Interconnect, 1 rack unit, the 6248UP delivers double the throughput, almost half the latency, more than quadruple the virtual interfaces per downlink, and quadruple the VLAN capabilities. Here’s a comparison chart to show how the three Fabric Interconnects compare to each other.

The UCS 2208XP Blade Chassis I/O Module (Fabric Extender)

The UCS 2208XP Blade Chassis I/O module is a new choice for UCS blade customers. The 2208 doubles the uplinks and quadruples the downlinks in comparison to the existing UCS 2104 blade chassis I/O module. Using dual 2208XP modules provides up to 160 Gbps worth of uplink bandwidth per chassis and up to 80 Gbps of downlink bandwidth to each half slot server. In addition, the 2208XP provides 8 class of service queues.

The UCS 1280 Virtual Interface Card (VIC)

The 1280 VIC is the world’s first 8 port 10GE adapter for an x86 blade server providing up to 40 Gbps throughput per dual fabric. The 1280 VIC consists of two groups of 4x 10GE ports that are automatically port channeled when paired with a UCS 2208XP chassis I/O module. All eight blade server slots in the UCS 5108 blade chassis can be equipped with the 1280 VIC and all eight blade server slots will have dual active 40 Gbps paths.

Using the Service Profile in UCS Manager, a user defines the number of NICs and HBAs that should be visible to the Operating System, up to a maximum of 116 virtual interfaces (software imposed limit at FCS) per 1280 VIC. The operating system or hypervisor host “sees” the Service Profile defined NICs and HBAs and not the two 40 Gbps paths. The Service Profile allows the user to set the QoS settings and speed of the NIC.

Here is a diagram that shows the existing UCS fabric components on the left and the new additions to the UCS portfolio on the right.

Here is the full selection of UCS fabric components now available to UCS customers. Just to reiterate what I said at the beginning of this post, the new hardware announced this week at CiscoLive is an addition to the UCS fabric portfolio, not a replacement of any hardware in the portfolio.

Competitive Comparison: When compared to an HP BladeSystem, a UCS B200 blade server with the UCS 1280 VIC plus the UCS 2208 IOM would be similar to deploying a HP BL4xx series server with 2 port 10GE FlexFabric LOMs, a 2 port 10GE FlexFabric mezzanine card in Mezz 1, a 4 port 10GE FlexFabric mezzanine card (non-existent) in Mezz 2, plus 8 Virtual Connect FlexFabric modules (at $18,499 each). In other words, $147,992 per HP enclosure just for the interconnects connecting 16 HP blade servers is several times the cost of two UCS chassis with dual 2208 I/O modules connecting 16 UCS blade servers.

Support for Chassis Uplink Port Channeling and Server Adapter Port Channeling

In addition to the increased bandwidth, one of the most beneficial features of the next generation UCS hardware is the ability to port channel both 2208 IOM chassis fabric uplinks and the server adapter port uplinks on the 1280 VIC. The major benefit of this feature is better distribution of server traffic load across more links – across both the server ports and the fabric uplink ports.

For the fabric uplinks, the user can choose whether to use discrete mode (today’s behavior of pinning blade slots to specific chassis uplinks) or port channel mode (creating a single virtual chassis uplink that’s shared for all blade servers in the chassis). In Port Channel mode, users can use 4, 8 or 16 uplinks depending on the amount of bandwidth per chassis needed.

Fabric uplink port channels require both the 6248 fabric interconnect and the 2208XP I/O module.

At the server port level, port channeling between the 1280 VIC and the 2208 is configurationlessable (SIC – I’m working on my presidential credentials by making up words). In other words, they are channeled automagically so that, in effect, the 1280 VIC behaves as a 2 port 40 Gbps server adapter. Server adapter port channels require both the 2208XP I/O module and the 1280 VIC server adapter.

NICs or HBAs (or VMs when using VM-FEX) created in the Service Profile have their traffic load balanced across all ports on the fabric to which they’re pinned. This means that any single NIC has access to up to 40 Gbps of bandwidth for transmits or receives (39.6 Gbps per the throughput test, to be exact). The traffic is load balanced across the channel based on MAC, IP, and TCP/UDP information so that multiple conversations for any particular NIC/HBA/VM is load balanced across multiple 10GE ports. Either side A or side B (diagram below) can burst up to 40 Gbps each, but, simultaneously, cannot exceed the PCIe Gen 2 x16 bus limitation of 64 Gbps. I can already hear the competitive FUD coming… If you need more than 64 Gbps of bandwidth in a half slot server or more than 128 Gbps of bandwidth in a full slot server… PLEASE let me know. I’d really like to buy you a beer and find out what you’re doing with all that bandwidth.

Backwards Compatibility and Interoperability

As you’ve probably already figured out, it’s very important to me to emphasis that Cisco’s announcement this week is about NEW ADDITIONS to the UCS fabric portfolio. I also want to specifically point out that the new and existing UCS fabric products are backwards compatible and interoperable (to the least common denominator feature). For example, if you own 6120s, 2104s, and M81KR CNAs today and decide to replace the 6120s with 6248s… it’s supported and works just fine. You want to just replace the 2208s and keep the 6120s/6140s and M81KRs? Great! We’ll support it. Would you just like to start deploying the new 1280 VIC but keep the existing 6120s and 2104s? Perfect! Supported! Would you like to deploy your new chassis with the 2208 but not retrofit your existing chassis and have all of them connected to the same pair of 6120s, 6140s or 6248s? Excellent! Supported! Last but not least, we’ll support you doing mixed combinations of 6120s/40s + 6248s, for example, during upgrades but not as a permanent production deployment.

One thing to keep in mind, though, is the features that require the new hardware. For example, without both 2208s and a 1280 VIC, you can’t port channel the ports on the server adapter. Or, without both the 6248s and 2208s, you can’t port channel the chassis fabric uplinks.

UCS 2.0 Platform Software Features

In addition to the new hardware discussed above, the UCS 2.0 firmware will add several platform or software features that will be available across both existing and future UCS deployments.

Support for Layer 2 Disjoint Networks in End Host Mode

UCS adds support for flexible VLAN configurations on Fabric Interconnect uplink ports while using End Host Mode. This feature provides for extremely flexible UCS deployments to support almost all combinations of upstream network configurations.

iSCSI Boot Support in UCS Service Profile

iSCSI boot is now supported in the UCS Service Profile for several of the UCS Converged Network Adapters – including the M81KR, 1280 VIC, and others. In addition to NICs and HBAs, users can now create iSCSI NICs for UCS servers. Unlike other solutions (ahem, HP FlexFabric), the UCS fabric is more flexible and allows the creation of both FC HBAs, iSCSI NICs, and Ethernet NICs all on the same server.

Support for VM-FEX in RedHat KVM

Cisco’s Fabric Extender (FEX) technology can be deployed at several levels (see graphic below) – Rack FEX (like a Nexus 5000 + 2000), Chassis FEX (UCS 6248 + 2208), Adapter FEX (M81KR or 1280 VIC) and VM-FEX (M81KR/1280 VIC plus hypervisor integration). VM-FEX allows using the UCS Virtual Interface Cards (M81KR or 1280 VICs) as remote switch line cards (FEXs) inside of a hypervisor and directly assigning an independent NIC to each and every Virtual Machine. This allows each VM to have its own logical switch port on the upstream Fabric Interconnect – complete with its own configuration, statistics, etc.

VM-FEX deployments have always been only for VMware environments until UCS 2.0. VM-FEX is now supported for RedHat KVM environments also.

Logical Operation of Cisco's Four Deployments of Fabric Extenders

Frequently Asked Questions

Q1: With the new announcement, does this mean UCS customers have to go through a forklift upgrade?
A1: Absolutely not. If a user doesn’t need the additional bandwidth or any of the other new features (like port channeling), none of the new hardware is needed for future deployments. A user is more than welcome to continue buying UCS 6120 or 6140 Fabric Interconnects, UCS 2104 IOMs, and UCS M81KR VICs. Future UCS software releases will support existing and new hardware for the foreseeable future.

Q2: When can I get it?
A2: General availability is scheduled for Q3CY2011 (3^rd quarter of this year).

Q3: Do I have to buy a new chassis to use the new 2208 IOM or the new 1280 VIC and get increased server bandwidth?
A3: Absolutely not. Cisco planned for the future and planned on providing customers with investment protection… the UCS 5108 chassis mid-plane provides support for 4 lanes to each of the eight server slots. Each lane is 10G-Base-KR rated. As such, inserting a 2208 IOM into an existing chassis immediately provides 4x 10GE downlinks to each of the eight server slots. Adding a UCS 1280 VIC to an existing server then allows the server to utilizes all eight of the 10GE downlinks.

Q4: Do I have to buy new servers?
A4: Absolutely not. The 1280 VIC works in all shipping servers today.

Q5: Are you End of Life-ing (EOLing) the existing UCS fabric hardware?
A5: Not only no, but _ _ _ _ no. J Choices folks, choices.

Q6: Do I have to have new hardware to get Port Channeling capabilities?
A6: Yes. If you want port channeling between the chassis and the fabric interconnect, you’ll need the UCS 6248UP and the UCS 2208 IOM. If you want port channeling between the IOM and the Server VIC, you’ll need the UCS 2208 IOM and the UCS 1280 VIC.

Q7: I have the 6120 or 6140 Fabric Interconnects (or the M81KR VIC or the 2104 IOMs). Can I use the new software features like disjoint L2, iSCSI boot, etc?
A7: Yes. Once the UCS Manager 2.0 software upgrade releases in Q3, you can upgrade your existing UCS environment and have those new software features (no new hardware requires and no software license fees).

If Charles Dickens were to write a book about it, it would begin like this:

“It was the best of predictions, it was the worst of predictions, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief from a new comer, it was the epoch of incredulity from an incumbent…”

Of course, I’m referring to the UCS market share predictions from Cisco’s CEO, John Chambers compared to the UCS statements made by HP’s senior management. Let’s summarize each side’s statements before we dive into the hard facts.

_

Prediction #1: John Chambers, Cisco’s CEO, on September 14^th, 2010 said: (1)

“UCS has already taken the #3 market share spot in US/Canada for x86 blade servers.”
-and-
“Cisco expects UCS to be 50% the market share of the #2 competitor for the worldwide x86 blade server market within the next 2 quarters.”

Prediction #2: Randy Seidl, HP Senior VP, on April 26^th, 2010 said: (2)

“A year from now the difference is (Cisco) UCS (Unified Compute System) is dead… ”

Extra Credit: Leo Apotheker, HP’s CEO, on March 16^th, 2011 said: (3)

Gallant:         “Speaking of Cisco in the server market, is it a threat or an annoyance to HP?”
Apotheker:   “Neither.”
Gallant:         “Can you expand?”
Apotheker: “We hardly ever see [Cisco UCS].”
Gallant:        ”They claim that sales are growing pretty rapidly of the UCS system, but you’re not                              seeing them in competitive situations?”
Apotheker: “They must be selling on planet Zircon.”

_

Yesterday, May 24^th, 2011, IDC announced the results of the Q1 2011 Worldwide Quarterly Server Tracker. (4) The results clearly proved the credibility of Mr. Chambers while doing exactly the opposite for HP’s Siedl and Apotheker. Here is a graphical breakdown of the results:

_

_

The chart above (5) shows x86 Blade Server Market share for both Worldwide and US sales. These results validate both of John Chambers’ statement that “Cisco has taken the #3 market share spot in the US/Canada for x86 blade servers“ and that “Cisco expects UCS to be 50% the market share of the #2 competitor for the worldwide x86 blade server market“.

The most interesting, and relevant, market share numbers for me are the US numbers. Reason being, this is the market that my team works in on a daily basis. You could say that we just received our report card and it’s not too shabby… after only two years, my team is responsible for close to $1 out of every $5 worth of x86 blade server factory revenue in the United States. Lastly, we were able to outpace IBM, who has been reporting x86 blade revenue since 2002, and Cisco UCS is now the #2 vendor for x86 blade server factory revenue in the US.

_

Given the predictions above from John Chambers vs. HP’s senior management and the IDC’s market share announcements, I have the following comments/questions:

• Our fearless leader, John Chambers, was spot on with his prediction. If he ever predicts the rapture, I’ll probably pay attention.
  _
  
• HP’s Siedl said UCS would be dead by April 26^th, 2011. Instead, less than one month later, IDC reports that Cisco UCS leapt to either the #2 or #3 position in x86 blade revenue in multiple markets. That’s impressive results for a “dead” product.
  _
  
• HP’s Apotheker said “We hardly ever see UCS”. How exactly do you NOT see the #2 blade server vendor in the US? Is it a case of head-in-sand syndrome, fingers-in-ears disease, or is he just saying what he wishes was true while clicking his ruby slippers together three times? Who knows…

_

In summary, I leave you with the chart below (6)… the trending lines for worldwide x86 blade market share. Cisco is clearly gaining at the expense of HP and IBM.  The UCS Naysayers’ favorite twitter hash tags for the past several months has been #NumbersDontLie, #HardFactsDontLie, and #RealIndustryLeaders.  For some reason, they’re strangely quiet these days… <tongue in cheek>

_

Sincerely,

M. Sean McGee, UCS Cheer Squad

P.S. As always, these thoughts are solely my own.

Sources:

1. http://blogs.cisco.com/ar/john_chambers_discusses_ucs_market_share__wow/
   
2. http://www.crn.com/news/channel-programs/224700514/q-a-hp-sales-chief-details-plan-to-take-down-cisco.htm?pgno=4
   
3. http://www.networkworld.com/community/blog/hp-beat-crap-out-cisco-others& http://www.networkworld.com/news/2011/031611-hps-leo-apotheker-were-heading.html
   
4. http://www.idc.com/getdoc.jsp?containerId=prUS22841411
   
5. http://www.flickr.com/photos/cisco_pics/5328636375/lightbox/
6. http://www.slideshare.net/MeredithSabye/ucs-impact-of-innovation

Recently, a couple of Twitter pals of mine, @veverything and @ChrisFendya, discussed VMware KB article 1013094 that stated an unsupported teaming type for the Cisco UCS B200 blade server. Their discussion made me realize that there wasn’t an easy-to-find resource for Cisco UCS customers that needed to know “OS Team Types Supported Per UCS Network Adapter” and “Unsupported Team Types”.

Below you will find both of these resources. I’ve listed a chart per Cisco UCS B-Series Network Adapter that shows the teaming/bonding types that will work per Operating System. Lastly, you’ll find a chart showing the team types that will not work.

Please keep in mind the following:

1. Just because I say it works, doesn’t mean Cisco, Intel, Broadcom, QLogic, Emulex, Linux, VMware, Microsoft, etc. supports it. I’m listing the teaming/bonding types that will work based on my experience and based on what I know about NIC Teaming and the UCS architecture.
   ___
2. This article isn’t intended to teach you which teaming/bonding type to use and I’m not necessarily making recommendations on the use of specific team types. I’m simply listing team types that should work if you choose to use them. I have another article queued up that discusses NIC Teaming/bonding in detail and helps you decide which one to use within a UCS environment. That’s coming in the near future. In the meantime, if you are really jonesing to learn more about NIC Teaming in general, you are welcome to read through a lengthy whitepaper I wrote in a former life HERE. While it’s specific to HP NIC Teaming, many of the concepts are the same across Intel, Broadcom, etc.
   ___
3. When it comes to NIC Teaming, Hyper-V is a special case. Since network redundancy is left to the NIC Teaming vendor (vs. the VMware model where a vSwitch handles network redundancy), it’s important to use an OEM NIC Team type that is “VM aware”…or a team type that will GratARP the VM’s MAC address after a NIC failover. If a NIC Teaming vendor (e.g. Emulex, QLogic, etc.) doesn’t specifically state that they support this capability in their teaming driver for Windows, I list “no recommendation”. In other words, I can’t recommend their team type under Hyper-V because I don’t know if it will properly handle quick failover of VM MAC addresses.
   ___
4. How to read the charts below: The columns show teaming types/bonding modes per OS. The rows represent the capability that the team type provides. The capabilities (on the right side of the chart) can be “Fault Tolerance” only, Fault Tolerance + Transmit Load Balancing (TX LB), or Fault Tolerance + TX LB + Receive Load Balancing (RX LB).  I list the capabilities like this since the capabilities are additive (e.g. there’s no such think as an RX Load Balancing team that doesn’t provide Fault Tolerance).
   __
5. Cisco UCS Fabric Failover is hardware-based NIC Teaming and does not require any type of OS driver or awareness. It is one of the many valuable and unique features of the Cisco UCS architecture. For a more in-depth discussion of how it works see this article written by my esteemed colleague, Brad Hedlund.
   __
6. The supported/unsupported team types are not blade model dependent. While the VMware article mentioned above specifically states the B200, in reality, it applies to all B-Series servers.
   ___

Enough of my yakking. On to the purpose of this post…

NIC Teaming For Cisco UCS B-Series Network Adapters

(M81KR Palo)

Sources: Palo Teaming on Windows: TBD; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

(M72KR-E Emulex)

Sources: Emulex Teaming on Windows: http://bit.ly/khTIHX; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

(M72KR-Q QLogic)

Sources: QLogic on Windows, Linux and VMware: http://bit.ly/mJU6Nx & http://bit.ly/lJxiNy; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

(M71KR-E and M71KR-Q)

Sources: Intel Teaming on Windows: http://intel.ly/lL0b5w; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

(82598KR-CI and M61KR-I)

Sources: Intel Teaming on Windows: http://intel.ly/lL0b5w; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

(M51KR-B Broadcom)

Sources: Broadcom Teaming on Windows: http://bit.ly/jwzGBE; Linux: http://bit.ly/kHGwHI; VMware: http://bit.ly/luVPnK

* Not supported according to Intel, not Cisco. Team type may be supported on full width servers for ports connected to same Fabric Interconnect.

I love a box of chocolates for one simple reason – it provides a variety of bite-sized samples.  I get to taste several small samples and quickly determine which ones interest me and which ones don’t.  Well, why not package technical product discussions the same way?  I mean, why not provide several time-sized sample discussions of the technical advantages of a product or solution and let the consumer quickly determine which ones interest them or not?

Allow me to introduce you to the first round of the “Cisco UCS Advantage Series”…or, as I refer to it, a box of UCS chocolates.  This series was developed by Cisco to give our customers a quick sampling of just a few of the technical advantages of Cisco’s Unified Computing System solution.  Go on…sample a few.  They’re low calorie.

Special recognition to Dan Hanson (@danhanso) and his team of exceptional UCS engineers for producing this series.

__We’ve now posted all of these videos in one place on Cisco.com. I recommend visiting this location for the most up-to-date list of videos:

http://www.cisco.com/en/US/prod/ps10265/ucs_advantage_video_library.html

Introduction to the Cisco UCS Advantage Series

UCS Server: Service Profiles and Templates

UCS Server: Server Pre-Provisioning

UCS Server: BIOS Policies

UCS Server: RAID Policies

UCS Server: Firmware Policies

UCS Server: Server Pools and Qualification Policies

UCS Server: Maintenance Policies

UCS Server: High Availability During Upgrades

UCS Server: Monitoring with BMC BPPM

UCS Server: Microsoft Hyper-V on UCS

UCS Server: Extended Memory Technology

UCS Server: Organizations and Roles

UCS I/O: Adapter Templates

UCS I/O: Network Interface Virtualization

UCS I/O: Adapter Fabric Failover

UCS I/O: Extend the Network to the Virtual Machine with VM-FEX

UCS I/O: Traffic Analysis of All Servers

UCS I/O: Ethernet Switching Modes

UCS I/O: Fibre Channel and Switch Modes

UCS I/O: FC Port Channels and Trunking

UCS Infrastructure: Lights-Out Management

UCS Infrastructure: Easy VM-FEX Deployment

UCS Infrastructure: Blade and Rack-Mount Management

UCS Infrastructure: Server Power Grouping

UCS Infrastructure: Manager Platform Emulator

UCS Infrastructure: Server Power Grouping and Prioritization (added May 6th, 2011)

UCS Infrastructure: Cisco Developer Network and Sandbox

Santa came early this year for Cisco UCS customers. Today, Cisco released UCS firmware version 1.4(1).  This release is the single most impressive feature enhancement release I’ve seen in all my 11 years of working on blade servers.  Allow me to walk you through this list of new features and provide a deeper dive into some of the details behind each one.

Note: The Release Notes are posted here: http://www.cisco.com/en/US/partner/docs/unified_computing/ucs/release/notes/OL_24086.html

Server Platform and Management Enhancements:

• Support for new B230 server blade
  See “The Cisco UCS B230 – the Goldilocks Blade Server” for more details on this new server blade.
  
  __
  _
• 
  UCS C-Series Rack server integration into UCS Manager – Unified Management for the entire UCS portfolio
  

  Yes, you read me right – Cisco is the 1^st server vendor to integrate rack server management into the “blade server and blade chassis management” management interface so that a single management tool configures and monitors both your blade and rack servers. This initial release includes support for the C200, C210, and the C250 Cisco UCS rack servers. Support for additional Cisco UCS rack servers will be added in the near future.

  UCS Manager features extended to C-Series Rack servers include: Service Profiles, Service Profile migration between compatible B-Series and C-Series servers, automated server discovery, fault and monitoring, firmware updates, etc.
  _
  _
  _
  _
  _
  _

• Chassis and multi-Chassis power capping for UCS B-Series Blade Servers
  

  Cisco has enhanced the facility manager’s control over UCS blade server power consumption by adding Group Level Power Capping, Dynamic Intra-chassis power redistribution, and Service Profile Priorities. Within the data center, power should be distributed to a blade chassis or groups of blade chassis, not to individual blade servers. If a server is “statelessly” moved using a Service Profile from one chassis to another, a statically defined power cap per server is mostly useless. What if you moved a bunch of servers with static power caps (in watts) to the same power distribution unit (PDU) – the sum of which exceeded the PDUs capability? Not very intelligent or practical, huh? A server’s power cap needs to be relative to the power cap of the blade chassis or blade chassis group so that the infrastructures maximum power draw can be guaranteed.

  Cisco’s approach allows for facilities administrators to define “power groups”, comprised of one or more physical blade chassis, and a “power cap” for the group based on the size of the power circuit that the chassis are connected to in the datacenter.  Power Groups, and their associated power cap, are decoupled from the physical blade server. Server administrators use Service Profiles to assign individual workloads (OS + Applications) a “Priority” relative to other workloads contained within the same power group. A workload can move between power capping groups and maintains its priority. The facilities manager never has to worry about power exceeding the define power cap per group no matter how the server administrators move servers/workloads around. This feature brings “change ready” and “facilities on demand” to UCS customers.

  To summarize: Cisco decoupled the power problem (facilities power cap) from the workload importance (priority). Facility admins control the infrastructure’s power cap and the server admins control a workload’s (server) priority within that infrastructure.

• Blade Chassis support increased to 20 chassis in UCS Manager
  

  You can now deploy, configure and manage up to 160 physical blade servers under a single UCS Manager interface. In other words, a single UCS Manager interface now replaces the equivalent of up to 10x HP Onboard Administrator interfaces, up to 10x Virtual Connect Manager interfaces, Virtual Connect Enterprise Manager, and most of the popular functions of HP Systems Insight Manager.

  Cisco UCS now only needs 3 infrastructure management IP addresses for 160 servers compared to HP’s need for up to 7 management IP addresses for each chassis – or 70 management IP addresses for 160 servers (up to 50 IPs if using FlexFabric).

  

  _
  __
  _

• Service Profile Deployment Scheduling
  Ever want to make one or more changes to one or more Service Profiles but had to wait to do all the work until your change window began? No more. With Service Profile Deployment Scheduling, you can queue up Service Profile changes for one or more servers, doing the work up front, and then schedule the changes to take effect during the next change window.
  

  Highlights are:

  • Service Profile changes to hardware are scheduled for future maintenance windows instead of taking effect immediately.
    _
  • Scheduling is centrally managed via Maintenance Policies
    _
  • Hardware resources are held/reserved until Service Profile deployment_
    _
    _
    _
    _
    _
    _
    _
    _
• IP address for CIMC (remote KVM) added to UCS Service Profile
  To further the concept of “server statelessness“, Cisco has added a CIMC (remote KVM) IP address to the UCS Service Profile. Now, the physical server blade owns a CIMC IP address and, optionally, the Service Profile owns a CIMC IP address. If the additional CIMC IP Address in the Service Profile is used, the server admin can reach the KVM console no matter which physical blade the Service Profile is assigned to.

  Prior to this feature, if Service Profile A moved from Physical Server Blade 1 to Physical Blade Server 2, the CIMC IP address changed and required the server admin to track down the new KVM console IP address. By using this optional feature, the server admin can always use the Service Profile owned CIMC IP address (example below: 10.21.32.46) and will always reach UCS Service Profile A no matter which physical server it’s assigned to.
  

  

• Service Profile “pre-flight” checks (impact analysis BEFORE committing changes)
  

  This feature allows a customer to run a pre-flight check on a physical server before attempting to apply a Service Profile to it. In cases where the Service Profile requires certain hardware (like a Cisco VIC “Palo” CNA), the pre-flight check will alert the server administrator BEFORE going through Service Profile assignment. In addition, the Service Profile will “remember” the hardware it was associated with and if the new hardware has meaningful differences, UCS Manager will warn the user.
  _

• SNMP GET support for ALL UCS components
  

  SNMP query (GET) support has been extended to cover all UCS components – Fabric Interconnects and Fabric Extenders, Blade Chassis, Blade Servers, and Rack Servers. The new 58 MIBs are available here: http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml
  _

• Syslog enhancements
  

  In addition to supporting categorization for different components and adding additional filtering capabilities per syslog server destination, UCS’s syslog functionality has also been enhanced to provide more descriptive syslog messages.
  _

• UCS 6100 Licensing Enforcement and Warnings
  

  Well, some may claim this is a feature enhancement for “Cisco” rather than for the “customer”. In reality though, this is a nice feature for customers that honestly want to stay in compliance with Cisco licensing requirements (e.g. Fabric Interconnect port licensing). A new GUI based licensing management interface and licensing warning messages are part of the “usability enhancements” of this feature.

  • UCS Manager can assign or revoke licenses
  • Port licenses are based on used fixed port count (no need to assign license to ports individually)
  • Expansion ports (GEM – Gateway Expansion Modules) don’t require port licenses
    _
• UCS Manager Usability Enhancements
  Several UCS Manager usability enhancement are also included in release 1.4(1). These include:
  

1. Firmware Upload using local file system instead of FTP/TFTP
   Yes, we finally added it! Upload your firmware directly to UCS via your local desktop! FTP/TFTP are no longer required. This feature is especially useful for demo or lab environments where FTP/TFTP servers are not readily available.

   

2. Enhanced UCS Firmware Descriptions
   UCS Manager now provides better descriptions for firmware images. The description allow you to quickly identify which hardware product the firmware image is intended for.

   

3. Service Profile Aliases
   Server administrators can now add a free-form (any character is legal) description to a Service Profile for quick identification of the Service Profile object they want to work on. The description is displayed at the end of a Service Profile name on the Server tab in the left panel of UCS Manager._

   _
   _
   _
   _
   _
   _

• Enhanced integration with Microsoft Active Directory
  

  UCS Manager now supports the ability to map Active Directory (AD) groups to user roles within UCS Manager. UCS Manager looks up AD user groups and allows the UCS Domain admin to assign UCS roles to the AD User Groups. This eliminates the per-user role assignment within UCS Manager that was required before 1.4(1).
  _
  _
  _
  _
  _
  _
  _
  _

• Simultaneous Support for all authentication methods (local, TACACS+, RADIUS, and LDAP/Active Directory) in UCS Manager
  

  When UCS initially launched in 2009, it supported authentication via local users, TACACS+, RADIUS, or LDAP/AD servers. However, UCS Manager only supported a single authentication method at a time. With release 1.4(1), UCS Manager now supports all authentication methods simultaneously.

  A user selects their authentication domain during login.

  _

• Support for authentication to multiple Active Directory Domains
  

  In addition to support for multiple authentication methods discussed above, UCS Manager also now allows for authenticating against multiple Active Directory domains. This is a key feature for supporting multi-tenancy environments with multiple AD Domains or key to supporting environments with separate AD Domains per region.

• Multi-user CIMC Enhancements
  

  Cisco UCS’s remote KVM feature, called Cisco Integrated Management Controller or CIMC, now provides enhancements for multi-user access. The first user accessing the KVM gets RW priviledges to the session while subsequent users are granted permission by the first user to join as read-only (by default). It also includes the ability for the UCS admin user to force termination of the KVM for a user.

  ___
  _
  _

• UCS “Server Packs”
  Support for server and adapter hardware can now be delivered independently of support for the infrastructure components. This allows customers to load supporting firmware packages for new server hardware and adapter hardware without having to upgrade their Fabric Interconnect or UCS Manager software at the same time.

  Server and Adapter Packs, or bundles, will be provided anytime new server or adapter hardware is released. These Server Packs or Adapter Packs can then be loaded into the “infrastructure” to provide immediate support of the new server or adapter hardware without upgrading the infrastructure firmware in UCS Manager or the Fabric Interconnects.

  _
  _
  _
  _
  _

Ethernet and Fibre Channel (FC) Networking Enhancements:

• New Fabric Interconnect Port Types: Ethernet Appliance, FC Target, and FCoE Target
  In addition to Ethernet and FC monitoring ports covered later, UCS release 1.4(1) introduces three new port types for the Fabric Interconnect uplinks:

  Ethernet Appliance: When a Fabric Interconnect uplink is configured as an Appliance port, a user can connect several types of “appliances” directly to the UCS Fabric Interconnects. Such appliances could be NFS/NAS/iSCSI storage targets, Security Appliances, Nexus 1010 appliances, etc. You can even use port channeling to increase the “pipe” to the appliance if needed. Prior to version 1.4(1), appliances could be directly connect to the Fabric Interconnect but only when “switch mode” was used. Release 1.4(1) adds support for “appliances” in “End Host Mode” also. This is a key feature since Cisco’s usual recommendation is to use End Host Mode instead of Switch Mode.

  FC and FCoE Target ports: UCS users can now direct connect FC targets and FCoE targets to UCS Fabric Interconnects. While the default zoning config is all that is supported for now, the Fabric Interconnect will inherit the zoning configuration from an upstream MDS switch (if necessary).

  
  
  __

• Support for 1024 VLANs per Fabric Interconnect
  Up to 1024 VLANs per Fabric Interconnect are supported. Prior to release 1.4(1), only 512 VLANs were supported per Fabric Interconnect.
  _
• SPAN (port monitoring) support for both Ethernet and Fibre Channel
  Cisco has added support for SPAN to release 1.4(1). SPAN, or Switch Port Analyzer, provides selective traffic mirroring from a source (one or more server ports or vNICs) to a destination (Ethernet or Fibre Channel uplink). Up to four simultaneous sessions are supported – two on each Fabric Interconnect. In addition, both the LAN and the SAN administrators have the ability to define their own SPAN session via the LAN or SAN tab, respectively, in UCS Manager.
  _
  In addition to traditional monitoring (NIC ->Ethernet Analyzer or HBA->FC Analyzer), users can now monitor both vNIC and vHBA traffic when SPANed to an ethernet destination uplink.  Also, when using Cisco Palo CNA with interface virtualization, each individual vNIC can be monitored/SPANed separately.  If the vNICs are used with Passthrough Switching in VMware, this allows monitoring traffic from every individual VM. When Palo is used with a bare metal OS install, this feature allows each NIC port presented to the OS to be monitored independently.

  
  _
  _

• Private VLAN (Isolated Access Port) Support
  Without Private VLAN (PVLAN) support, network administrators would be required to use VLANs to maintain Layer 2 separation between physical or virtual servers. This method of secure separation doesn’t scale well. Instead, Private VLANs can be used to enforce a Layer 2 boundary between physical or virtual servers assigned to the same VLAN. UCS release 1.4(1) provides support for isolated PVLAN support for physical server access ports or for Palo CNA vNIC ports.

  Example: Below, all hosts (bare metal and two VMs) are all in the same VLAN A and assigned IP addresses in the same subnet. All three hosts and communicate with the same external devices external to UCS, however, none of the three hosts can communicate with each other. They are all separated/isolated from each other at Layer 2.
  _

• FabricSync
  Instead of reinventing the wheel and spending the time to define this feature, I’ll refer you to a blog (link below) written by an esteemed colleague of mine named Brad Hedlund. Brad explains how Fabric Failover works for ‘implicit’ MAC addresses and how Fabric Failover works with ‘learned’ MAC addresses (as of release 1.4(1)). The synchronizing of ‘learned’ MAC addresses between Fabric Interconnects is referred to as “FabricSync” now (even though Brad doesn’t use the ‘FabricSync’ feature name in his blog article).

  http://bradhedlund.com/2010/09/23/cisco-ucs-fabric-failover/

  P.S. This is one of my favorite new features because I helped come up with the name – FabricSync. <insert humility here> J

• Support for FET-10G FEX transceivers
  UCS Fabric Interconnects and Fabric Extenders (FEX) now support FET-10G FEX transceivers. FET stands for “Fabric Extender Transceiver”. These transceivers are based on multimode fiber and support distances of 25 or 100 meters between the FEX and Fabric Interconnect. In addition, the FET-10G transceivers are low power (~1W per transceiver) and extremely low latency (~.01 ms).
  _
• Management Interface Monitoring and Failover
  VIP, or Virtual IP, is the equivalent of a cluster IP address for UCS Manager. The VIP IP address needs to be available via whichever management port is available on either Fabric Interconnect. If the management port on Fabric Interconnect A is the ‘active’ port and it fails, the VIP needs to failover to the management port on Fabric Interconnect B so that users can still access UCS Manager.

  As of this release, 1.4(1), Cisco has augmented VIP availability so that the management ports are actively monitored for connectivity to a Pingable ARP target, a Pingable Gateway target, in addition to link failure. After a failure, the VIP address is failed over to the new active management port. Also, the CIMC (remote KVM)/IPMI/SSH sessions to each blade server are also failed over to the new active management port.

  Note: After a failover of the management instance you will need to re-authenticate to the new instance.

  
  _

• FC Port Channeling on FC uplinks
  Port Channeling is now supported on Fibre Channel uplinks. The main benefit of FC port channeling is that host logins assigned to a failed FC uplink in a port channel can be quickly moved to another FC uplink in the same port channel without re-logging the host into the upstream fabric.
  _
• FC VSAN Trunking on FC uplinks
  Fibre Channel VSAN trunking is similar to VLAN trunking on an Ethernet port – a single physical port (or port channel) can carry multiple VSANs.
  _

In summary, this new release by the Cisco UCS development team absolutely blew my socks off. The ability of our development, test, beta, services, support, and field sales teams to work together to once again deliver a whole slew of new features based on customer requests would be impressive even for one of the legacy server vendors – much more so for a team that is completing their second year of shipping server products. No existing or potential UCS customer should doubt Cisco’s commitment to this product line or doubt the technical ability of our people. They’re top-notch and they’ve out done themselves once again.

“Products looking for customers” are very similar to “people looking for jobs” – the need for references that vouch for promised capabilities is a must. One or two people saying you can walk on water or turn water into wine probably won’t convince anyone that you told the truth on your resume. However, show up with 50+ quotable references from respectable sources and your prospective employer realizes you’re the real deal.

Well, Cisco’s entry into the server market is the real deal and you can believe everything on Cisco’s server resume. Just for starters, here are just 50 or so references from respectable sources that I’ve come across recently. Unfortunately, I can only list references with public statements regarding their UCS experiences. In other words, this list is just the tip of the iceberg of UCS deployments.

So, the next time a doubting Thomas questions why you hired Cisco as your server vendor, provide them with this list of references.

______________________________________________________________________

Cisco UCS Enterprise/Commercial/SLED Customers

Corporate Express:
“According to another Corporate Express executive, Chad Singleton, much of the reason the company picked the Cisco platform was its high degree of integration with VMware. He noted Cisco’s server rivals didn’t offer the same level of integration yet.”
http://delimiter.com.au/2010/10/26/corporate-express-deploys-cisco%E2%80%99s-ucs/
http://www.computerworld.com.au/article/365623/corporate_express_deploys_cisco_infrastructure/

City & County of San Francisco:
Delivers cloud services with Cisco’s Unified Computing System and saw a 300 to 400 percent improvement in service to their city and county customers:
http://www.serverwatch.com/trends/article.php/3906216/Government-Bodies-Embrace-Cloud-Computing.htm

Examworks:
Saved over US$200,000 in annual IT costs using Cisco’s UCS, avoided US$333,000 in costs for new desktop computers and was able to reduce the IT support staff to employee ratio to 1 to 250:
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/case_study_c36-580410.pdf
http://www.youtube.com/watch?v=lEP6RXzG_kA

Hay Group:
Accelerated the time to deploy new services from weeks to minutes leveraging Cisco UCS over stand alone server solutions:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/case_study_c36-623712.html

ING:
The choice for Cisco and the UCS platform was taken based on the performance of the blade platform…:
http://translate.google.com/translate?js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&sl=auto&tl=en&u=http://www.automatiseringgids.nl/artikelen/2010/43/ing-zet-mes-in-datacentra.aspx

Molina Healthcare:
Cisco UCS allows Molina to efficiently scale their data center resources and provides 20% cost savings:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/case_study_molina_datacenter.pdf

Moses Cone:
Reduced the time to implement a VMware ESX host from two days to one hour and saved 96 hours on server configuration with their initial install:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/case_study_c36-588298.html

Nighthawk Radiology:
Reduced physical server count by more than 50%, Lowered per-chassis cabling requirements by a factor of five, gained the ability to provision servers in 15-20 minutes with UCS.
“With the Cisco Unified Computing System and Nexus solutions, we’ve experienced extremely high availability, coupled with lower costs and simplified managment.”
“Decreased the time needed to deploy and manage servers by 80%.”
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/case_study_c36_604664.html
http://www.youtube.com/watch?v=lEP6RXzG_kA

Monsanto Company:
“[Monsanto] is working with BMC Software and the Cisco Unified Computing System to achieve its goal of 70 percent server virtualization as it begins deploying a cloud-based infrastructure that will be the cornerstone of Monsanto’s IT.”
http://www.virtualization.net/1004-cloud-strategy-with-bmc-software-and-cisco-ucs-agriculture-company/

Pitt Ohio Express:
Virtual Machines can now be restored 5 minutes to an hour compared to 8 to 12 hours with legacy servers, cabling connections to each server dropped from 40 to 6 and the 20 fold increase in staff productivity allows them to focus on more strategic, proactive projects:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/Cisco_Pitt_Ohio_CS_090810.pdf

State of New Mexico:
Will leverage Cisco UCS to deliver infrastructure-as-a service to dramatically reduce both IT and energy costs:
http://gcn.com/articles/2010/09/28/state-of-new-mexico-private-cloud.aspx

Slumberland:
Provisioned 28 logical servers in 3 days, spending 74% less compared to their previous architecture, reduced management costs per server from $1574 to $80 and the efficiencies provided by the Cisco UCS freed up 10 hours weekly, per person, to spend on tasks that add business value:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/case_study_C36-587967_ns224_Networking_Solutions_Case_Study.html

TASER:
Cisco UCS enabled TASER to deploy a new service called Evidence.com, to help law enforcement agencies to increase operational excellence and administrative efficiency. Planning to completion of the infrastructure took just 98 days, they saved $900,000 in up front capital costs and reduced annual energy costs by $37,000.
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/Service_Provider_case_study.pdf
http://www.youtube.com/watch?v=lEP6RXzG_kA

Tele Sistemi Ferroviari:
Expects revenues from this new Infrastructure-as-a-Service to reach 2-3% of total revenues within its first year, while reducing cabling costs by 70%, new server provisioning costs by 25% and power consumption by 60%:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/TSF_case_study.pdf

Travelport :
Was able to reduce their overall data center footprint while reducing the time to deploy both virtual and non-virtual applications much more quickly, which is critical in the global travel industry:
http://newsroom.cisco.com/dlls/2010/prod_062310c.html

Tutor Perini:
Reduced their data center footprint by 60%, reduced power consumption by 38% and Cisco UCS enabled them to support 4 times as many servers on a VMware ESX host as they did previously. They were able to provision 22 new VMware ESX hosts and their I/O properties in just two hours:
http://www.cisco.com/web/services/casestudy/docs/External_CaseStudy_TutorPerini_August2009.pdf

University of Seattle:
Is leveraging Virtual Desktops with Cisco UCS in their labs to reduce operating expenses and extend the lifecycle of their desktops. They now have the ability to deploy specific applications for any education or administrative department on demand:
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/case_study_seattle_university.pdf

Westpac:
“…Cisco’s Unified Computing System is core to the Westpac vision”. “Westpac chief technology officer Sarv Girn said the bank had already cut down the time it took to provision a new server “from weeks to days”.
http://www.techworld.com.au/article/364010/westpac_deploys_vce_private_cloud

Winterflood:
Cisco UCS provides the flexibility to keep up with rapidly changing business demands in the competitive trading environment, future server replacement costs are reduced by 90% and new trading applications can be provisioned in hours as opposed to weeks.
“Reduction of operational costs by 30% and capital costs by 20% while delivering agility to meet customer demand.”
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/Winterflood_V2CS.pdf
http://www.youtube.com/watch?v=lEP6RXzG_kA

______________________________________________________________________

Cisco UCS Service Provider Customers

Bridgewater:
David Sharpley, Senior Vice President, Bridgewater Systems: “Supporting Cisco’s Unified Computing System reflects Bridgewater’s portfolio strategy to ensure service providers can deploy our complete and pre-integrated control plane portfolio to increase scalability and business agility, and to reduce total cost of ownership.”
http://www.bridgewatersystems.com/2010-News-Releases.aspx?newsid=397

Cable and Wireless:
“Cable&Wireless Worldwide has selected Cisco’s Unified Computing System (UCS) as its next-generation data centre platform to deliver an enterprise cloud computing solution held within the Wide Area Network (WAN).”
http://www.cw.com/cable-wireless-worldwide-wins-global-government-telecoms-contract-2/
http://www.commsbusiness.co.uk/RSS_News_Articles.cfm?NewsID=11991

Cincinnati Bell:
“CBTS is extending its collaboration with SAP and the Virtual Computing Environment coalition to provide hosting and virtualization services in support of SAP applications that run on Vblock, integrated IT offerings built with best-in-class virtualization, compute, storage, security and management technologies from Cisco, EMC and VMware.”
http://www.cbts.cinbell.com/news/default.aspx?newsid=88

Entel :
“Entel, one of the most important telecommunications and information technology companies in Chile, has chosen Cisco, NetApp and VMware as technology suppliers of its data center in Ciudad de los Valles, the biggest and most modern data center for on-demand services in the country.”
http://newsroom.cisco.com/dlls/2010/prod_081010.html

ECKD:
According to Otto, the Cisco UCS and the Nexus 1000V virtual Cisco switches are among the new ECKD architecture’s most innovative elements. “There are many IT virtualization strategies out there, but no one has managed to make virtualization as far-reaching as Cisco UCS and VMware vSphere 4 have,” the managing director says.
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/solution_overview_c22-592014.pdf

NASA Ames:
The containerized datacenter solution includes the Cisco Unified Computing System (UCS) as well as Silicon Mechanics servers in a FOREST Container from Verari Systems. This approach enables the rapid deployment of a fully integrated IT infrastructure, including computing, storage, networking, virtualization, and systems management.:
http://www.hpcwire.com/offthewire/NASA-Ames-Selects-Containerized-Datacenter-Solution-from-Verari-and-Cisco-for-Nebula-Cloud-Platform-78349897.html

Navisite:
NaviSite has fully integrated the Cisco Unified Computing System as a key technology foundation for NaviCloud, NaviSite’s cloud computing platform. Cisco Unified Computing System unites computational, network, storage access, and virtualization into a scalable, modular architecture that is managed as a single system.
“We calculated that operating 200 PeopleSoft servers on Cisco UCS costs 20% less than on blade servers.”
http://www.navisite.com/3ede08c0-3472-4eaf-98a2-df135926a979/news-and-events-press-releases-detail.htm
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/case_study_c36-623923.html
http://www.youtube.com/watch?v=lEP6RXzG_kA

Orange:
Orange Business Services together with Cisco, EMC and VMware have formed a business alliance, called Flexible 4 Business, to offer end-to-end cloud computing services for enterprises.
http://www.orange-business.com/mnc/press/press_releases/2010/business-alliance.html

Parentix:
Parentix decided to purchase UCS based on the solution’s performance metrics alone, a bold move that is typical of the company’s innovative approach. “We were the first company in the Netherlands to move to the Cisco UCS platform, and we made that decision even before it became available for testing,” says technical director Vincent Kemp. “That was a clear sign of our trust in Cisco, and we were not disappointed.”
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/Parentix_case_study.pdf

PINS:
You cannot achieve virtualization by reading any one white paper,” says Etienne van Rijn, manager infrastructure and solutions at PINS. “In our case, we use a range of technologies from IBM, EQllogic, Intel, HP, VMware, and Cisco. UCS gave us a practical path to unify this mixed environment and extend our virtualization strategy.”
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/PINS_V1CS.PDF

Portugal Telecom:
Zeinal Bava, chief executive officer, Portugal Telecom: “Portugal Telecom is a market-leading innovator in a privileged position to deliver premium, advanced cloud computing services in the Portuguese market. In order to keep giving our customers the most advanced services we have to work with partners like Cisco, who share our passion for customer-focused innovation.”
http://newsroom.cisco.com/dlls/2010/prod_092210.html?CMP=AF17154&vs_f=News@Cisco:+News+Releases&vs_p=News@Cisco:+News+Releases&vs_k=1

Savvis:
Savvis will integrate the Cisco Unified Computing System (UCS) as the cornerstone of Savvis Symphony, Savvis’ next-generation private clouds platform…
http://cloudhosting.ulitzer.com/node/1216064prise-GradeCloudComputingSolutions.aspx

Singtel:
SingTel is the first Asian operator to announce plans to use VCE’s products. It will use the VCE products to offer hosted computing services to corporate customers.
http://www.networkworld.com/news/2010/091410-vce-coalition-signs-up-first.html?source=nww_rss

Saudi Telecom Company:
Samir Matboly, vice president, Enterprise Business Unit, STC: ” The Cisco team has demonstrated the ability to provide the STC Data Center 3.0 vision and technology solutions to equip us with the right technology for managed data center services and cloud computing.”
http://newsroom.cisco.com/dlls/2010/prod_052410b.html

SunGard:
SunGard’s cloud platform … combines best-in-class technology including Cisco’s Unified Computing System (UCS), Nexus 1000v and Multilayer Director Switches (MDS), EMC CLARiiON storage (secured by RSA), and the VMware vSphere™ platform.
http://www.availability.sungard.com/NewsAndEvents/PressReleases/Pages/SunGardAvailabilityServicesExpandsEnterprise-GradeCloudComputingSolutions.aspx

Telstra:
Cisco’s unified computing systems and VMware’s software virtualisation technology will provide the base for Telstra’s cloud offering. Oracle and SAP are among the enterprise software packages to be made available.
http://www.theaustralian.com.au/australian-it/silver-lining-as-telstra-moves-into-cloud-computing/story-e6frgakx-1225850068359

Tieto:
“The coalition of Cisco, EMC and VMware proposed to us optimal solution, says Director-General Tieto in Russia and CIS. – Because of their technologies we are ready to completely satisfy the need of our clients…”
http://babelfish.yahoo.com/translate_url?doit=done&tt=url&intl=1&fr=bf-home&trurl=http%3A%2F%2Fwww.cio-world.ru%2Fit-news%2F545071%2F&lp=ru_en&btnTrUrl=Translate

TSF:
Cisco Services then validated the entire project and helped TSF to create a scaling model for the Cisco Unified Computing System (UCS). They also implemented UCS in the new data center and supported TSF throughout the deployment.”
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/TSF_case_study.pdf
http://www.youtube.com/watch?v=lEP6RXzG_kA

______________________________________________________________________

Cisco UCS Partners & Distributors

Acadia:
Cisco and EMC, together with VMware, form coalition to accelerate pervasive Data Center Virtualization and Private Cloud Infrastructures:
http://newsroom.cisco.com/dlls/2009/corp_110309.html

Block:
Managing director Jon Pickering said, “Cisco UCS is really compelling from a customer standpoint and with the pipeline we have in place we are expecting a big year in data centre, especially around UCS.  It provides a very positive message to take to our customers.”
http://www.block-solutions.net/news-a-knowledge/news-a-events/185-strong-growth-as-barnsley-hospital-selects-cisco-ucs

CSC:
“Today’s announcement confirms VCE’s Vblock [including UCS] as the preferred technology stack for CSC’s cloud offerings to help customers more easily and securely migrate their business processes to the cloud.”
http://www.csc.com/vmworld/press_releases/52690-csc_forms_strategic_cloud_computing_alliance_with_the_vce_coalition_and_affiliated_partners_cisco_and_emc
http://www.arnnet.com.au/article/336237/csc_signs_google_australian_cloud_strategy/

Logicalis (UCS with CA Technologies):
“Chris Gabriel, director of marketing for Logicalis, provides case study at CA World on how the Cooperative Enterprise Cloud, a private and public hybrid cloud service, uses the Cisco Unified Computing System (UCS) and CA Service Automation and Service Assurance solutions to provide a superior service for Logicalis customers.”
http://community.ca.com/blogs/automation/archive/2010/05/18/ca-technologies-and-cisco-power-logicalis-innovative-cooperative-enterprise-cloud-service.aspx

MTI Europe:
“MTI, the specialist cloud infrastructure consultancy and integrator for Virtual Computing Environment (VCE) coalition, a collaboration of Cisco, EMC, and VMware, today announces the opening of its third MTI Solution Centre showcasing VblockTM Infrastructure Packages in Croissy, Paris, France.”
http://www.mti.com/Portals/0/French%20Solution%20Centre%20Release.pdf

Tech Data Opens Cisco Unified Computing System B- And C-Series Data Center:
“The addition of the Cisco Unified Computing System B- and C-Series Servers ensures the center remains a premier solutions demo and education facility, and we encourage Cisco channel partners to take full advantage of it.”
http://eon.businesswire.com/news/eon/20100811005158/en/Cisco-CSCO/Cisco-Unified-Computing-System-B-C-Series/Tech-Data-TECD

Tata Consultancy Services (TSC):
“Cisco’s game changing Unified Computing Systems and TCS ‘ Application and Infrastructure Management skills will accelerate time to market and deliver real results to global businesses.”
http://www.ciscotcs.com/default-inner.html

Wipro:
“Wipro views UCS as a game changer which addresses the needs of next-generation DC viz. cloud computing, consolidated, virtualized and automated data center environment, power efficient computing, proactive management and predictable operations.”
http://www.wipro.com/corporate/cisco/services/data-center-consolidation.htm

UCS with BMC:

• BMC
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns955/ns957/case_study_c36-610999.html
  http://documents.bmc.com/products/documents/24/43/142443/142443.pdf

• Force 3:
  “Force 3, delivering the best in federal security, unified communications and data center solutions, today announced that its Data Center team has been awarded a six-month, $6 million contract by the Department of Defense’s Defense Media Activity (DMA) to provide comprehensive server [Cisco UCS], network, virtualization and management infrastructure to meet BRAC initiatives.”
  http://www.prweb.com/releases/2010/09/prweb4575904.htm

UCS with Intel

• Terremark:
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/Intel_ESS_Terremark_case_study_HRS.pdf
  
• Alphawest:
  http://www.intel.com/references/pdfs/Alphawest_IDC.pdf

UCS with NetApp

• Examworks:
  http://media.netapp.com/documents/examworks.pdf
• Terremark:
  http://media.netapp.com/documents/terremark.pdf

• NetApp Global Dynamic Lab:
  http://www.netapp.com/us/library/customer-stories/global-dynamic-lab.html

• Entel:
  http://newsroom.cisco.com/dlls/2010/prod_081010.html

• University of Scranton / IntegraOne:
  University picks Cisco UCS over HP BladeSystem
  http://www.crn.com/news/data-center/226700071/tech-datas-demo-center-helps-var-close-cisco-ucs-deal.htm;jsessionid=cuSsYNQ9edrwLsobs1l4pg**.ecappj01

• McKenna, Long & Aldridge, LLP
  http://searchstorage.techtarget.com/generic/0,295582,sid5_gci1517650,00.html
  

UCS with Oracle

• Navisite:
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/case_study_c36-623923_ns944_Networking_Solutions_Case_Study.html

• EMC:
  www.emc.com/collateral/hardware/white-papers/h8012-emc-it-replatform-wp.pdf

• Cisco IT:
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns955/ns967/OracleUCS_cStudy1007.pdf
  
• Oracle Open World Keynote with Soni Jiandani, VP, Cisco Server Access and Virtualization Group and Wim Coekaerts, SVP, Oracle Linux and Virtualization Group
  http://www.cisco.com/web/solutions/data_center/events/oracleworld10.html
  

UCS with SAP

• Levis:
  http://www.sap.com/press.epx?pressid=13294

• Pacific Coast Builders:
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/case_study_C36_603155.html
  
• Scientific Atlanta:
  http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns955/ns970/white_paper_c11-618212.pdf

As the fairy tale goes, Goldilocks thought the chairs were either too big or too small. Unfortunately, she could have also been describing many blade servers out there today – they’re either too big (taking up too many expensive blade chassis/mini-rack slots) or they’re too small (providing too little memory). In a direct response to this problem, Cisco has developed the first blade server that isn’t too big or too small, but “just right”…

On Tuesday, September 14th Cisco announced the 13th server in the UCS server portfolio – the Cisco UCS B230 M1. This innovative blade has a unique form factor and memory density relative to the rest of the industry. This is a breakthrough form factor that offers the highest memory density and up to 16 cores in a single slot blade. A 6U blade chassis can now hold up to 256 DIMMs and 128 cores. This means you can put more blades, more memory, and more cores in a smaller space, reducing costs and complexity.

May 2011: The Cisco B230 M2 just announced. The M2 supports the Intel E7-2800 processor family and adds support for 16 GB DIMMs (512GB per server or 28TB per rack)
http://www.cisco.com/en/US/products/ps10280/prod_models_comparison.html

To truly understand how very cool this server is, we need to compare it to the latest blade servers from Cisco’s competitors.

As you can see in the chart above, the Cisco B230s pack an impressive punch in a single 42U rack – 56 servers delivering 896 cores and 14+ TB of RAM.

Here’s a closer look at the B230 M1:

The B230 will ship supporting the following processors:

The B230 will ship supporting the following mezzanine-based network adapters:

The B230 has achieved two performance benchmarking world records:

• SPECjbb2005
  #1 2-Socket x86 world record: Intel Xeon 7560 Scored 1,015,802 BOPS on Cisco UCS B230 M1
  (http://www.spec.org/jbb2005/results/jbb2005.html)
  _
• SPECjAppServer2004
  #1 world record dual-node:  Intel Xeon 7560 Scored 11,283.80 JOPS on Cisco UCS B230 M1
  (http://www.spec.org/jAppServer2004/results/jAppServer2004.html)

While reading a very informative blog post by Aaron Delp regarding VMware’s new NetIOC capability, I realized that Cisco and VMware are on the same page when it comes to server network traffic control. VMware’s NetIOC best practices plainly state exactly what Cisco has been advocating for so long – the user should plan for network contention but should not needlessly limit bandwidth in the absence of contention. 

VMware’s NetIOC Best Practices states the following:  

• Page 5: “[Using shares instead of limits] means that unused capacity will be redistributed to other contending flows and won’t go to waste.“

• Page 24: “Best practice 1: When using bandwidth allocation, use “shares” instead of “limits,” as the former has greater flexibility for unused capacity redistribution. Partitioning the available network bandwidth among different types of network traffic flows using limits has shortcomings. For instance, allocating 2Gbps bandwidth by using a limit for the virtual machine resource pool provides a maximum of 2Gbps bandwidth for all the virtual machine traffic even if the team is not saturated. In other words, limits impose hard limits on the amount of the bandwidth usage by a traffic flow even when there is network bandwidth available.”
  

Brad Hedlund wrote an excellent blog post discussing how Cisco uses QoS (shares) vs. HP Virtual Connect only using rate limiting (limits) and he provides some great animated graphics and analogies. I encourage you to read it…and then come back. J  

Two questions: 1. What are shares vs. limits? 2. How does Cisco UCS + VMware compare to HP Virtual Connect + VMware?
  

Shares (minimums) vs. Limits (maximums)

• A “share” is a VMware term that represents a relative value (relative to share values assigned to other flows) used to determine a particular traffic flow’s minimum bandwidth percentage during times of congestion. The greater the ratio of shares a traffic flow is assigned, the greater the amount of bandwidth it will receive during times of congestion. For example, let’s assume Flow A is assigned 10 shares, Flow B assigned 5 shares, and Flow C assigned 25 shares. The sum of shares is 40 (10+5+25). Flow A receives a minimum bandwidth share of 25% (10/40), Flow B receives a minimum of 12.5% (5/40), and Flow C receives a minimum of 62.5% (25/40). 25% + 12.5% + 62.5% = 100% of dvUplink bandwidth (whatever speed the link is operating at). During times of no congestion, each flow would have the ability to consume up to 100% of the link bandwidth. During times of congestion, each flow would be guaranteed their minimum ‘share’ of the bandwidth percentage.

• A “limit” is a VMware term that represents a static value used to define, in absolute units of Mbps, the maximum bandwidth a particular flow is able to consume on the overall vDS. In other words, you define a set bandwidth in Mbps and the flow is not able to exceed that maximum setting even if there is no contention on the dvUplinks. The bandwidth can be available but the flow is not eligible to use it.

To spin Brad’s highway analogy a slightly different way, I think of these two approaches to traffic control as “HOV Lanes” (shares) vs. “metered on-ramps” (limits) on the highway:  

An HOV lane guarantees of a minimum of one lane of highway “bandwidth” for high occupancy vehicles. Can high occupancy vehicles use other lanes if those lanes aren’t being used? Absolutely. If there are lots of high occupancy vehicles and very little other traffic, are the high occupancy vehicles forced to crowd into one lane? Absolutely not. What is the maximum number of lanes high occupancy vehicles can use? No maximum. What is the minimum number of lanes they are guaranteed? At least one.  

A “metered on-ramp” guarantees a maximum number of cars entering the highway in an attempt to avoid congestion on the highway. The on-ramp light allows only so many cars onto the highway during a given time period, regardless of how busy the highway is. Do most on-ramp metering systems adjust to the actual presence of highway congestion? No. They are programmed to allow a maximum number of cars entering the highway per minute, period. Will you be frustrated sitting idle in line on the on-ramp waiting on the light while seeing that the highway is wide open? Absolutely.  

Highway Analogy: QoS and Rate Limiting

Questions You May Have:  

• Which is more important? Metered on-ramps (limits) or HOV lanes (shares)?
  IMHO, if you can only have one, pick HOV lanes (shares). Metered on-ramps do not take into account congestion throughout the highway system. They also don’t guarantee any vehicle access to a highway lane free of congestion. HOV lanes (shares) can guarantee lanes to cars as the cars move through the highway system over great distances.
  _
• Can metered on-ramps (limits) co-exist with HOV lanes (shares)? Absolutely!
  You may want to set a hard limit on transmit or receive (maximum) while letting different traffic classes (flows) fluctuate bandwidth usage during times of no congestion.
  _
• Can there be more than one HOV lane?
  Absolutely! You didn’t read Brad’s post above, did you?
  _
• Can there be limits placed on on-ramps (egress traffic on NIC) and off-ramps (ingress traffic on NIC)?
  Absolutely! (But it depends on whose product you’re using. See below.)
  _

Cisco UCS + VMware vs. HP Virtual Connect + VMware Design Comparison

A proper highway design takes into account both bidirectional ramp metering and highway HOV lanes, not just on-ramp metering. The same goes for network design. Simply rate limiting what the NIC transmits only addresses part of the problem and doesn’t address the bigger picture – servers to server communication over the fabric.  

• VMware provides HOV lanes for on ramps (shares per flow per dvUplinks) plus metered on and off ramps (vDS-wide egress rate limiting per flow).
• Cisco UCS provides metered on-ramps (egress rate limiting per Palo logical NIC), metered off-ramps (ingress rate limiting per Palo logical NIC), HOV lanes for on-ramps and off-ramps (shares per priority on Palo/Menlo NIC ingress/egress), and HOV lanes on highways between ramps (fabric-wide shares per flow),
• HP Virtual Connect provides metered on-ramps (egress rate limiting per Virtual Connect FlexNIC), but no metered off-ramps (no ingress rate limiting per Virtual Connect FlexNIC), no HOV lanes for on-ramps or off-ramps, and no HOV lanes between ramps.

In the HP Virtual Connect + VMware example, you don’t have any HOV lanes. Virtual Connect only provides metered on-ramps (FlexNIC Tx speed limit). Once frames leave the HP server NIC and enter the on-ramp towards Virtual Connect, all frames are equal…first in first out. Good luck if a backup or VMotion starves out your VM’s production traffic somewhere inside the Virtual Connect domain (red arrows in graphic below). No amount of configuration in VMware or in the external network can control how Virtual Connect prioritizes traffic during times of internal congestion. In addition, FlexNIC speed limits artificially limit traffic needlessly during times of no contention/congestion – like on-ramp metering at 2 am on a Sunday!?!  

In the Cisco UCS + VMware example, the user has full traffic control – rate limiting in and out for VMware dvUplinks, rate limiting ingress and egress for Cisco NICs plus traffic shaping (shares per priority) in the fabric from end to end. In addition, when a rate limit is applied to a Cisco Palo interface (IF), the rate limit speed is reflected in the OS as the speed of the individual Palo interface. For example, with one Palo card, I could have 58 Palo NICs presented to the OS and each could be running at a different speed based on its rate limit as defined in the UCS Manager Service Profile.  

In summary, let’s reread VMware’s NetIOC Best Practice #1. “When using bandwidth allocation, use “shares” instead of “limits,” as the former has greater flexibility for unused capacity redistribution. Partitioning the available network bandwidth among different types of network traffic flows using limits has shortcomings.“  

Like Cisco, they ‘get’ server networking. Kudos to them.  

(Special Thanks to Doron Chosnek and Brad TerEick for their input)

Here’s a quick overview of the server network adapter options for Cisco’s UCS B-Series blade servers.

Cisco has three classes of adapters:

1. Cost Optimized:
   82598KR-I (Intel), M61KR-I (Intel), and M51KR-B (Broadcom)
   _
2. Compatibility Optimized:
   M71KR-E (Intel+Emulex), M71KR-Q (Intel+QLogic), M72KR-E (Emulex), and M72KR-Q (QLogic)
   _
3. Virtualization Optimized:
   M81KR (Palo)

Here are the interfaces that an OS ‘sees’ depending on which adapter you choose (click to enlarge):

_

And the obligatory “Feature Comparison Chart”…

_

Some Frequently Asked Questions:

Q. Are there special FCoE drivers I have to load in the OS for any of the adapters that support FCoE in hardware?
A. No, the operating system is oblivious to the fact that FCoE is even being used – just like the external LAN and SAN is oblivious.

Q. Can I use FCoE in the 82598KR-I, M61KR-I, and M51KR-B?
A. Yes, you can use FCoE software initiator packages with certain operating systems. These software packages allow the use of a regular Ethernet NIC. Open-FCoE.org would be an example. However, you do lose many features of a hardware-based FCoE solution.

Q. Why would I need up to 58 interfaces (using Palo) in a single server?
A. Virtualization and flexibility. For virtualization, Palo provides this many interfaces so that our customers can use VMware Pass Through Switching (PTS) and Hypervisor Bypass (VM Direct Path). See this colleague’s blog for more details: http://www.unifiedcomputingblog.com/?p=116. For flexibility, Palo allows our customers to deploy a single network adapter that can be programmatically configured to provide the NICs + HBAs combination that any server in their data center needs. Instead of buying a different hardware configuration for different types of servers, a single hardware configuration can be purchased. If a file server needs 2 x NIC interfaces, a single Palo can do it. If a clustered server needs 2 x NIC interfaces for production, 1 x NIC interface for cluster interconnect, and 2 x HBA interfaces, a single Palo can do it. If a hypervisor host A needs 4 x NIC interfaces for production, 1 x NIC interface for Service Console, 1 x NIC interface for VMotion, and 2 x HBA interfaces, a single Palo can do it.

Q. Which NICs use which physical port on Palo? I see 56 NICs and two physical ports.
A. Cisco provides a feature called Fabric Failover, or fabric-based NIC Teaming, that allows all NICs to use a single physical port during a path failure condition on one side. During normal operation, the NICs are evenly distributed across the two physical ports.  During a failure condition, all NICs can use a single physical port so that all NICs stay connected to the network.  Fabric Failover happens in the fabric, as the name suggests, and does not require any OS driver or OS configuration.  Fabric Failover is configured as part of the UCS Manager Service Profile for each server.  The HBAs always use a single physical port and never failover – as any SAN admin would want it. Multipathing would be used instead.

Q. Where can I find information on supported firmware, drivers, Operating Systems, etc. for these adapters?
A. See the latest version of the UCS “Hardware and Software interoperability Matrix” here: http://www.cisco.com/en/US/products/ps10477/prod_technical_reference_list.html

Now in our second year of shipping Cisco Unified Computing System (UCS) blade servers, it seems that most of our 1700+ UCS customers are migrating to Cisco blades from HP blades. Two teammates of mine, Jeff Allen (Twitter) and Doron Chosnek (Twitter), have developed a process to help these customers migrate from HP blade servers to Cisco UCS blade servers in a matter of minutes. Their migration process, based on publicly available documentation from HP and Cisco, allows a UCS customer to move the “server identity” of the retired HP blade server to the newly deployed Cisco UCS blade server. For a discussion of what the “server identity” is, see this previous blog entitled “The State of Statelessness: Cisco UCS vs. HP Virtual Connect“.         

Click Picture to Enlarge

Doron and Jeff accomplish this HP-to-Cisco migration by extracting the server identity information out of HP Virtual Connect and then importing that server identity information into Cisco UCS Manager. Fundamentally, they export an HP Virtual Connect Server Profile and import it into Cisco UCS Manager as a Cisco UCS Service Profile.     

 Server identity migration can be done seamlessly from HP to Cisco because Cisco’s UCS Service Profile is a superset of HP’s Virtual Connect Server Profile. Cisco manages ~96 server identity settings while HP only manages ~12 server identity settings. The reverse process, migrating Cisco-to-HP, would be difficult (if not impossible), since HP supports so few server identity settings in comparison to Cisco. I’ve provided an updated Cisco UCS vs. HP BladeSystem server identity comparison below. See Table 1.        

 In the following video, Doron and Jeff demonstrate their process by migrating a workload from an HP BL460c to a Cisco B200 M1 server. In their setup, both the HP blade and the Cisco blade have access to the same Ethernet and Fibre Channel networks. The soon-to-be-retired HP blade is booting from SAN. During the migration, WWN and MAC addresses, boot from SAN target and LUN number, etc. are taken from the HP server and given to the Cisco server. Since the FC network and storage controller see the same server identity on the new UCS blade, this allows the Cisco blade to immediately boot from the same FC LUN that the retired HP server was booting from. The result for the customer is an upgraded server running the same instance of the operating system with no changes required on the LAN or SAN.       

UCS Documentation References:
Overview of Cisco UCS Manager
Cisco UCS Manager CLI Command Reference Guide        

____        

Table 1: Comparison Chart Showing User Interfaces Required To Configure A Server (click table to enlarge)