Thursday, December 16, 2010

Basic Firewall Rule Config using the Cisco FWSM

Hello All,
Per a request from on of my server admins I am adding some basic rule configuration for a Cisco FWSM.    If you are looking for basic firewall configurations and getting started I recommend the Cisco Install Guide for the FWSM.  This site is available without login.

Where confusion has happened in the past in my department is how and where to create rules to allow traffic to pass through the firewall and block traffic that you do not want.

All my examples will be using this logical diagram:


This network could be physically designed as 1 6500 with the supervisor being used as the router a FWSM blade and the different networks being different vlans.

Lets start with a few concepts:

Interfaces:
Like all Cisco products the FWSM has interfaces, these are virtual and created from the VLANs assigned to the module in the 6500 or the 7500 config (See the link above to see how to configure this).  Unlike other Cisco products all interfaces are assigned a security level.  This security level is a ranking of interfaces from least secure to most secure, the higher the number the more secure the interface.  Understand this ranking may or may not reflect "actual" security a bad config could compromise any level of security that you may have.  Later in this blog I will refer to a higher and lower secure interfaces this comes back to this ranking.  For example, an interface ranked security 100 is a higher security level than an interface of rank 10 or even rank 99.  You can have two interfaces ranked at an equal level and you can configure the firewall to interpret that in two ways.  First, it will treat the other interface as a lower security level than itself so all the rules I talk about later would apply in this case, or you can treat it as an open link and the firewall will freely allow traffic to pass, assuming the access lists applied allow it.  Usually your external Internet connection is the lowest security interface on the firewall.

So basic config of the interfaces assuming a FWSM:
interface Vlan15
  description Internet Connection
  nameif Internet
  security-level 0
  ip address 10.0.0.1 255.255.255.0
interface Vlan20
  description DMZ Interface
  nameif DMZ
  security-level 10
  ip address 10.0.1.1 255.255.255.0
interface Vlan25
  description Inside Interface
  nameif Inside
  security-level 20
  ip address 10.0.0.1 255.255.255.0

NATs:
To traverse from a higher security interface to a lower security interface you need to create a NAT.  What you are doing is exposing the machine that is on the higher security interface to the lower security interface.  You need some sort of NAT to facilitate any communication from a lower security interface to a higher security interface.  The source is always the higher security interface and the destination is always the lower security interface.  If you find you are creating a NAT from the lower security interface as the source and the higher security interface as the destination you are probably doing something wrong. There are two types of NATs static and dynamic and each of those has two types.

Static NATs: 
This is a 1 to 1 NAT that is a permanent translation.  You use static NATs for things like servers on the higher security interface that machines in the lower security interface need to initiate a connection to.  In our example, the web server communicating back to a database server, the database server is be on the higher security interface and the web server the lower, but the web server would initiate all communications to the database server.  The Cisco recognizes two types standard and transparent.  A standard NAT translates the IP address from the original IP address out the higher security interface to a local address (on the same subnet) on the lower security interface.  The source address does not have to be on the same subnet as the higher security interface, but the firewall's routing table must have a working route back to the subnet the source address is on.  With a transparent NAT you use the original source address for both the source address and the destination address.  You still set the source interface as the higher security interface and the destination as the lower security interface.

So configuring a standard static NAT for the database server we would assign an IP address on the DMZ vlan for the server we will use 10.0.1.51.  Now the config is as follows:
static (Inside,DMZ) 10.0.1.51 10.0.3.50 netmask 255.255.255.255

This command is a little confusing I am going to rewrite it with identifiers not an actual workable command.
static (<Source Interface>,<Destination Interface>) <Translated Address> <Source Address> netmask <NETMASK>

Now the web server will address the database server with the IP address 10.0.1.51 not the address 10.0.3.50.  This can cause complete nightmares with DNS servers so to solve this issue we use transparent NAT, the configuration looks almost exactly the same but the translated address is the same as the source address.

Transparent NAT config:
static (Inside,DMZ) 10.0.1.50 10.0.3.50 netmask 255.255.255.255

Now the web server would access the database server as it's original IP address of 10.0.3.50.

Dynamic NATs:
Dynamic NATs do not create a permanent translation but will dynamically create a translation when a machine on a higher security interface wants to contact a machine on a lower security interface.  This is a more secure connection type because the machine's IP address seen by the machine on the lower security interface will change over time.  DO NOT assume this is the only security you need, once the NAT is dynamically created it will be accessible from the lower security interface.  The timeout here by default on the FWSM is 3:00 hours, and that counter is reset every time the machine communicates across the firewall.  Dynamic NATs come in two flavors, a 1 to 1 setup also called a dynamic NAT or a many to 1 setup also called a PAT.  In the 1 to 1 setup a machine on the higher security interface gets assigned an unique IP address on the lower security interface. In the many to 1 setup many machines on the higher security interface get assigned to a single IP address on the lower security interface. 

Configuring dynamic NATs can be a bit convoluted on the FWSM.  We use two commands global and NAT.  What my config example here will show is it will do a 1 to 1 translation into the DMZ and a many to 1 translation to the Internet.  Again the commands are almost the same, but the many to 1 translation requires a range of IP addresses.

FWSM Config:
global (DMZ) 5 10.0.1.100-10.0.1.150 netmask 255.255.255.0
global (Internet) 5 10.0.0.100 netmask 255.255.255.0
nat (Inside) 5 10.0.2.0 255.255.255.0
nat (Inside) 5 10.0.3.0 255.255.255.0

So the global command assigns the translated IP addresses and the NAT command limits the source addresses.  So in the above example anything on the 10.0.2.0 or the 10.0.3.0 network would be allowed to access the DMZ and the Internet connection using the above IP addresses.  The 5 after the interface name allows for the firewall to attach the global command group to the NAT command group.  The following two examples will show uses of multiple groups.

In this example I will assign the 10.0.2.0 network different address ranges than the 10.0.3.0 network.
global (DMZ) 5 10.0.1.100-10.0.1.150 netmask 255.255.255.0
global (Internet) 5 10.0.0.100 netmask 255.255.255.0
global (DMZ) 6 10.0.1.151-10.0.1.200 netmask 255.255.255.0
global (Internet) 6 10.0.0.101 netmask 255.255.255.0
nat (Inside) 5 10.0.2.0 255.255.255.0
nat (Inside) 6 10.0.3.0 255.255.255.0

In the above example the 10.0.2.0 network will get translated to the range 10.0.1.100-10.0.1.150 in the DMZ and 10.0.0.100 when going to the Internet.  The 10.0.3.0 range will get translated to the range 10.0.1.151-10.0.1.200 in the DMZ and 10.0.0.101 when going to the Internet.

The second example I will assign an outgoing range for the DMZ to get to the Internet.
global (DMZ) 5 10.0.1.100-10.0.1.150 netmask 255.255.255.0
global (Internet) 5 10.0.0.100 netmask 255.255.255.0
global (Internet) 30 10.0.0.151 netmask 255.255.255.0
nat (Inside) 5 10.0.2.0 255.255.255.0
nat (Inside) 5 10.0.3.0 255.255.255.0
nat (DMZ) 30 10.0.1.0 255.255.255.0

So here we have two different interfaces allowed to go out.  Notice I did not create a rule for the DMZ to the inside.  As I said earlier if you are creating a NAT translation from a lower security interface to a higher one you are probably doing something wrong.

We are now half way there to allowing traffic to flow through our firewall.

Access Lists:
The last thing you need is to give permission for data to pass from one interface to another.  In the FWSM any inbound traffic that is not from an already established connection will need a rule in an access list to go through the firewall.  Here is where you need to be careful with the FWSM, because unlike the old PIXs you will need to permit traffic going to lower security interfaces, but if you do a permit all rule it will allow the lower security interfaces to connect to any machine that has a current NAT into that interface including dynamic ones.  So for your lowest security interface you end with the implicit deny all and for your highest security interface you add as a last line a permit all, the problem is the middle security interfaces where you need rules to deny to higher security interfaces but permit to all the lower ones. It is important to note that access lists look for applicable rules from top to bottom, second unless you specify line numbers if you add a rule to the access list it will be added to the end of the list.

Here is a basic access list example we are permitting traffic on tcp port 1433 from the web server to the database server.

We will assume we have the standard NAT in place:
static (Inside,DMZ) 10.0.1.51 10.0.3.50 netmask 255.255.255.255

First we need to write the access list:
access-list DMZ_in extended permit tcp host 10.0.1.50 host 10.0.1.51 eq 1433

So we have source IP destination IP (translated IP) and port.

Now we need to assign an access list to the interface:
access-group DMZ_in in interface DMZ

This next example adds access list lines to allow the DMZ interface to talk to the Internet but not talk to the internal networks without setting specific permissions.

access-list DMZ_in extended permit tcp host 10.0.1.50 host 10.0.1.51 eq 1433
access-list DMZ_in extended deny ip any 10.0.2.0 255.255.255.0
access-list DMZ_in extended deny ip any 10.0.3.0 255.255.255.0
access-list DMZ_in extended permit ip any any

Two things here, first the access list is applied before the NAT translation happened on the inbound interface, so we are using the preNAT IP address for the database server.  Second we want a permit any at the end so we do not have to configure every IP range on the Internet.

So now we move on to the basic steps to adding another rule.

1. Make sure the traffic passes through the firewall.
2. Make sure you have a NAT translation for the machine on the higher security interface.
3. Make sure you add a rule to the access-list on the lower security interface, and the higher one too if applicable.

Finally, I will provide a full config for what we have talked about here, I am going to add an additional NAT to the Internet interface to allow port 80 and 443 traffic to go to the web server.

Final Config:
interface Vlan15
  description Internet Connection
  nameif Internet
  security-level 0
  ip address 10.0.0.1 255.255.255.0
interface Vlan20
  description DMZ Interface
  nameif DMZ
  security-level 10
  ip address 10.0.1.1 255.255.255.0
interface Vlan25
  description Inside Interface
  nameif Inside
  security-level 20
  ip address 10.0.0.1 255.255.255.0
static (Inside,DMZ) 10.0.1.51 10.0.3.50 netmask 255.255.255.255
static (DMZ,Internet) 10.0.0.80 10.0.1.50 netmask 255.255.255.255
global (DMZ) 5 10.0.1.100-10.0.1.150 netmask 255.255.255.0
global (Internet) 5 10.0.0.100 netmask 255.255.255.0
global (Internet) 30 10.0.0.151 netmask 255.255.255.0
nat (Inside) 5 10.0.2.0 255.255.255.0
nat (Inside) 5 10.0.3.0 255.255.255.0
nat (DMZ) 30 10.0.1.0 255.255.255.0
access-list Internet_in extended permit tcp any host 10.0.0.80 eq www
access-list Internet_in extended permit tcp any host 10.0.0.80 eq 443
access-list DMZ_in extended permit tcp host 10.0.1.50 host 10.0.1.51 eq 1433
access-list DMZ_in extended deny ip any 10.0.2.0 255.255.255.0
access-list DMZ_in extended deny ip any 10.0.3.0 255.255.255.0
access-list DMZ_in extended permit ip any any
access-list Inside_in extended permit ip any any
access-group Internet_in in interface Internet
access-group DMZ_in in interface DMZ
access-group Inside_in in interface Inside

Happy Networking
--Chris

Tuesday, December 14, 2010

Emulex OCE10102 cards

Hello,
Well if you have seen my coworkers blog we have had some issues with Emulex cards and the Nexus 1000V.  So tonight I am going to talk about the Emulex OCE10102 cards and what we saw and what fun we had fixing them.

Part 1 The Cards:
First with the Emulex OCE10102 cards, these are the converged network/FCoE cards, makes absolutely sure you match the driver and firmware version.  If you do not you will run into weird issues with the cards, for instance we have had the card come up on the Nexus1000V but refuse to transmit any traffic on non system vlans.  I know we come from a world of update the driver and everything will be fine.  This is ABSOLUTELY NOT TRUE with these cards, when checking for drivers check for firmware.  VMWARE will show the drivers on their driver page but not the firmware.  Here is the link to Emulex's download site for these converged adapters.  For VMWare they will redirect you to VMWare's site for the driver, but the firmware (or boot code as they call it) will be there.  They now have an installer that works in Windows, ESX 4.1, and ESX 4.0 to update the firmware.  If you are having trouble and are not on the newest drivers and firmware go back, beat yourself, and update them both now.

A special note on the Emulex OCE10102:
Apparently these cards are not compatible with the Dell R900 series of servers.  If you configure one on Dell's website they will sell you the card with it.  So we have 4 Exchange 2003 servers and at least 3 ESX 4.1 servers that are running on the Dell R900 hardware with these Emulex cards.  Our Exchange servers are Blue Screen of Deathing about once a week, if you look in the bios log of the server we see a fatal error in the PCI bus.  Our exchange servers are running Windows 2003 with the latest service pack and patches.  The only response from Dell we can get is these cards are not supported even though they will sell me a R900 with the card in it.  We have used this card in IBM servers and Dell R810s and had no problem after we fixed the driver issue.  Our ESX servers purple screen of death less often but still do. I may have to go to Dell and kick someone soon.

Part 2 configuring converged adapters on a Nexus 5000:

First make sure you have all the drivers installed on your server for the Emulex OCE10102 card.  There are 3 possible drivers, the Ethernet driver, the ISCSI driver, and the FCoE driver.  Your card, depending on the flavor you bought, will either do Ethernet and FCoE or Ethernet and ISCSI, as far as I know it will not do all three. 

ISCSI version:
For Ethernet and ISCSI there is no real special configuration on the switch side for this other than a trunk that allows all vlans to that the card needs.

Example:
This example config puts the ISCSI vlan as 20 and the Ethernet vlan as 30. And we will say the server is plugged into port 1/1.  The funny part about these cards is that if you do not specifically configure the Ethernet drivers for dot1q trunking, it will send out all packets untagged, on the other hand the HBA side of the card will tag the packets.

Nexus 5000 config:
vlan 20
  name ISCSI
  state active
vlan 30
  name ethernet
  state active
interface Ethernet 1/1
  description To ISCSI Ethernet CNA
  switchport mode trunk
  switchport trunk allowed vlan 20,30
  switchport trunk native vlan 30
  no shutdown

Most people using ISCSI want jumbo frames, so I will include the config here to turn them on.  This is stolen directly from this link on Cisco's web site.  We are removing our ISCSI devices in favor of fiber-channel and FCoE and nfs devices so I do not have a working config and will have to trust Cisco got it right.

Jumbo Frames on the nexus 5000:
policy-map type network-qos jumbo
  class type network-qos class-default
    mtu 9216
    exit
  exit
system qos
  service-policy type network-qos jumbo


What is nice about this config is that you can use Port-Channels to add redundancy and multiple links for throughput.

Now for the config for the FCoE version of the card:
This one is fun and comes with some caveats.  First, the card is not supported in a Port-Channel.  You have to dig around Cisco's website to find it, but it is there.  You will have to create a VLAN and assign it to a VSAN the VLAN has to be local to the switch.  The VSAN can cross multiple devices but as of yet the VLAN has to be local to the device you are working on, I hear rumblings on that changing but we aren't there yet.  You will need to create a VSAN even if you do not have the fiber channel add in cards for the Nexus 5000. So now on to the fun part the configuration.  If you have Cisco's Fabric Manager Server and the Enterprise License on a Cisco MDS series switch you can do some of this in there, after the config I will explain what you can do in the Fabric Manager.

First we need to turn on the FCoE feature, this is a licensed feature so make sure you have the license file loaded and installed.

Nexus 5000 command:
feature fcoe

Next we need to create a vlan and a vsan.

Nexus 5000 commands:
vlan 2
 fcoe vsan 2
 name FCoE_VLAN
 state active
vsan database
 vsan 2 name "FCOE_VSAN"

Now we need to create a virtual interface and bind it to a physical interface, in this example I am again using interface ethernet 1/1.  The bind command will fail if the interface is in a port-channel.

Nexus 5000 commands:
interface vfc1
 bind interface ethernet1/1
 no shutdown


We now need to add the interface vfc1 to the vsan database.

Nexus 5000 commands:
vsan database
 vsan 2 interface vfc1

Finally we need to configure the interface.  Make sure you make this interface a trunk and allow the san vlan and the ethernet vlan as configured above the san vlan is 2 so we will use that here, we will stick with the first example's ethernet vlan being 30.  In this case the above config makes sure that the FCoE packets get on to the proper vlan.  We are assuming the ethernet side of the card is still not configured to tag packets.

Nexus 5000 commands:
interface Ethernet 1/1
 description To CNA with Ethernet and FCoE
 switchport mode trunk
 switchport trunk native vlan 30
 switchport trunk allowed vlan 2,30

Once you have this all up you need to create zones.  I strongly recommend you find the WWN of the cards.  This means if you ever, heaven for bid, move the server from one port to another you will not have to change the zone.  The OCE10102 cards have the WWN of each port on a sticker on the card.  My WWNs are fake.  So you have source and destination. You can only activate one zone on a particular VSAN so you will see I use the AllZones zone to activate my individual zone.  It is good practice to have only 1 initiator per zone, so separate out those zones people, please.  You will save yourself the trouble and in many cases get better performance.

Nexus 5000 commands:
zone name NewZone vsan 2
 member pwwn 11:22:33:44:55:66:77:88
 member pwwn 88:77:66:55:44:33:22:11
zone name AllZones vsan 2
 member NewZone
zoneset activate name AllZones vsan 2

Yay you now have a FCoE Converged CNA configured, this will work for any converged CNA I have been able to test.

Here is the complete command list in one place:
feature fcoe
vlan 2
 fcoe vsan 2
 name FCoE_VLAN
 state active
vsan database
 vsan 2 name "FCOE_VSAN"
interface vfc1
 bind interface ethernet1/1
 no shutdown

vsan database
 vsan 2 interface vfc1
interface Ethernet 1/1
 description To CNA with Ethernet and FCoE
 switchport mode trunk
 switchport trunk native vlan 30
 switchport trunk allowed vlan 2,30
zone name NewZone vsan 2
 member pwwn 11:22:33:44:55:66:77:88
 member pwwn 88:77:66:55:44:33:22:11
zone name AllZones vsan 2
 member NewZone
zoneset activate name AllZones vsan 2

Ok finally in this blog that has become a wall of text I promised what parts you could do in the Cisco Fabric Manager.  The physical interface has to be configured manually.  In the device configuration of the fabric manager you can create and configure the vfc interface and assign it to the VSAN.  I as of yet have not figured out how to noshut the interface, and have to go into the switch command line to do it.  Once you have all the first parts done, sit back for a minute and allow the fabric manager to detect the new WWNs you put on the network.  Once it does you can use the zone manager to configure the individual zones, add it to the zone group, and activate it.  In a later blog post I'll post some screen shots of doing all this.

Happy networking.
--Chris

New Blog

Hello I am the manager of the networking department at Northern Kentucky University.  I plan to use this blog to discuss things I have discovered here working on our University network.  Some of my early posts will be the trials and tribulations of using the Nexus 1000V switch, MPLS, and dealing with an open network where you do not own over 50% of the machines that access it.

I may post on stupid user tricks every so often.

Even quotes from El Jeffe Grande as some of my employees call me.