Troubleshooting DOS attacks on PIX / ASA

Posted on November 23, 2010

0



1. Identify the attack.
1.1. How did Customer notice the attack?

– Slow Internet access
– High performance on the attacked device:
– local server
– the firewall
– etc…

1.2. How can we confirm the attack?
Depending on the time of the issue we can have an attack:
A, happened in the past.
We need historical data for the time the attack happened. If it is not accessible than we are not able to provide any answer.
B, is happening now.
That is better for us. Even if no logging and any output Customer has, we should configure it and get it immediately.

1.2.1. Logging
In both cases we need the log for the time of the attack.
Configure the syslog severity level to debugging (In code 7.0):

ciscoasa# conf t
ciscoasa(config)# logging on
ciscoasa(config)# logging trap debugging
ciscoasa(config)# logging host inside 10.10.10.1

Where the 10.10.10.1 is the IP of the syslog server that resides behind the inside interface.
Syslog server link

The following log entries during the attack should be counted per minute and per 10 minutes*:

%PIX|ASA-6-302013: Built {inbound|outbound} TCP connection_id for interface:real-address/real-port (mapped-address/mapped-port) to interface:real-address/real-port (mapped-address/mapped-port) [(user)]
%PIX|ASA-6-302015: Built {inbound|outbound} UDP connection number for interface_name:real_address/real_port (mapped_address/mapped_port) to interface_name:real_address/real_port (mapped_address/mapped_port) [(user)]
%PIX|ASA-6-302020: Built {in | out}bound ICMP connection for faddr {faddr | icmp_seq_num} gaddr {gaddr | cmp_type} laddr laddr

%PIX|ASA-2-106001: Inbound TCP connection denied from IP_address/port to IP_address/port flags tcp_flags on interface interface_name
%PIX|ASA-2-106002: protocol Connection denied by outbound list acl_ID src inside_address dest outside_address
PIX|ASA-2-106006: Deny inbound UDP from outside_address/outside_port to inside_address/inside_port on interface interface_name.
%PIX|ASA-3-106010: Deny inbound protocol src interface_name:dest_address/dest_port dst interface_name:source_address/source_port
%PIX|ASA-6-106015: Deny TCP (no connection) from IP_address/port to IP_address/port flags tcp_flags on interface interface_name.
%PIX|ASA-2-106016: Deny IP spoof from (IP_address) to IP_address on interface interface_name.
%PIX|ASA-1-106021: Deny protocol reverse path check from source_address to dest_address on interface interface_name
%PIX|ASA-4-106023: Deny protocol src [interface_name:source_address/source_port] dst interface_name:dest_address/dest_port [type {string}, code {code}] by access_group acl_ID

*For this it is good to know some useful unix commands (awk / uniq -c / grep / wc –l)

Logging messages for OS 7.2.2:
http://www.cisco.com/en/US/docs/security/asa/asa72/system/message/logmsgs.html

1.2.2 Show commands
– show interface
If you know the attacker or the victim IP, then you should know on which interfaces of the firewall the traffic goes through.
Check the input, output queues on those interfaces. Record the actual values.
– show cpu
This will only confirm if the PIX is overloaded.
– show mem
This will only confirm if the PIX is overloaded.
– show conn / show conn count
The “show conn” will give us with the flags at the and of ech line the status of the connections. See the following table to understand the flags:

Inbound Connections:

Outbound Connections:

What we can look for is – for example – the number* of the saA or SaAB flags. If they are too high it can be a SYN flood attack.
Also we can check which destination or which source has that highest number of connections.*
*For this it is good to know some useful unix commands (awk / uniq -c / grep / wc –l)
show processes
?
show asp drop
?

1.3. What kind of attack the Customer noticed?
If we got the logs or show commands output we have to clarify the followings:
A, What kind of traffic is used?
– What protocol is used for the attack?
– What ports are attacked?
B, How much traffic?
Somehow we have to get information about the rate of packets that the attacker is sending. It is useful to determine the impact the attack is having on the network and or router / firewall.
C, Who is the victim and who is the attacker?

2. Make actions to block DOS attacks. Best effort.

2.1 Deny attacker traffic.
– ACLs for the attacker IP.
– shunning

2.2 Customize the thresholds.

– decrease limits in:

  • nat rules
  • connection settings
  • change the timeouts for embryonic connections / connections

2.1 Check the sources.

  • reverse lookup check (Enable reverse lookup check if not already enabled.)

3. Keep monitoring firewall after actions.

  • What changes have you experienced after the actions taken against the attack?
  • Normal CPU usage Normal memory usage
  • Usual connection count Normal Internet access
  • Check the logs again and if the following log entries are there:

%PIX|ASA-3-201002: Too many TCP connections on {static|xlate}global_address! econns nconns
%PIX|ASA-2-201003: Embryonic limit exceeded nconns/elimit for outside_address/outside_port (global_address) inside_address/inside_port on interface interface_name

%PIX|ASA-3-201010: Embryonic connection limit exceeded econns/limit for dir packet from source_address/source_port to dest_address/dest_port on interface interface_name
%PIX|ASA-3-201011: Connection limit exceeded cnt/limit for dir packet from sip/sport to dip/dport on interface if_name

%ASA-6-201012: Per-client embryonic connection limit exceeded curr num/limit for [input|output] packet from IP_address/ port to ip/port on interface interface_name

%ASA-3-201013: Per-client connection limit exceeded curr num/limit for [input|output] packet from ip/port to ip/port on interface interface_n

%PIX|ASA-3-202011: Connection limit exceeded econns/limit for dir packet from source_address/source_port to dest_address/dest_port on interface interface_name

%PIX|ASA-1-106021: Deny protocol reverse path check from source_address to dest_address on interface interface_name

%PIX|ASA-4-407002: Embryonic limit nconns/elimit for through connections exceeded.outside_address/outside_port to global_address (inside_address)/inside_port on interface interface_name

Logging messages for OS 7.2.2:
http://www.cisco.com/en/US/docs/security/asa/asa72/system/message/logmsgs.html

4. Useful links
4.1 For PIX / ASA.
– How to configure TCP intercept or block denial of service (DoS) attacks on the PIX Firewall
https://supportforums.cisco.com/docs/DOC-2652

– How to block DoS attacks with the access-list command or another feature on the PIX 500 Series Firewall with software version 7.x
https://supportforums.cisco.com/docs/DOC-3695

– Preventing Network Attacks
http://www.cisco.com/en/US/docs/security/asa/asa72/configuration/guide/protect.html#wpxref86536

Check those:

  • TCP and UDP connection limits and timeouts,
  • Preventing IP Spoofing
  • TCP sequence number randomization
  • TCP normalization
  • Basic IPS Support

– Defining Actions Using a Layer 3/4 Policy Map
http://www.cisco.com/en/US/docs/security/asa/asa72/configuration/guide/mpc.html#wp1054769

4.2 For Cisco Router.

– How to configure the router to minimize a Denial of Service (DoS) attack
https://supportforums.cisco.com/docs/DOC-3378

– Configuring TCP Intercept (Preventing Denial-of-Service Attacks)
http://www.cisco.com/en/US/docs/ios/11_3/security/configuration/guide/scdenial.html

– How to Configure IP Source Tracker
http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/ipst.html