Protecting Against DoS Attacks with the TMG Firewall

Introduction

Have you ever been the victim of a denial of service attack? Most of us have, and it can be pretty scary. At first you’re probably trying to figure out what’s happening. Then you might be wondering if you did something wrong to create a flood. When you figure out that it wasn’t your fault and that you are in fact under attack from an outside source, then you can start attempting to determine what you can do to remedy the situation.

Denial of service attacks through network floods are one of the more common types of attacks that you’re likely to see on the Internet. In most cases, a Denial of Service (DoS) occurs when a compromised computer or a real time human attacker floods your network or Internet-facing service with a large amount of traffic. Flood attacks can be carried out using a number of varying transports, which includes both network transport protocols (TCP and UDP) as well as “application transport” protocols (such as HTTP). There are ICMP based flood attacks, as well.

Advertisement

Blast from the Past: the TCP SYN Flood Attack

One example of a flood attack is seen when an attacker targets a network by flooding an IP address or by using a host name as a target to open multiple TCP connections. This is done by sending a very large volume of TCP SYN packets in a short period of time.

SYN floods traditionally have been a common method used to trigger denial of service and have been around for a long time. Hacker magazine 2600 published source code to perform SYN flood attacks back in the mid-1990s. Since then, many effective ways to prevent such attacks have been developed, but we still see variations on the theme.

Let’s look at how a basic SYN flood attack works. There are a number of mechanisms that can lead to the denial of service when a TCP SYN attack takes place. These include:

• Creating high CPU loads on the target device. The device being attacked needs to allocate processor resources to handle each half-open connection. If the speed and number of packets is great enough, the processor will be overtaxed and will not be able to respond to requests from required processes running on the device and this can end up cratering the device.
• Forcing high memory consumption. Each half-open TCP socket requires a certain amount of memory. And this isn’t just any memory, this is non-paged pool memory (if we’re talking about Windows devices). Non-paged pool memory is a subset of the total memory installed on the computer and therefore is more liable to be used up and thus create a DoS on the target.
• Generating a disk full situation due to logging. The SYN requests are logged and can reach such a high number and frequency that they exceed the ability of the system to log these connections as they fill up all available disk space. Disk space exhaustion is a serious problem for the TMG firewall if logging is being done on the local disk and this will lead to the firewall no longer accepting new connections.
• Filling the network pipe. TCP half open attacks don’t only have an impact on the TMG firewall itself; they can also fill up the pipe to the extent that no other connections will be able to arrive at the corporate network or leave the corporate network.

TMG Comes to the Rescue by Protecting Against Flood Attacks

The good news is that the TMG firewall can help protect you from many varieties of the network flood attacks. While the TMG firewall can’t protect from all such attacks, it does have a number of clever mechanisms in place that will mitigate some of the more important or common attack methods. For example, you can configure the TMG firewall to limit the number of concurrent connections from a particular address. If that number is exceeded, no additional connections will be allowed.

Flood mitigation protection can be complex to configure. Therefore, you might want to stick with the default settings until you learn more. These default settings configure connection limits for devices that connect to or through the TMG firewall. These settings were determined to be optimal (in the typical networking scenario) based on testing done by the Microsoft TMG Firewall team. Given the results of these tests, the TMG firewall team determined that these values are a good starting point for a “generic” deployment.

Let’s take a look at how you would configure flood mitigation settings. First, click the Intrusion Prevention Systemnode in the left pane of the TMG firewall console. Then click the Configure Flood Mitigation Settings link in the middle pane.

Figure 1

This opens the Flood Mitigation dialog box, as seen in the figure below.

Figure 2

You can set the connection limits for a variety of different traffic types. The exception to this is maximum half-open TCP connections. The value used by the TMG firewall for this setting is automatically calculated and configured by the firewall itself and is based on the maximum concurrent TCP connections per IP address, as shown in the figure below.

Figure 3

For many of the potential attacks, you will be able to set a Custom Limit.

You can also configure exceptions to the limits. Why would you need to do that? One reason might be that you need to exempt specific computers that routinely require a large number of open connections. A good example of this would be a DNS server that the TMG firewall is configured to use for name resolution. Remember that for web proxy and firewall clients, the TMG firewall is responsible for resolving names on behalf of those clients, and that means there is the potential for a very large number of DNS requests coming from the firewall to support those clients.

In addition, the TMG firewall might be configured with FQDN-based access rules. In this case, the TMG firewall will query a DNS server to obtain the IP address to determine whether the IP address is blocked by a rule. This, too, could lead the firewall to reach the maximum number of allowed connections within the predefined time period.

Figure 4

The following table describes possible flood attacks and how the TMG firewall can help protect against them.

Attack	TMG Mitigation	Default   Values
Flood Attack A specific IP address attempts to connect to various IP addresses, causing a flood of connection attempts and disconnections.	TCP connect requests per minute, per IP address TMG will only allow certain number of TCP requests from a specific IP in a minute, after which it will be blocked for the remainder of that minute.	By default TMG limits the number of TCP requests per client to 600 per minute. By default the custom limit applying to the IP exception list is set to 6,000 connection requests per minute.
Flood Attack A specific IP address attempts to flood either TMG or a server protected by TMG by opening multiple TCP connections concurrently.	TCP concurrent connections per IP Address TMG will limit concurrent connections per IP address to prevent a host from opening multiple TCP connections concurrently.	By default TMG limits the number of   concurrent TCP connections per client to 160. By default the custom limit applying to IP exceptions is 400 concurrent connections per client.
Half Open Attack An attacker attempts to flood either the TMG firewall or a server protected by the TMG firewall by sending numerous SYN packets in succession, accepting the TMG SYN_ACK response but not providing an ACK to the TMG SYN_ACK response, therefore not completing the TCP 3-way handshake.	TCP half-open connections The TMG firewall limits the number of half-open connections by monitoring the state of the connection and closing any half-open connections that exceed this limit	By default the TMG firewall limits this to half the TCP concurrent connections per IP address. You cannot modify this default setting without changing the TCP concurrent connection per IP address limit.
Denial of Service (DoS) attack using HTTP An attacker attempts to launch a DoS attack by sending numerous HTTP connection requests in succession.	HTTP requests per minute, per IP address The TMG firewall mitigates this attack by only allowing a predefined number of HTTP requests   per minute from a specific IP address.	The TMG firewall limits the number of HTTP requests per client to 600 requests per minute by default. The default custom limit applying to IP exceptions is 6,000 HTTP requests per client per minute.
Denial of Service (DoS) non TCP attack An attacker uses an infected computer to send numerous non-TCP packets, such as ICMP in succession, to flood the network or a server.	Non-TCP new sessions per minute, per rule If a non-TCP session is allowed by a rule, the TMG firewall limits the number of new sessions per rule in a minute.	The TMG firewall limits the number of non-TCP new session to 1,000 per minute for specific rules by default.
User Datagram Protocol (UDP) flood attack An attacker sends numerous UDP packets to the target or victim computer, causing flooding.	UDP concurrent sessions per IP address The TMG firewall limits the concurrent UDP connections per IP address. In case of a UDP flood attack, TMG discards all older sessions so that no more than the specified numbers of connections are allowed concurrently.	The TMG firewall limits the number of concurrent UDP sessions per IP address to 160 by default. The custom limit applying to IP exceptions is 400 concurrent UDP sessions per IP address by default.

Table 1

When the TMG firewall blocks a connection after it exceeds its connection limit, it remains blocked for the remainder of the minute. Okay, you might be thinking, What does that mean? What minute?

Good question. Let’s look at an example. Suppose you set a connection limit for concurrent TCP connections to 1000 for a particular client. The TMG firewall detects that in the last 45 seconds, the client generated 1000 TCP requests and connections. How much time remains in the minute? That’s right, 15 seconds.

Suppose you had a really busy client and it generated those 1000 connections in 10 seconds. How long would the client remain blocked? If you said 50 seconds, you win the prize.

Also, be aware that for TCP connections, no new connections are accepted from the source IP address of the attacker after the flood mitigation limit is exceeded. For non-TCP connections (e.g., raw IP and UDP), existing connections are torn down when the flood mitigation limit is exceeded. This allows newer connections to be created.

Summary

In this article, we talked about what flood attacks are and how they can be used to generate a denial of service condition. After we learned about flood attacks and how they work, we went over what the TMG firewall has to offer to help protect you against flood attacks of various shapes and sizes. I hope you learned a thing or two in this article and hope that you continue to make the most of your TMG firewall. Thanks! –Deb.