F5 Agility Labs - Index¶
Welcome¶
Welcome to the F5 Solutions for DDoS lab at F5 Agility 2018
The content contained here leverages a full DevOps CI/CD pipeline and is sourced from the GitHub repository at https://github.com/f5devcentral/f5-agility-labs-ddos. Bugs and Requests for enhancements can be made by opening an Issue within the repository.
Getting Started¶
Lab Topology¶
Access and Credential Summary¶
You will using the Win7 jumpbox to access other systems for all labs. You will use Putty that has been preconfigured with appropriate keys in order to access the DHD CLI, Good Client, and the Attacker systems. The short cuts are on the desktop. You will be logged in as “root”.
Lab Components¶
System | Username | Password |
---|---|---|
Ravello | Given at site | Given at site |
Win7 Jumpbox | external_user | f5DEMOs4u |
Hybrid Defender - WebUI | admin | f5DEMOs4u |
Hybrid Defender - CLI | root | f5DEMOs4u |
Good Client | ubuntu | Use key |
Attacker | ubuntu | Use key |
Auction CLI | root | default |
Lamp CLI | root | default |
Lamp X-Server Shell | xubuntu | <no password> |
Accessing the Lab Environment¶
Task 1 – Open your RDP client and connect to your Windows Jumpbox¶
- A URL will be provided by your Instructor at the training site that will access the training portal.
- Click the Jumpbox RDP link.
Note
Use the show options to provide details.
- Login to the Jumpbox
- User name: Jumpbox external_user. Password: f5DEMOs4u
- Click YES at the warning
Note
All Exercises/Tasks are to be completed from the Windows Jumpbox. There are various shortcuts – Chrome Incognito, Putty shortcuts, Licensing Folders on the jumpbox that you will use through the exercises.
Class 1: Introduction to DDoS with F5 Herculon¶
DDoS Hybrid Defender, a hybrid DDoS solution that offers comprehensive protection, high availability, and is easy to deploy and manage. It guards against aggressive volumetric and targeted DDoS attacks, includes hardware-assisted DDoS mitigation, and optionally, connects with Silverline, a cloud-based scrubbing service.
This class covers the following topics:
- Initial Set-up, Device Configuration and Protected Object Configuration.
DDoS Hybrid Defender Setup¶
In this module you will learn how to complete the initial setup of F5 Networks DDoS Hybrid Defender
Lab 1 – DDoS Hybrid Defender Setup¶
Estimated completion time: 45 minutes
Task 1 – Initial Set-up¶
- Open a web browser and access supplied link.(Given at Location)
- Login to the BIG-IP Configuration Utility via your preferred browser?
Note
When you first power up a F5 DHD device you would go through the steps of Licensing and Provisioning. We have assigned the management IP, hostname, NTP and DNS servers. You will be re-activating the license using a new license key.
On the System > Platform page configure the following, and then click Update.
Host Name <your name>.f5demo.com Root Account (Password and Confirm) f5DEMOs4u Admin Account (Password and Confirm) f5DEMOs4u This will log you out. Log back in
On Device Management->Devices select the device and then click “Change Device Name…”. Update the device name to match the hostname you have chosen. Retain Current Authority
Click Update to save changes
Review and Verify the following: System -> Configuration -> Device -> NTP page add pool.ntp.org to the Time Server List, and then click Update.
Review and Verify the following: System -> Configuration -> Device ->DNS page add 8.8.8.8 to the DNS Lookup Server List, and then click Update.
Open the System > License page and re-activate the BIG-IP system using the new development license key using Manual mode. Copy and Paste License file.
Click Next and explore Resource Provisioning page
Note
The above task ensures that you are using a purpose built DDoS Hybrid Defender. If you are familiar with other F5 Modules/Technology that you have used in the past, you will notice that we have none of those provisioned.
When done click Submit.
Access the Jumbox via RDP. PuTTY into the Hybrid Defender. Login with
root
and restart servicesbigstart restart
Take a break, ask questions, talk to your neighbor ..it will take several minutes to restart
Note
You MUST re-activate, even if the current license key hasn’t expired. For Silverline access each BIG-IP system must use a unique license key.
Task 2 – DDoS Hybrid Defender iApp and Base Configuration¶
In the BIG-IP Configuration Utility, open DoS Protection > Quick Configuration page.
Select Install RPM method of Onboard
Click Install
Open the About page
This page displays the current version of DDoS Hybrid Defender (DHD). You use this page to install and update the iApp LX version for DHD when newer versions are released.
In the BIG-IP Configuration Utility, click iApps, Templates and Import, importing the two templates located on the jumpbox documents folder.
Use the Browse and Upload buttons. (You will do this once for each template)
In the BIG-IP Configuration Utility, open iApps > Application Services and select Create
You will be creating two services based on the two Silverline Templates:
- F5.silverline_connector
- F5.silverline_dos_monitor
Use the default settings for the Silverline connector
Use the Silverline username and password supplied
Note
This is case sensitive – make sure email address is all lowercase
Create the 2nd service for the Silverline DOS Monitor (f5.silverline_dos_monitor)
Use the default settings for the dos_connector except for Volumetric Attack Event Monitoring, switch the network object from interface to VLAN.
Open the DoS Protection > Quick Configuration Network Configuration page.
In the Default Network section click default VLAN.
Configure the VLANs using following information, and then click Done Editing.
Internal: VLAN Tag 20 Internal: Interfaces 1.2 Untagged Internal: IP Address / Mask 10.1.20.240/21 (Click Add) External: VLAN Tag 10 External: Interfaces 1.1 Untagged (Click Add) At the bottom of the page click Update to create the default network.
Open the Network > VLANs > VLAN Groups page and click defaultVLAN.
A Bridged (VLAN Group) L2 configuration consistent recommended practices for most deployments was automatically created
Open the Network > DNS Resolvers > DNS Resolver list page and click Create.
Enter default_DNS_resolver and then click Finished.
A DNS resolver is required by bot signatures to allow for proper detection of benign search engines such as Google and Bing.
On the Jumpbox desktop, PuTTY to the BIG-IP
Login as
root
Verify DNS by typing the following
nslookup api.f5silverline.com
Type the following to verify the correct date setting:
date
If the BIG-IP system date is not accurate, correct it using the following commands:
bigstart stop ntpd ntpdate 10.1.1.254 bigstart start ntpd
Task 3 – Configure Silverline Signaling¶
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page.
Open the Silverline page.
Configure using following information, and then click Update.
Username dhd2017us@f5agility.com Password HybridDefense!Wins! Service Address https://api.f5silverline.com Register the device with the Silverline iApp, to provide bandwidth utilization updates in iApps->Application Services->Applications->silverline_connector. In the iApp, select Reconfigure and then click Finished. This will cause the iApp to register under the new device name.
Use a web browser and access https://portal.f5silverline.com.
Log in with the above credentials
In the Silverline browser, open the Config->Hybrid Configuration->Hybrid Device Management page.
Locate your DHD device by searching for (<your name prefix>.f5demo.com) .
Click the Approve button to approve device registration.
Note
For Silverline device registration to function properly there must be some specific considerations. The BIG-IP system must have a unique device ID, which is comprised of attributes like Base MAC and registration key. In Ravello and similar virtual environments the Hybrid Defender VE must be re-licensed uniquely each time.
Task 4 – Configure DHD Device Bandwidth Thresholds¶
- In the DoS Protection > Quick Configuration page, open the
Protected Objects page.
In the Network Protection section click Create.
Configure using following information, and then click Save.
Maximum Bandwidth: Specify 500 Scrubbing Threshold: Type Percentage 1.20Scrubbing Threshold: Value 75 Advertisement Method Silverline Scrubber Details: Type Advertise All That completes the setup for BIG-IP DDoS Hybrid Defender with Silverline integration.
Lab 2 – Start Baseline Traffic Generation¶
Task 1 – Create Protected Objects that the baseline traffic will be targeting¶
In the BIG-IP Configuration Utility, open the DoS Protection>>Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Server5 IP Address 10.1.20.15 Port * Protocol All Protocols VLAN Any Protection Settings: Action Log and Mitigate Protection Settings: Silverline Yes (selected) Protection Settings: DDoS Settings IPv4, TCP, This protected object will be used for Auto-Threshold
Task 2 – Run Scripts to start L4 traffic generation – Good Traffic¶
Putty SSH (use the shortcut) to open a shell to the good client system.
Login as user :
ubuntu
. The session is preconfigured to authenticate with a certificate.Start the auto-threshold baselining script with:
# sudo bash # cd ~/scripts # ./baseline_l4.sh
Note
Ignore the “sudo: unable to resolve host xjumpbox” when you issue the sudo bash command throughout the labs.
Lab 3 – Configuring Hybrid Defender DDoS protection¶
Task 1 – Disable Device-Level DHD DoS Protection¶
In this lab you will disable device-level DoS flood protection, and then issue an ICMPv4 flood and review the results.
PuTTY to the BIG-IP CLI (10.1.1.245) and resize window by making it wider. Login with root/f5DEMOs4u.
At the config prompt, type (or copy and paste) the following command:
tcpdump -i 0.0
Open a second PuTTY window and Load the Attacker Saved Session at 10.1.1.7 and log in as ubuntu. I’t will use a pre-loaded public key as the credentials.
At the config prompt, type (or copy and paste) the following command:
ping 10.1.20.12
The attacker can successfully communicate with a back-end resource behind the BIG-IP DHD.
- Examine the tcpdump window and verify ICMP packets are flowing through the BIG-IP DHD.
Note
The listener for the ICMP packets is the VLAN group.
Cancel the
ping
command, then verify thetcpdump
stops receiving ICMP packets, and then press Enter several times to clear the recent log entries.In the Configuration Utility, in the DoS Protection, Quick Configuration, Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Don’t Enforce”
In the Flood row click the + icon, and then click ICMPv4 flood.
Note
If you minimize by clicking the + icon, it will make seeing the other sections easier.
On the right-side of the page select the drop-down to “Don’t Enforce”
- Apply the settings above for TCP SYN flood and UDP Flood., and then click Update.
On the Jumpbox in the Attacker PuTTY window type (or copy and paste) the following:
# sudo su # cd scripts # ls
Note
Ignore the “unable to resolve host Attacker message”
These are the different scripts we’ll be using during the exercises to simulate DoS attacks.
Type (or copy and paste) the following commands:
for i in {1..10}; do ./icmpflood.sh; done
This script launches 1,000,000 ICMP requests and then repeats for a total of ten occurrences.
View the
tcpdump
window and verify that ICMP attack traffic is reaching the back-end server.Let the attack run for about 15 seconds before moving on.
In the Configuration Utility, open the Statistics > Performance > Performance page.
View the Active Connections and Total New Connections charts.
There is a drastic spike in active connections.
View the Throughput (bits) and Throughput (packets) charts.
There is also a drastic spike in both bits per second and packets per second.
- Open the Security > Event Logs > DoS > Network > Events page.
The log file is empty as we disabled device-level flood protection on BIG-IP DHD.
- On the Jumpbox Attacker shell slowly type Ctrl + C several times
until back at the
scripts
prompt.
Task 2 – Re-enable Device-Level DHD DoS Protection¶
In this task you will re-configure device-level DoS protection, and then issue an ICMPv4 flood and review the results.
In the Configuration Utility, in the Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Enforce”
In the Flood row click the + icon, and then click ICMPv4 flood.
On the right-side of the page select the drop-down to “Enforce”
Click Update.
Note
This returns the configuration back to factory supplied device level enforcement.
On the Jumpbox in the Attacker A PuTTY window re-run the following command:
for i in {1..10}; do ./icmpflood.sh; done
Let the attack run for about 15 seconds before moving on.
In the Configuration Utility, open the Security > Dos Protection > DoS Overview > page
You should see the attacks and statistics. Explore the sections
In the Configuration Utility, open the Security > Event Logs > DoS > Network > Events page.
Note
You may need to refresh this page several times before the log files display.
- Sort the event by Time in descending order.
There are now log entries showing dropped packets.
- The DoS Source is Volumetric, Aggregated across all SrcIP’s, Device-Wide attack, metric:PPS.
- The type is ICMPv4 flood.
- The action is Drop.
- On the Jumpbox Attacker shell slowly type Ctrl + C several times
until back at the
scripts
prompt.
In the Configuration Utility, open the DoS Protection > Quick Configuration page and click Device Configuration.
In the Flood row click the + icon, and then click ICMPv4 flood.
On the right-side of the page configure using the following information, and then click Update.
Detection Threshold PPS Infinite Rate/Leak Limit Infinite
Task 3 – Configure Protected Object-Level IPv4 Flood DHD DoS Protection¶
- In this task you will configure object-level DoS IPv4 flood protection, and
- then issue an ICMPv4 flood and review the results.
On the Protect Objects page, in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name ServerNet IP Address 10.1.20.0/22 Port * Protocol All Protocols Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings IPv4 In the IPv4 row click the + icon, and then click ICMPv4 flood.
On the right-side of the page configure using the following information, and then click Create at the bottom of the page.
Detection Threshold PPS Specify: 1000 Detection Threshold Percent Infinite Rate/Leak Limit Specify: 1000 On the Jumpbox in the Attacker A PuTTY window re-run the following command:
for i in {1..10}; do ./icmpflood.sh; done
Examine the
tcpdump
window to see if there are any ICMP packets hitting the back-end server.Let the attack run for about 30 seconds before moving on.
In the Configuration Utility, click DoS Protection > Quick Configuration > ServerNet, and then in the IPv4 row click the + icon.
Open the Security > Event Logs > DoS > Network > Events page.
The DoS Source is Volumetric, Aggregated across all SrcIP’s, VS-Specific attack, metric:PPS.
The context column displays /Common/ServerNet, identifying this is protected object-level protection.
The action is Drop.
The difference between packets in per second and dropped packets is roughly 1000.
On the Jumpbox slowly type Ctrl + C several times until back at the
scripts
prompt.In the BIG-IP PuTTY window type Ctrl + C to stop the tcpdump.
Task 4 – View the DoS Visibility Page¶
- You can now use the new DoS Visibility page to view statistics about the
- DoS attacks you submitted during this exercise.
Open the Statistics > DoS Visibility page.
Note
It may take a couple of minutes for the correct data to display.
In the Attack Duration window there are several attacks.
Mouse over several of the attacks to get additional details of each attack.
Scroll down in the left-side of the page to view the Attacks section.
You can see the number of high, moderate, and low attacks in addition to the types of attacks (HTTP, DNS, Network) and the severity levels.
View the details at the bottom of the Attacks section.
This table displays details of each attack that has occurred.
Sort this table by Vector.
Scroll down in the left-side of the page to view the Virtual Servers section.
You can see the details of device-wide attacks (Device Level) and protected object-level attacks (/Common/ServerNet).
- Scroll down in the left-side of the page to view the Countries section.
- View the details at the bottom of the Countries section.
This table displays the attack details from each country.
View the various widgets in the panel on the right-side of the page.
Click Network to filter out only the network-level attacks (all the attacks so far have been network-level).
If it’s not already expanded, expand the Virtual Servers widget, and then select /Common/ServerNet.
This filters the results to only attacks at this protected object-level. Notice the changes to the map on in the Countries section.
Click /Common/ServerNet to remove the filter.
Drag the resize handle on the right-side of the main window as far to the left as possible.
Expand the Vectors widget, and then select ICMPv4 flood.
Expand the Client IP Addresses widget.
Question: How many client IP addresses contributed to this attack?
Expand the Countries widget.
Sort the countries by Dropped Requests.
Select China, and then view the changes to both the Client IP Addresses widget and the map.
At the top of the page open the Analysis page.
Note
The requests are still filtered for the ICMPv4 flood results for China.
Drag the resize handle on the as far to the right as possible.
Examine the Avg Throughput (Bits per second) graph.
Place your mouse over the peak in the graph.
Question: What is the Average client in throughput during the attack?
Feel free to examine more of the Dashboard page and the Analysis page.
Lab 4 - Multi-vector Demo¶
In this simple demo you will launch a small number of network attacks and show the configuration, logging and reporting capabilities of the Hybrid Defender. The point of this demo is to provide context for a UI walkthrough with some live data.
Task 1 - Access DoS Quick Configuration and display the ServerNet protected object¶
This protected object is defending all ports/protocols for 10.1.20.0/24, which is the network behind the Hybrid Defender. Attacks will be launched at 10.1.20.12, which is an interface on the LAMP server. Verify that the following vectors are configured:
Launch the attacks and show the behavior
Open the following tabs in the DHD UI:
DoS Protection->Quick Configuration->ServerNet
Security->DoS Protection->DoS Overview (leave the filter at default: ’DoS Attack’)
Statistics->DoS Visibility
Access the Attacker System CLI and run the attack
# cd ~/scripts # sudo bash # ./multivector.sh
- Click Refresh on the DoS Overview page. You will see some attacks mitigated by Device Configuration and some mitigated by the more specific settings on the ServerNet Protected Object.
Navigate to Security->Event Logs->DoS->Network->Events.
Click on “custom search…” link.
Drag one of the values from the “Attack Type” column into the custom search builder. From the Action column, drag Drop into the search builder. Click “Search”.
Further explore the DoS Event logs as needed for your demo. For example, clear the search and identify the “Stop” and “Start” times for an attack, etc.
In the Hybrid Defender WebUI, access the DoS Visibility reporting tool at Statistics->DoS Visibility.
Note
DoS Visibility is a reporting tool, not a real-time monitoring tool. Events are displayed, much like other AVR-based reporting, in 5 minute windows. Do not expect events to be shown here immediately after running an attack. Be aware of this timing when doing a demo. Quicker/real-time monitoring of on-going DoS attacks is best accomplished in the DoS Event Logs and DoS Overview areas of the WebUI
You should see the attacks in the timeline and a variety of details in the windows. Use the slider to shorten the timeframe if needed, and click the Network filter, to focus on L4 activities.
Note
that you can select events from the timeline and see details about the attacks
Log in to Silverline at https://portal.f5silverline.com.
Navigate to Monitor and Analyze > Stats > Hybrid Device. Locate your device and explore the interface.
Lab 5 - Bad Actor Detection Demo¶
In this demo you will run an attack from specific IP addresses. The Hybrid Defender will be configured to perform bad actor detection, limit the attack on a per-IP basis with more aggressive thresholds and then, based on this detection, automatically blacklist the offending IP address adding them to the (hardware-accelerated) dynamic blacklist
Task 1 - Open the following tabs in the DHD UI:¶
- DoS Protection->Quick Configuration->ServerNet
- Security->DoS Protection->DoS Overview (leave filter at default: “DoS Attack”)
- Statistics->DoS Visibility
- Security->Event Logs->Network->IP Intelligence
Task 2 – Configure the following UDP Flood vectors for ServerNet:¶
DoS Protection->Quick Configuration->ServerNet
Access the Attacker system CLI and run the UDP flood attack:
# sudo bash # cd ~/scripts # ./udp_flood.sh
From the menu, select ‘1’ to start the attack
root@attacker-a:~/scripts# ./udp_flood.sh 1)Attack start 2)Attack end 3)Quit # ?
Note
This attack is relatively short-lived. You can launch it again if the attack ends and you are not finished showing the various reports. Simply type ‘1’ again, to re-run the attack
In the Hybrid Defender UI, show the Security > DoS >DoS Overview page. Note the blocks by Bad Actor.
In the Hybrid Defender UI, show the Security > Events > Network > IP Intelligence Event Logs. Note the IP addresses that are being added to the denial_of_service blacklist.
In the Hybrid Defender WebUI, show the Statistics > DoS Visibility. Expand the Vectors inspector and select UDP Flood. When it updates, select a flood from the timeline. Note in the Attacks panel the #IPs blocked is 10.
From the menu, select ‘2’ to end the attack
or
# sudo bash
# killall -9 hping3
Lab 6 - Auto-threshold demo¶
This demo will simulate a newly configured Protected Object where the security administrator is unsure what values to assign to a few common vectors. Note that auto-thresholding is useful at both the Device and Protected Object levels.
In the interest of having a repeatable demo in an environment where many different types of traffic are executed, we are focusing on the per-VS/per-PO auto-thresholding
Note
This demo may place significant stress on the demo environment. Due to the virtual environment limitations, this may make the DHD UI less responsive. This is unavoidable since for auto-thresholding to block the attack, the attck must be damaging enough to cause stress, which will push the CPU on the VE very high. Rememberthis is a virtual environment under high stress and that the Hybrid Defender appliances mitigate these attacks in dedicated hardware.
Open the following tabs in the Hybrid Defender WebUI:
DoS Protection->Quick Configuration
Security->DoS Protection->DoS Overview (set filter to Virtual Server->Server5)
Security->Event Logs->DoS->Network->Auto Threshold
Statistics->DoS Visibility
On the Good Client, if you have not already done so, start the network baselining
# cd ~/scripts # sudo bash # ./baseline\_l4.sh
3. In the Hybrid Defender UI, in Quick Configuration, select the Server5 Protected Object and verify that the IP and TCP vectors are all at default thresholds with auto-threshold disabled
In the Hybrid Defender CLI, restart auto-thresholding
# cd ~/scripts # ./autothreshold-reset.sh
In the Hybrid Defender WebUI, in the Server5 Protected Object configuration, enable auto- thresholding for the following vectors: ICMPv4 Flood, TCP SYN Flood, TCP Push Flood, TCP RST Flood, TCP SYN ACK Flood by selecting each vector and clicking the Auto- Threshold Configuration radio button. When all vectors are configured, click Update at the bottom of the screen
In the Hybrid Defender WebUI, show the Auto Threshold event log (Security->Event Logs->Dos->Network->Auto Threshold).
The system is updating the detection thresholds. With auto-thresholding, the system adjusts the detection thresholds based on observed traffic patterns. However, mitigation rate limits are always dynamic based on detected system or protected object stress. If anomalous levels of traffic are running, but there is no stress, the Hybrid Defender will generate alerts but will not block traffic. Under stress, the rate limits are automatically created and adjusted dynamically
Let’s create some stress with a SYN Flood attack. In the Attacker CLI start the auto- threshold SYN flood
# cd ~/scripts # sudo bash # ./autot\_flood.sh
This is a long duration attack. You can terminate it with ctrl-C when finished.
In the Hybrid Defender WebUI, show the Auto Threshold event log. Now you will see that Rate limits are being automatically set and adjusted to mitigate the flood attack
In the Hybrid Defender WebUI, show the Security > DoS > DoS Overview page. Note that the SYN Flood attack is being mitigated and the rate limit thresholds for each of the auto-threshold vectors have been adjusted based on stress, including vectors that are not detecting or blocking an attack
Terminate the attack in the Attacker CLI with ctrl-C
After the attack has ended, in the Hybrid Defender WebUI, show the DoS Visibility page. Under Vectors, select TCP SYN Flood. Identify the Critical attack and show the details
Clean-up. On the Attacker CLI, if the attack is still running be certain to end it with ctrl-C.
Clean-up. For repeatability, it is necessary to disable the auto-thresholding for the ICMPv4 Flood, TCP RST Flood, TCP Push Flood, TCP SYN ACK Flood and TCP SYN Flood vectors on the Server5 protected object
Clean-up. After disabling auto-thresholding, clear the learning on the Hybrid Defender CLI with
# cd ~/scripts # ./autothreshold-reset.sh
Learn More¶
*F5 DDoS Education*
Web based training and product information
- Product Training https://university.f5.com/
- DDoS Protection Reference Architecture
- DDoS Protection Recommended Best Practices
- *F5 DDoS Hybrid Defender overview and user guide*
*Silverline DDoS Education *
Web based training and product information
- Product Training https://university.f5.com/
Onboarding Tech. Notes on f5.
Class2: Intermediate DDoS Protection¶
This class covers the following topics:
- Topics here
DDoS Hybrid Defender Setup¶
In this module you will learn how to complete the initial setup of F5 Networks DDoS Hybrid Defender
Lab 1 – DDoS Hybrid Defender Setup¶
Task 1 – BIG-IP Herculon Hybrid Defender Licensing and Provisioning¶
Note
When you first power up a F5 DHD device you would go through the steps of Licensing and Provisioning. We have assigned the management IP, hostname, NTP and DNS servers. You will be re-activating the license using a new license key.
Note
For Silverline device registration to function properly there must be some specific considerations. The BIG-IP system must have a unique device ID, which is comprised of attributes like Base MAC and registration key. Hence we are re-licensing the device as all student instances are spun up using the same license.
Use a web browser (Chrome in incognito mode) to log into the WebUI of your DHD at https://10.1.1.245 . or use the bookmarked shortcut. Accept the SSL warning and proceed to connect.
Username : admin
Password : f5DEMOs4u
Click System>>License and Click Re-activate
Click Edit button, replace the existing key by entering your student license key. Select the “Manual” radio button and Click Next.
Select all in the Dossier frame and copy. Click on “Click here to access F5 Licensing Server”
You will be taken to the F5 Activation Site. Enter your Dossier that you copied in the step above and click next. Accept User Legal Agreement - Check box to agree to terms of license and click next.
Select Everything the License frame and copy it.
Go back to your F5 DHD management and paste the contents copied from above into Step 3: License and Click Next.
The bigip will restart daemons and a window will pop up indicating system configuration has changed. Please wait for it to reconnect and click Continue. Your device is now licensed. Click Next.
On the Resource Provisioning page validate that Management and DDOS Protection are provisioned.
Click Submit once.
Note
The above task ensures that you are using a purpose built DDoS Hybrid Defender. If you are familiar with other F5 Modules/Technology that you have used in the past, you will notice that we have none of those provisioned.
Task 2 – BIG-IP Herculon Hybrid Defender Initial Setup¶
Click System>>Platform
Change the hostname to <yourfirstinitiallastname>.hybriddefender.f5agility.com. For example, John Smith would register as jsmith.hybriddefender.f5agility.com. This is needed so that we can register your DHD to Silverline and uniquely identify it. Click Update.
Click Device Management>>Devices select the device and then click “Change Device Name…”. Update the device name to match the hostname you have chosen and click Update
Use Putty Shortcut to ssh to the F5 DHD and login as: root password: f5DEMOs4u
From the Hybrid Defender shell, restart services with:
# bigstart restart
Note
Be patient as services are restarting. The DHD will change state to INOPERATIVE and then to Active. You can check in the ssh window when the prompt changes.
- Click System>>Configuration>>Device>>NTP and review that NTP server is configured
- Click System>>Configuration>>Device>>DNS and review that DNS server lookup is configured
In the BIG-IP Configuration Utility, open the DoS Protection>>Quick Configuration page. Click Install. This installs the onboard package for quick configuration of DDoS Hybrid Defense
Once the installation is completed. Open the About page.
This page displays the current version of DDoS Hybrid Defender (DHD). You use this page to install and update the iApp LX version for DHD.
The System is installed with the latest version of the iApp LX. The below steps are for future reference on how to obtain the latest iApp LX and use the above step to install. Do not download and install during the Agility labs.
Newer versions of iApp LX packages are made available on the F5 downloads site under Security>>DDoS Hybrid Defender.
Open the Network Configuration page
In the Default Network section click defaultVLAN.
Configure the VLANs using following information, and then click Done Editing. Make sure to Click “Add”
Internal: VLAN Tag 20 Internal: Interfaces 1.2 (Untagged checked)
(Click Add)
Internal: IP Address / Mask 10.1.20.240/21 External: VLAN Tag 10 External: Interfaces 1.1 (Untagged checked)
(Click Add)
Click UPDATE.
Open the Network>>VLANs>>VLAN Groups page and click defaultVLAN.
A transparent L2 configuration consistent with recommended practices for most deployments was automatically created.
- Open the Network >> DNS Resolvers >> DNS Resolver list page and click Create.
- Enter default_DNS_resolver for the name and then click Finished.
A DNS resolver is required by bot signatures to allow for proper detection of benign search engines such as Google and Bing. This is a workaround and its setup is planned to be added to the Quick Configuration, it’s not included in the version accompanying the installed release for the labs.
In the BIG-IP putty ssh window verify DNS by typing (or copying and pasting) the following:
nslookup api.f5silverline.com
Type the following to verify the correct date setting:
date
Do this step only if the BIG-IP system date is not accurate, correct it using the following commands:
bigstart stop ntpd ntpdate pool.ntp.org bigstart start ntpd
Use a Firefox web browser and access https://portal.f5silverline.com.
Log in as dhd2017us@f5agility.com / HybridDefense!Wins!
In the BIG-IP Configuration Utility, open the DoS Protection >>Quick Configuration page.
Open the Silverline page in Dos Protection>>Quick Configuration
Configure using following information, and then click Update. Make sure to use all lowercase for username.
Username dhd2017us@f5agility.com Password HybridDefense!Wins! Service Address https://api.f5silverline.com In the Silverline portal browser page, open the Config>>Hybrid Configuration>>Hybrid Device Management page.
Locate your DHD device (<yourfirstinitiallastname>.hybriddefender.f5agility.com) and click Approve for ALL instances of YOUR device
On the DHD WebUI go to DoS Protection>>Quick Configuration. In the Configuration Utility, open the Protected Objects page.
In the Network Protection section click Create.
Configure using following information, and then click Save.
Maximum Bandwidth: Specify 500 Scrubbing Threshold: Type Percentage Scrubbing Threshold: Value 75 Advertisement Method Silverline Scrubber Details: Type Advertise All
This completes the initial setup for BIG-IP DDoS Hybrid Defender including registration with Silverline.
Lab 2 – Start Baseline Traffic Generation¶
Task 1 – Create Protected Objects that the baseline traffic will be targeting¶
In the BIG-IP Configuration Utility, open the DoS Protection>>Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Server5 IP Address 10.1.20.15 Port * Protocol All Protocols VLAN Any Protection Settings: Action Log and Mitigate Protection Settings: Silverline Yes (selected) Protection Settings: DDoS Settings IPv4, TCP, This protected object will be used for Auto-Threshold
Task 2 – Run Scripts to start L4 traffic generation – Good Traffic¶
Putty SSH (use the shortcut) to open a shell to the good client system.
Login as user : ubuntu. The session is preconfigured to authenticate with a certificate.
Start the auto-threshold baselining script with:
# sudo bash # cd ~/scripts # ./baseline_l4.sh
Note
Ignore the “sudo: unable to resolve host xjumpbox” when you issue the sudo bash command throughout the labs.
Lab 3 – Configuring Network Attack Protection¶
Task 1 – Disable Device-Level DHD DoS Protection¶
Disable device-level DoS flood protection, and then issue an ICMPv4 flood and review the results.
Ssh (putty) into the BIG-IP DHD using the shortcut provided. Resize the BIG-IP putty ssh window by making it wider.
At the config prompt, type (or copy and paste) the following command:
tcpdump -i 0.0 host 10.1.20.12
Open a second putty window and ssh to the Attacker (use shortcut on the desktop) and log in as ubuntu. It will authenticate using the ssh key provided automatically.
At the attacker config prompt, type (or copy and paste) the following command:
ping 10.1.20.12
The attacker can successfully communicate with a back-end resource behind the BIG-IP DHD
Examine the tcpdump window and verify ICMP packets are flowing through the BIG-IP DHD.
Cancel the ping command (Ctrl+C), then verify the tcpdump stops receiving ICMP packets, and then press Enter several times to clear the recent log entries.
In the Configuration Utility, in the Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Don’t Enforce”
In the Flood row click the + icon, and then click ICMPv4 flood.
On the right-side of the page select the drop-down to “Don’t Enforce”
Click Update.
In the Attacker putty window type (or copy and paste) the following:
# sudo bash # cd ~/scripts # for i in {1..10}; do ./icmpflood.sh; done
This script launches 1,000,000 ICMP requests and then repeats for a total of ten occurrences.
View the tcpdump window and verify that ICMP attack traffic is reaching the back-end server.
Let the attack run for about 15 seconds before moving on.
In the Configuration Utility, open the Statistics >> Performance >> Performance page.
View the Active Connections and Total New Connections charts.
There is a drastic spike in active connections.
View the Throughput (bits) and Throughput (packets) charts.
There is also a drastic spike in both bits per second and packets per second.
Open the Security >> Event Logs >> DoS >> Network >> Events page.
The log file is empty as we disabled device-level flood protection vector on BIG-IP DHD.
In the Attacker putty ssh shell slowly hit Ctrl + C several times until the prompt is back at the /scripts.
Task 2 – Re-enable Device-Level DHD DoS Protection¶
In the Configuration Utility, in the Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Enforce”
In the Flood row click the + icon, and then click ICMPv4 flood.
On the right-side of the page select the drop-down to “Enforce”
Click Update.
This returns the configuration back to factory supplied device level enforcement.
Task 3 – Configure Protected Object-Level Network DoS Protection¶
With the DHD device wide protection provides a line of defense and is enforced for all traffic flowing through the device. For more granular control, we use protected objects and configure mitigation settings for those objects to be enforced. In this task we will configure object-level DoS network multi-vector protection, and then issue an attack and review the results in the next task.
Go to Dos Protection>>Quick Configuration
On the Protect Objects page, in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name ServerNet IP Address 10.1.20.0/24 Port * Protocol All Protocols Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings IPv4,TCP,UDP,
Sweep
Maximum Bandwidth: Specify 200 Enable External Redirection Checked Scrubbing Threshold: Percentage 90% Scrubbing Silverline Silverline Checked Verify the newly created protected object:
Click on the “ServerNet” object and configure the following vectors and click Update.
Vector Detection Thresh. PPS Detection Thresh % Rate Limit ICMP Fragment 100 500 200 ICMPv4 Flood 100 500 200 IP Fragment Flood 100 500 200 TCP SYN Flood 100 500 200 TCP SYN Oversize 100 500 200
Task 4 – Launch the attack and view the results¶
Click DoS Protection>>Quick Configuration->ServerNet
Open the following as new tabs (right click and select open link in a new tab) in the DHD UI (Google Chrome Window):
Security>>DoS Protection>>DoS Overview (leave the filter at default: ‘DoS Attack’ and change auto refresh to 20 seconds)
Statistics>>DoS Visibility
Access the Attacker System CLI/shell and launch the attack:
# sudo bash # cd ~/scripts # ./multivector.sh
The attacks will be detected immediately. Let the attacks run for a couple of minutes. Click Refresh on the DoS Overview page and it will start to populate. You will see some attacks mitigated by Device Configuration and some mitigated by the more specific settings on the ServerNet Protected Object:
Navigate to Security>>Event Logs>>DoS->Network>>Events.
Click on “custom search…” link.
Drag one of the values from the “Attack Type” column into the custom search builder. From the Action column, drag Drop into the search builder. Click “Search”
Further explore the DoS Event logs. For example, clear the search and identify the “Stop” and “Start” times for an attack, etc.
In the Hybrid Defender WebUI, access the DoS Visibility reporting tool at Statistics>>DoS Visibility. If you get a time-skew warning, then please ignore it as it’s the Windows PC that can’t keep the clock right.
Note
The DoS Visibility is a reporting tool, not a real-time monitoring tool. Events are displayed, much like other AVR-based reporting, in 5 minute windows. Do not expect events to be shown here immediately after running an attack. Quicker/real-time monitoring of on-going DoS attacks is best accomplished in the DoS Event Logs and DoS Overview areas of the WebUI.**
You should see the attacks in the timeline and a variety of details in the windows. Use the slider to shorten the timeframe if needed, and click the Network filter, to focus on L4 attacks and mitigation.
Stop the attack (Ctrl+C) in the Attacker CLI (ssh window).
Task 5 – Configure Bad Actor Detection¶
Add bad actor detection for a for the UDP flood protection.
In the Configuration Utility, open the DoS Protection >> Quick Configuration page and in the Protected Objects section click ServerNet.
In the UDP row click the + icon, and then click UDP Flood.
On the right-side of the page configure using the following information in the table, and then click Update.
Set the UDP Flood vector settings:
Setting Value Enforce selected Manual Configuration selected Detection Threshold PPS 100 Detection Threshold Percent 500 Rate Limit 200 Bad Actor Detection selected Per Source IP Detection 100 PPS Per Source IP Rate Limit 30 PPS Blacklist Attacking Addresses selected Detection Time 15 seconds Duration 120 seconds Open the following in new tabs (Google Chrome - right click and select open link in new tab) in the DHD UI:
DoS Protection>>Quick Configuration>>ServerNet
Security>>DoS Protection>>DoS Overview (leave filter at default: “DoS Attack” and set refresh rate to 20s)
Statistics>>DoS Visibility
Security>>Event Logs>>Network->IP Intelligence
Access the Attacker system CLI (putty ssh) and run the UDP flood attack:
# sudo bash # cd ~/scripts # ./udp\_flood.sh From the menu, select ‘1’ to start the attack root@attacker-a:~/scripts# ./udp\_flood.sh 1) Attack start 2) Attack end 3) Quit #?
Note
This attack is relatively short-lived. You can launch it again if the attack ends and you are not finished showing the various reports. Simply type ‘1’ again, to re-run the attack. You may have to run the attack multiple times using ‘1’.
In the DoS Overview page observe the blocks by Bad Actor
In the IP Intelligence Event Logs observe the IP addresses that are being added to the denial_of_service blacklist.
In the DoS Visibility tab expand the Vectors inspector and select UDP Flood. When it updates, select a flood from the timeline. Note in the Attacks panel the #IPs blocked is 10
End the UDP_Flood attack script by typing ‘2’ to kill any still running processes and then ‘3’ to exit the script.
Clean-Up : Be sure to stop all hping3 processes by using the following command:
# sudo bash # killall -9 hping3
Lab 4 – Using Auto Thresholding¶
This exercise will simulate a newly configured Protected Object where the security administrator is unsure what values to assign to a few common vectors. Note that auto-thresholding is useful at both the Device and Protected Object levels
Note
This demo may place significant stress on the demo environment. This may make the DHD UI less responsive. This is unavoidable since for auto-thresholding to block, the attack must be damaging enough to cause stress, which will push the CPU on the Virtual Environment very high. Remember that this is a virtual environment with minimal resources for lab under high stress and that the Hybrid Defender appliances mitigate these attacks in dedicated hardware.**
Task 1 – Configure Auto Thresholding¶
On the Good Client, if you have not already done so, start the network baselining. This step is needed if you didn’t start the good traffic generation in Exercise 2 or accidently stopped it.
# sudo bash # cd ~/scripts # ./baseline_l4.sh
In the Hybrid Defender UI, in Quick Configuration, select the Server5 Protected Object and verify that the IP and TCP vectors are all at default thresholds with auto-threshold disabled:
Setting Value All Detection Thresholds 30000 pps All Rate Limits Infinite Auto Thresholding Disabled In the Hybrid Defender CLI (BIGIP ssh window), restart auto-thresholding:
# tmsh run security dos device-config auto-threshold-relearn # tmsh run security dos virtual name Server5 auto-threshold-relearn
In the Hybrid Defender WebUI, in the Server5 Protected Object configuration, enable auto-thresholding for the following vectors: ICMPv4 Flood, TCP SYN Flood, TCP Push Flood, TCP RST Flood, TCP SYN ACK Flood by selecting each vector and clicking the Auto-Threshold Configuration radio button. When all vectors are configured, click Update at the bottom of the screen.
In the Hybrid Defender WebUI, view the Auto Threshold event log by navigation to Security>>Event Logs>>DoS>>Network>>Auto Threshold.
The system is updating the detection thresholds. With auto-thresholding, the system adjust the detection thresholds based on observed traffic patterns. However, mitigation rate limits are always dynamic based on detected system or protected object stress. If anomalous levels of traffic are running, but there is no stress, the Hybrid Defender will generate alerts but will not block traffic. Under stress, the rate limits are automatically created and adjusted dynamically.
Task 2 – Create Stress to trigger Auto Thresholding and view Reports.¶
Let’s create some stress with a Flood attack. In the Attacker CLI start the auto-threshold flood:
# sudo bash # cd ~/scripts # ./autot_flood.sh
This is a long duration attack. You can terminate it with Ctrl+C when finished.
In the Hybrid Defender WebUI, review the Auto Threshold event log. You will see that Rate limits are being automatically set and adjusted to mitigate the flood attack.
In the Hybrid Defender WebUI, view the DoS Overview. Note that the ICMP Flood attack is being mitigated and the rate limit thresholds for each of the auto-threshold vectors have been adjusted based on stress, including vectors that are not detecting or blocking an attack.
Select the filter type to Virtual Server (DoS protected) and Server5 and view how various Thresholds are dynamically adjusted based on the stress
Terminate the attack in the Attacker CLI with Ctrl+C.
After the attack has ended, in the Hybrid Defender WebUI, navigate to the DoS Visibility page. Under Vectors, select ICMPv4 Flood. View various details.
Clean-up: On the Attacker CLI, if the attack is still running be certain to end it with Ctrl-C.
Clean-up: For repeatability, it is necessary to disable the auto-thresholding for the ICMPv4 Flood, TCP RST Flood, TCP Push Flood, TCP SYN ACK Flood and TCP SYN Flood vectors on the Server5 protected object. Switch them back to Manual Configuration.
Clean-up: After disabling auto-thresholding, clear the learning on the Hybrid Defender CLI with:
# tmsh run security dos device-config auto-threshold-relearn # tmsh run security dos virtual name Server5 auto-threshold-relearn
Clean-up: Stop the baseline traffic generation from the good-client if still running using CTRL+C
Lab 5 – Configuring DNS Attack Protection¶
DNS DoS attacks come in many flavors and target different resources. DNS query, reverse flood and amplification attacks are some such DNS attacks.
DNS Query Flood¶
This type of DoS of service attack has a couple possible resource impacts.
- Overwhelm the DNS server’s ability to respond by sending too many requests
This can be done just by asking for more requests than the server can reply with and prevent the server from servicing legitimate requests. It doesn’t really matter if the clients are spoofed or not, it only matters that the DNS server just can’t keep up.
Mitigation Options¶
DNS DoS mitigation generally requires an awareness of what you’re trying to protect. This allows you to apply the appropriate mitigations and push the problem upstream until the next step is to force it off premises and in to a cloud solution. Load balancing is one remedy to this solution (anycast). Spreading the requests across pools of servers can help mitigate against these types of attacks. DNS Express is another option to increase the capacity of your DNS infrastructure. Layering in DHD DNS DoS vector mitigation also stops common DNS attacks.
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the
In the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name DNSServer IP Address 10.1.20.14 Port 53 Protocol UDP Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings DNS In the DNS row click the + icon, and then click DNS A Query.
On the right-side of the page configure using the following information, and then click Create.
Detection Threshold PPS Specify: 75 Detection Threshold Percent Specify: 500 Rate Limit Specify: 100
In the Attacker putty window type (or copy and paste) the following command:
./dnsdosrate.sh
Wait for the attack to run for about 30 seconds before moving on.
In the Configuration Utility, review the DoS Overview page
The A query DOS attack vector is now dropping attack traffic.
The script will also record the number of drops if any as a result of the attack rate limit being hit.
In the Configuration Utility open the Statistics >>DoS Visibility page. View details in various sections
DNS Reverse flood¶
Sometimes DNS responses are used in flooding network resources. A small request has a disproportionately larger response and since the transport protocol is UDP it can easily be spoofed. The outbound pipe can easily get congested responding to a smaller number of requests with large responses.
Use tcpdump and dig to view DNS request and response packets. A small request produces a large response. You will open two ssh sessions to the attacker.
Putty to the Attacker CLI (use the shortcut).
Putty to the Attacker CLI (use the shortcut).
In the first ssh window on the attacker start a tcpdump using the following command:
# sudo bash # cd ~/scripts # tcpdump -i lo &
In the second ssh window on the attacker issue a dig against the loop back with “ANY”
# sudo bash # cd ~/scripts # dig ANY floodzone.local @localhost
In the first ssh window on the attacker view the results of the tcpdump : Notice the difference in the size of the request (44) vs the response (628). Your values maybe different. The point is that a small request can generate an enormous response.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes 07:33:55.737892 IP localhost.47406 > localhost.domain: 3896+ [1au] ANY? floodzone.local. **(44)** 07:33:55.738563 IP localhost.domain > localhost.47406: 3896\* 17/0/13 A 10.10.1.2, A 10.10.1.3, A 10.10.1.4, A 10.10.1.1, A 10.10.1.5, SOA, NS ranger.floodzone.local., NS langley.floodzone.local., NS lexington.floodzone.local., NS attacker-a.f5d emo.com., NS saratoga.floodzone.local., AAAA ::1, MX enterprise.floodzone.local. 20, MX hornet.floodzone.local. 40, MX ess ex.floodzone.local. 50, MX yorktown.floodzone.local. 10, MX wasp.floodzone.local. 30 **(628)**
In the second ssh window on the attacker issue a dig against the loop back with “ANY” for a larger response.
# sudo bash # cd ~/scripts # dig ANY ripe.net @localhost +dnssec
In the first ssh window on the attacker view the results of the tcpdump : Notice the difference in the size of the request (37) vs the response (2715). Your values maybe different. The point is that a small request can generate an enormous response.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes 07:43:44.018212 IP localhost.51272 > localhost.domain: 58304+ [1au] ANY? ripe.net. **(37)** 07:43:44.018889 IP localhost.domain > localhost.51272: 58304$ 18/8/15 RRSIG, SOA, RRSIG, RRSIG, A 193.0.6.139, RRSIG, DNSKEY, DNSKEY, DNSKEY, RRSIG, DS, NS manus.authdns.ripe.net., NS a2.verisigndns.com., NS a1.verisigndns.com., NS tinnie.arin.net., NS sns-pb.isc.org., NS sec3.apnic.net., NS a3.verisigndns.com. **(2715)**
Once you’re done, type ‘fg’ and ‘CTRL+C’ to stop the tcpdump.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
fg
tcpdump -i lo
^C
0 packets captured 0 packets received by filter 0 packets dropped by kernel root@Attacker:~/scripts#
This can easily overwhelm the server or overwhelm the outbound network pipe disrupting traffic responses for legitimate requests and/or other applications.
One industry accepted way to mitigate this type of attack is to rate limit the responses on the DNS servers. More information on Response Rate Limiting can be found here:
https://www.isc.org/wp-content/uploads/2014/11/DNS-RRL-LISA14.pdf
Because DoS policies are applied to traffic flows on ingress to the DHD, response rate limiting isn’t currently available. But you still can limit the types of queries that can disproportionately consume bandwidth. The ANY query used in the previous example is one such example.
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page
In the Protected Objects section click DNSServer.
In the DNS row click the + icon, and then click DNS ANY Query.
On the right-side of the page configure using the following information, and then click Update.
Detection Threshold PPS Specify: 50 Detection Threshold Percent Specify: 500 Rate Limit Specify: 75 In the BIG-IP Configuration Utility, open the Security>>DoS Protection>> DoS Overview page and set the Filter type to “Virtual Server (DoS protected) / DNSServer” Set Auto-Refresh to 20 seconds.
In the attacker ssh window issue dns reverse flood attack as follows:
# sudo bash # cd ~/scripts # ./dnsReverseFlood.sh
Observe the DoS Overview as it gradually starts to drop the ANY queries.
In the attacker ssh window stop the attack by hitting “CTRL+C” many times
Lab 6 – Configuring L7 Attack Protection¶
In this exercise we will use a protected object and enforce mitigation for low and slow/encrypted layer 7 attacks.
Task 1 – Create Protected Object and Launch Attack¶
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Server1 IP Address 10.1.20.11 Port 443 VLAN (Selected) defaultVLAN (uncheck ANY) Protection Settings: Action Log and Mitigate Protection Settings: Silverline Yes (selected) Protection Settings: DDoS Settings IPv4, TCP Launch attacks without any layer 7 protection configured
Open the following in separate tabs in the Hybrid Defender WebUI:
DoS Protection>>Quick Configuration
Security>>Reporting>>DoS>>Analysis
From a firefox browser go to https://10.1.20.11. Ignore SSL warning and Add Exception. Note that this bypasses the Hybrid Defender and accesses the server directly, showing the availability and/or performance of the site directly. Click around a few links. This is the site we will launch an attack against and mitigate.
Verify that the configuration is providing no L7 protections by taking the server offline with a slowloris attack. Note that apache will try to clean up the slow flows, but they will do so inefficiently and the server is impacted (which will show as an outage, missing objects and/or slower responsiveness). Run the slowloris attack from the Attacker CLI:
# sudo bash # cd ~/scripts # ./slowloris.sh
The tool will rapidly show the site offline (10-15 seconds, with trivial traffic load):
Refresh https://10.1.20.11 to show the effects of the attack. [Note that since we are running locally from the Win7 system in a virtualized environment, you may be able to access the site, however it will be slower and often the GIFs will not load. An internet user would not be able to “fight through” the attack to get to the server as often as a system on the local LAN.]
Stop the slowloris attack by using CTRL+C.
Start a more effective Slow Read attack.
This attack is harder for DoS mitigation tools to mitigate and can be very effective even with a tiny number of concurrent connections trickling in very slowly to the server to fly below the radar of network detections. In our example we will open 10 connections per second and read the response data at 1 byte / sec. The attack would be effective even at 1 cps, it would just take a bit longer to build up the connections.
From the Attacker CLI/shell start the slowread attack:
# sudo bash # cd ~/scripts # ./slowread.sh
As soon as the site is down (service available: NO), refresh https://10.1.20.11 to show that it is down/slow/intermittent.
Task 2 – Configure Protection/Mitigation, launch attack and view reports¶
In the Hybrid Defender WebUI, access the Server1 Protected Object.
Enable SSL.
Select the default certificate and key. In your environment you would select a valid/cert key for your application.
Enable ‘Encrypt Session to Server’ to avoid any server reconfiguration.
Enable the HTTPS mitigation family.
Click Update.
View the Attacker CLI/shell. The slow read attack is now no longer showing the site as down (service available: YES) because Proactive Bot Detection has mitigated the attack.
Refresh https://10.1.20.11 to see that the site behavior has returned to normal.
You were able to mitigate an encrypted layer 7 attack quickly and with only a few simple steps.
In the Hybrid Defender WebUI, view various reports in the Security>>Reporting>>DoS>>Analysis
HTTP Report (Scroll towards the bottom) shows Proactive Mitigation.
Stop the Slow Read attack by using CTRL+C.
This concludes your hands on labs. In this class you learned how to mitigated various DDoS attacks using F5 BIGIP Hybrid Defender (DHD).
Class 3: DDoS Hybrid Defender¶
DDoS Hybrid Defender, a hybrid DDoS solution that offers comprehensive protection, high availability, and is easy to deploy and manage. It guards against aggressive volumetric and targeted DDoS attacks, includes hardware-assisted DDoS mitigation, and optionally, connects with Silverline, a cloud-based scrubbing service.
This class covers the following topics:
- Initial Set-up, Device Configuration and working with basic device-level DDoS vectors to mitigate the most commonly encountered attacks. Then we will cover Auto-thresholding, bad actor detection, DNS reflection and amplification, real-time blackholing, mitigation of L7 floods, low-and-slow attacks and behavioral attacks.
DDoS Hybrid Defender Setup¶
In this module you will learn how to complete the setup of F5 Networks DDoS Hybrid Defender
Lab 1 – DDoS Hybrid Defender Setup¶
Task 1 – Initial Set-up¶
- Login to the BIG-IP Configuration Utility via the desktop shortcut (DHD WEB GUI). You will land on the welcome page.
Note
When you first power up a F5 DHD device you would go through the steps of Licensing and Provisioning. We have assigned the management IP, hostname, NTP and DNS servers. We have already licenesed the device for you.
Review and Verify the following: System -> Configuration -> Device -> NTP page. This should be already populated with pool.ntp.org
Review and Verify the following: System -> Configuration -> Device ->DNS page. This should be already populated with 8.8.8.8
Click System and explore Resource Provisioning page.
Note
The above task ensures that you are using a purpose built DDoS Hybrid Defender. If you are familiar with other F5 Modules/Technology that you have used in the past, you will notice that we have none of those provisioned. We have a new section DDOS Protection only.
Task 2 – DDoS Hybrid Defender iApp and Base Configuration¶
In the BIG-IP Configuration Utility, open DoS Protection > Quick Configuration page.
If not already installed, Select Install RPM method of Onboard.
Click Install.
After the RPM is installed you will see the following:
Open the About page.
This page displays the current version of DDoS Hybrid Defender (DHD). You use this page to install and update the iApp LX version for DHD when newer versions are released.
Open the DoS Protection > Quick Configuration Network Configuration page.
In the Default Network section click default VLAN.
Configure the VLANs using following information, and then click Done Editing.
Internal: VLAN Tag 20 Internal: Interfaces 1.2 Untagged Internal: IP Address / Mask 10.1.20.240/21 (Click Add) External: VLAN Tag 10 External: Interfaces 1.1 Untagged (Click Add) At the bottom of the page click Update to create the default network.
Open the Network > VLANs > VLAN Groups page and click defaultVLAN.
Note
A Bridged (VLAN Group) L2 configuration consistent with recommended practices for most deployments was automatically created. Also called “Bump in the Wire”. F5 can support Routed mode, SPAN and Netflow as well.
Open the Network > DNS Resolvers > DNS Resolver list page and click Create.
Enter default_DNS_resolver and then click Finished.
A DNS resolver is required by bot signatures to allow for proper detection of benign search engines such as Google and Bing.
On the Jumpbox desktop, SSH to the BIG-IP, it will log you in automatically as user
root
, using the shortcut.Verify DNS by typing the following:
nslookup api.f5silverline.com
Verify the Date by typing the following:
date
If the BIG-IP system date is not accurate, correct it using the following commands:
bigstart stop ntpd ntpdate 10.1.1.254 bigstart start ntpd
Task 3 – Explore DHD Device Bandwidth Thresholds¶
In the DoS Protection > Quick Configuration page, open the Protected Objects page.
In the Network Protection section click Create.
This page is where you would supply values to protect your bandwidth and integrate with Silverline or use BGP to change your routing to go through a scrubbing center.
Click Cancel when done exploring the available settings.
That completes the initial setup for BIG-IP DDoS Hybrid Defender.
Lab 2 – Configuring Hybrid Defender DDoS protection¶
Task 1 – Disable Device-Level DHD DoS Protection¶
In this lab you will disable Device-level DoS flood protection, and then issue an ICMPv4 flood and review the results.
PuTTY to the BIG-IP CLI (10.1.1.245) from your jumpbox desktop shortcut and resize window by making it wider. You will be logged on as
root
.At the config prompt, type (or copy and paste) the following command:
tcpdump -i 0.0 host 10.1.20.12
PuTTY to the Attacker host from your jumpbox desktop shortcut. You will be logged in as root. I’t will use a pre-loaded public key as the credentials. Accept the warning.
At the config prompt, type (or copy and paste) the following command:
ping 10.1.20.12
The attacker can successfully communicate with a back-end resource behind the BIG-IP DHD.
- Examine the tcpdump window and verify ICMP packets are flowing through the BIG-IP DHD.
Note
The listener for the ICMP packets is the VLAN group.
Cancel the
ping
command, then verify thetcpdump
stops receiving ICMP packets, and then press Enter several times to clear the recent log entries.In the Configuration Utility, in the DoS Protection, Quick Configuration, Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Don’t Enforce”
In the Flood row click the + icon, and then click ICMPv4 flood.
Note
If you minimize by clicking the + icon, it will make seeing the other sections easier.
On the right-side of the page select the drop-down to “Don’t Enforce”
Apply the settings above for TCP SYN flood and UDP Flood and then click Update.
On the Jumpbox in the Attacker PuTTY window type (or copy and paste) the following:
# cd scripts # ls
These are the different scripts we’ll be using during the exercises to simulate DoS attacks.
Type (or copy and paste) the following commands:
for i in {1..10}; do ./icmpflood.sh; done
This script launches 1,000,000 ICMP requests and then repeats for a total of ten occurrences.
View the
tcpdump
window and verify that ICMP attack traffic is reaching the back-end server.Let the attack run for about 15 seconds before moving on.
In the Configuration Utility, open the Statistics > Performance > Performance page.
View the Active Connections and Total New Connections charts.
There is a drastic spike in active connections.
View the Throughput (bits) and Throughput (packets) charts.
There is also a drastic spike in both bits per second and packets per second.
- Open the Security > Event Logs > DoS > Network > Events page.
The log file is empty as we disabled device-level flood protection on BIG-IP DHD.
- On the Jumpbox Attacker shell slowly type Ctrl + C several times
until back at the
scripts
prompt.
Task 2 – Re-enable Device-Level DHD DoS Protection¶
In this task you will re-configure device-level DoS protection, and then issue an ICMPv4 flood and review the results.
In the Configuration Utility, in the Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page select the drop-down to “Enforce”
Note
Bad Source is enabled to be able to add the IP addresses to the blacklist.
In the Flood row click the + icon, and then click ICMPv4 flood.
On the right-side of the page select the drop-down to “Enforce”
Apply the settings above for TCP SYN flood and UDP Flood and then click Update.
Note
This returns the configuration back to factory supplied device level enforcement.
Task 3 – Configure Protected Object-Level IPv4 Flood DHD DoS Protection¶
The DHD device wide protection is enforced for all traffic flowing through the device. For more granular control, we use protected objects and configure mitigation settings for those objects to be enforced. In this task you will configure object-level DoS IPv4 flood protection, and then issue an ICMPv4 flood and review the results.
On the Protect Objects page, in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name ServerNet IP Address 10.1.20.0/24 Port * Protocol All Protocols Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings IPv4 In the IPv4 row click the + icon, and then click ICMPv4 flood.
On the right-side of the page configure using the following information, and then click Create at the bottom of the page.
Detection Threshold PPS Specify: 1000 Detection Threshold Percent Infinite Rate/Leak Limit Specify: 1000 On the Jumpbox in the Attacker PuTTY window re-run the following command:
for i in {1..10}; do ./icmpflood.sh; done
Examine the
tcpdump
window to see if there are any ICMP packets hitting the back-end server.Let the attack run for about 30 seconds before moving on.
In the Configuration Utility, click DoS Protection > Quick Configuration > ServerNet, and then in the IPv4 row click the + icon.
Open the Security > Event Logs > DoS > Network > Events page.
The DoS Source is Volumetric, Aggregated across all SrcIP’s, VS-Specific attack, metric:PPS.
The context column displays /Common/ServerNet, identifying this is protected object-level protection.
The action is Drop.
On the Jumpbox Attacker shell slowly type Ctrl + C several times until back at the
scripts
prompt.In the BIG-IP CLI type Ctrl + C to stop the tcpdump.
Lab 3 – Start Baseline Traffic Generation¶
Task 1 – Create Protected Objects that the baseline traffic will be targeting¶
In the BIG-IP Configuration Utility, open the DoS Protection>>Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Server5 IP Address 10.1.20.15 Port * Protocol All Protocols VLAN Any Protection Settings: Action Log and Mitigate Protection Settings: Silverline (un-selected) Protection Settings: DDoS Settings IPv4, TCP This protected object will be used for the Auto-Thresholding lab.
Task 2 – Run Scripts to start L4 traffic generation – Good Traffic¶
Putty SSH (use the desktop shortcut) to open a shell to the good client system.
Accept the SSH Warning.
You will be logged in as user :
root
. The session is preconfigured to authenticate with a certificate.Start the auto-threshold baselining script with:
# cd ~/scripts # ./baseline_l4.sh
Lab 4 - Multi-vector Demo¶
In this simple demo you will launch a small number of network attacks and show the configuration, logging and reporting capabilities of the Hybrid Defender. The point of this demo is to provide context for a UI walkthrough with some live data.
Task 1 - Access DoS Quick Configuration and display the ServerNet protected object¶
This protected object is defending all ports/protocols for 10.1.20.0/24, which is the network behind the Hybrid Defender. Attacks will be launched at 10.1.20.12, which is an interface on the LAMP server. Verify that the following vectors are configured:
- Add the TCP vectors under DDoS Settings.
- Click Update when finished.
You will now launch the attacks and show the behavior
Open the following tabs in the DHD UI:
DoS Protection->Quick Configuration->ServerNet
Security->DoS Protection->DoS Overview (leave the filter at default: ’DoS Attack’)
Statistics->DoS Visibility
Access the Attacker shell and run the following commands/attack
# cd ~/scripts # ./multivector.sh
Click Refresh on the DoS Overview page. You will see some attacks mitigated by Device Configuration and some mitigated by the more specific settings on the ServerNet Protected Object.
Navigate to Security->Event Logs->DoS->Network->Events.
Click on “custom search…” link.
Drag one of the values from the “Attack Type” column into the custom search builder. From the Action column, drag Drop into the search builder. Click “Search”.
Further explore the DoS Event logs. For example, clear the search and identify the “Stop” and “Start” times for an attack, etc.
Task 2 – View the DoS Visibility Page¶
You can now use the new DoS Visibility page to view statistics about the DoS attacks you submitted during this exercise.
- In the Hybrid Defender WebUI, access the DoS Visibility reporting tool at Statistics->DoS Visibility.
Note
DoS Visibility is a reporting tool, not a real-time monitoring tool. Events are displayed, much like other AVR-based reporting, in 5 minute windows. Do not expect events to be shown here immediately after running an attack. Quicker/real-time monitoring of on-going DoS attacks is best accomplished in the DoS Event Logs and DoS Overview areas of the WebUI.
You should see the attacks in the timeline and a variety of details in the windows. Use the slider to shorten the timeframe if needed. You might have to hit refresh several times.
You can select events from the timeline and see details about the attacks.
In the Attack Duration window view the attack.
- Scroll down in the left-side of the page to view the Attacks section.
View the details at the bottom of the Attacks section.
This table displays details of each attack that has occurred.
Sort this table by Vector.
Scroll down in the left-side of the page to view the Virtual Servers section.
You can see the details of device-wide attacks (Device Level) and protected object-level attacks (/Common/ServerNet).
Scroll down in the left-side of the page to view the Countries section.
View the details at the bottom of the Countries section. This table displays the attack details from each country.
View the various widgets in the panel on the right-side of the page.
Click Network to filter out only the network-level attacks (all the attacks so far have been network-level).
If it’s not already expanded, expand the Virtual Servers widget, and then select /Common/ServerNet.
This filters the results to only attacks at this protected object-level. Notice the changes to the map on in the Countries section.
Click /Common/ServerNet to remove the filter.
Drag the resize handle on the right-side of the main window as far to the left as possible.
Expand the Vectors widget, and then select ICMPv4 flood.
Expand the Client IP Addresses widget.
Question: How many client IP addresses contributed to this attack?
Expand the Countries widget.
Sort the countries by Dropped Requests.
Select China, and then view the changes to both the Client IP Addresses widget and the map.
At the top of the page open the Analysis page.
Note
The requests are still filtered for the ICMPv4 flood results for China.
- Drag the resize handle on the as far to the right as possible.
- Examine the Avg Throughput (Bits per second) graph.
- Place your mouse over the peak in the graph.
Question: What is the Average client in throughput during the attack?
- Feel free to examine more of the Dashboard page and the Analysis page.
- Type Ctrl + C to stop the attack.
Lab 5 - Bad Actor Detection Demo¶
In this demo you will run an attack from specific IP addresses. The Hybrid Defender will be configured to perform bad actor detection, limit the attack on a per-IP basis with more aggressive thresholds and then, based on this detection, automatically blacklist the offending IP address adding them to the (hardware-accelerated) dynamic blacklist.
Task 1 - Open the following tabs in the DHD UI¶
- DoS Protection->Quick Configuration->ServerNet
- Security->DoS Protection->DoS Overview (leave filter at default: “DoS Attack”)
- Statistics->DoS Visibility
- Security->Event Logs->Network->IP Intelligence
Task 2 – Configure the following UDP Flood vectors for ServerNet¶
- DoS Protection->Quick Configuration->ServerNet
- Set the following:
DDoS Settings: UDP, Sweep
- Click UDP Flood
- Detection Threshold PPS: 1000
- Detection Threshold Percent: 500
- Rate Limit: 2000
- Bad Actor Detection - Check
- Per Source IP Detection PPS: 100
- Per Source IP Rate Limit PPS: 2000
- Blacklist Attacking Address
- Detection Time: 15
- Duration: 120
- Click UDP Flood
Click Update when finished.
Access the Attacker system CLI and run the UDP flood attack:
# cd ~/scripts # ./udp_flood.sh
From the menu, select ‘1’ to start the attack
root@attacker-a:~/scripts# ./udp_flood.sh 1)Attack start 2)Attack end 3)Quit # ?
Note
This attack is relatively short-lived. You can launch it again if the attack ends and you are not finished viewing the various reports. Simply type ‘1’ again, to re-run the attack.
In the Hybrid Defender UI, show the Security > DoS Protection >DoS Overview page. Note the blocks by Bad Actor.
In the Hybrid Defender UI, show the Security > Event Logs > Network > IP Intelligence Event Logs. Note the IP addresses that are being added to the denial_of_service blacklist.
In the Hybrid Defender WebUI, show the Statistics > DoS Visibility. Expand the Vectors inspector and select UDP Flood. When it updates, select a flood from the timeline. Note in the Attacks panel the #IPs blocked is 10.
From the menu, select ‘2’ to end the attack
or
# sudo bash
# killall -9 hping3
Lab 6 – Using Auto Thresholding¶
This exercise will simulate a newly configured Protected Object where the security administrator is unsure what values to assign to a few common vectors. Note that auto-thresholding is useful at both the Device and Protected Object levels.
Note
This demo may place significant stress on the demo environment. This may make the DHD UI less responsive. This is unavoidable since for auto-thresholding to block, the attack must be damaging enough to cause stress, which will push the CPU on the Virtual Environment very high. Remember that this is a virtual environment with minimal resources for lab under high stress and that the Hybrid Defender appliances mitigate these attacks in dedicated hardware.
Task 1 – Configure Auto Thresholding¶
On the Good Client, if you have not already done so, start the network baselining. This step is needed if you didn’t start the good traffic generation in Exercise 3 or accidently stopped it.
# cd ~/scripts # ./baseline_l4.sh
In the Hybrid Defender UI, in Quick Configuration, select the Server5 Protected Object and verify that the IPv4 and TCP vectors are all at default thresholds with auto-threshold disabled:
Setting Value All Detection Thresholds 30000 pps All Rate Limits Infinite Auto Thresholding Disabled In the Hybrid Defender CLI (BIGIP ssh window), restart auto-thresholding:
# tmsh run security dos device-config auto-threshold-relearn # tmsh run security dos virtual name Server5 auto-threshold-relearn
In the Hybrid Defender WebUI, in the Server5 Protected Object configuration, enable auto-thresholding for the following vectors: ICMPv4 Flood, TCP SYN Flood, TCP Push Flood, TCP RST Flood, TCP SYN ACK Flood by selecting each vector and clicking the Auto-Threshold Configuration radio button. When all vectors are configured, click Update at the bottom of the screen.
In the Hybrid Defender WebUI, view the Auto Threshold event log by navigating to Security>>Event Logs>>DoS>>Network>>Auto Threshold.
The system is updating the detection thresholds. With auto-thresholding, the system adjusts the detection thresholds based on observed traffic patterns. However, mitigation rate limits are always dynamic based on detected system or protected object stress. If anomalous levels of traffic are running, but there is no stress, the Hybrid Defender will generate alerts but will not block traffic. Under stress, the rate limits are automatically created and adjusted dynamically.
Task 2 – Create Stress to trigger Auto Thresholding and view Reports¶
Let’s create some stress with a Flood attack. In the Attacker CLI start the auto-threshold flood:
# cd ~/scripts # ./autot_flood.sh
This is a long duration attack. You can terminate it with Ctrl+C when finished.
In the Hybrid Defender WebUI, review the Auto Threshold event log. You will see that Rate limits are being automatically set and adjusted to mitigate the flood attack.
In the Hybrid Defender WebUI, view the DoS Overview. Note that the ICMP Flood attack is being mitigated and the rate limit thresholds for each of the auto-threshold vectors have been adjusted based on stress, including vectors that are not detecting or blocking an attack.
Select the filter type to Virtual Server (DoS protected) and Server5 and view how various thresholds are dynamically adjusted based on the stress.
Terminate the attack in the Attacker CLI with Ctrl+C.
After the attack has ended, in the Hybrid Defender WebUI, navigate to the DoS Visibility page. Under Vectors, select ICMPv4 Flood. View various details.
Clean-up: On the Attacker CLI, if the attack is still running be certain to end it with Ctrl-C.
Clean-up: For repeatability, it is necessary to disable the auto-thresholding for the ICMPv4 Flood, TCP RST Flood, TCP Push Flood, TCP SYN ACK Flood and TCP SYN Flood vectors on the Server5 protected object. Switch them back to Manual Configuration.
Clean-up: After disabling auto-thresholding, clear the learning on the Hybrid Defender CLI with:
# tmsh run security dos device-config auto-threshold-relearn # tmsh run security dos virtual name Server5 auto-threshold-relearn
Clean-up: Stop the baseline traffic generation from the good-client if still running using CTRL+C
Lab 7 – Configuring DNS Attack Protection¶
DNS DoS attacks come in many flavors and target different resources. DNS query, reverse flood and amplification attacks are some such DNS attacks.
DNS Query Flood¶
This type of DoS of service attack has a couple possible resource impacts.
- Overwhelm the DNS server’s ability to respond by sending too many requests.
This can be done just by asking for more requests than the server can reply with and prevent the server from servicing legitimate requests. It doesn’t really matter if the clients are spoofed or not, it only matters that the DNS server just can’t keep up.
Mitigation Options¶
DNS DoS mitigation generally requires an awareness of what you’re trying to protect. This allows you to apply the appropriate mitigations and push the problem upstream until the next step is to force it off premises and in to a cloud solution. Load balancing is one remedy to this solution (anycast). Spreading the requests across pools of servers can help mitigate against these types of attacks. DNS Express is another option to increase the capacity of your DNS infrastructure. Layering in DHD DNS DoS vector mitigation also stops common DNS attacks.
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page.
In the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name DNSServer IP Address 10.1.20.14 Port 53 Protocol UDP Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings DNS In the DNS row click the + icon, and then click DNS A Query.
On the right-side of the page configure using the following information, and then click Create.
Detection Threshold PPS Specify: 75 Detection Threshold Percent Specify: 500 Rate Limit Specify: 100
In the Attacker putty window type (or copy and paste) the following command:
./dnsdosrate.sh
Wait for the attack to run for about 30 seconds before moving on.
In the Configuration Utility, review the DoS Overview page.
Change the selection to Virtual Server and DNSServer.
The A query DOS attack vector is now dropping attack traffic.
The script will also record the number of drops if any as a result of the attack rate limit being hit.
DNS Reverse flood¶
Sometimes DNS responses are used in flooding network resources. A small request has a disproportionately larger response and since the transport protocol is UDP it can easily be spoofed. The outbound pipe can easily get congested responding to a smaller number of requests with large responses.
Use tcpdump and dig to view DNS request and response packets. A small request produces a large response. You will open two ssh sessions to the attacker.
Open two windows via Putty to the Attacker CLI (use the shortcut).
In the first ssh window on the attacker start a tcpdump using the following command:
# cd ~/scripts # tcpdump -i lo &
In the second ssh window on the attacker issue a dig against the loop back with “ANY”
# cd ~/scripts # dig ANY floodzone.local @localhost
In the first ssh window on the attacker view the results of the tcpdump : Notice the difference in the size of the request (44) vs the response (628). Your values maybe different. The point is that a small request can generate an enormous response.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes 07:33:55.737892 IP localhost.47406 > localhost.domain: 3896+ [1au] ANY? floodzone.local. **(44)** 07:33:55.738563 IP localhost.domain > localhost.47406: 3896\* 17/0/13 A 10.10.1.2, A 10.10.1.3, A 10.10.1.4, A 10.10.1.1, A 10.10.1.5, SOA, NS ranger.floodzone.local., NS langley.floodzone.local., NS lexington.floodzone.local., NS attacker-a.f5d emo.com., NS saratoga.floodzone.local., AAAA ::1, MX enterprise.floodzone.local. 20, MX hornet.floodzone.local. 40, MX ess ex.floodzone.local. 50, MX yorktown.floodzone.local. 10, MX wasp.floodzone.local. 30 **(628)**
In the second ssh window on the attacker issue a dig against the loop back with a query to RIPE.NET and with DNSSEC for a larger response.
# sudo bash # cd ~/scripts # dig ANY ripe.net @localhost +dnssec
In the first ssh window on the attacker view the results of the tcpdump : Notice the difference in the size of the request (37) vs the response (2715). Your values maybe different. The point is that a small request can generate an enormous response.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes 07:43:44.018212 IP localhost.51272 > localhost.domain: 58304+ [1au] ANY? ripe.net. **(37)** 07:43:44.018889 IP localhost.domain > localhost.51272: 58304$ 18/8/15 RRSIG, SOA, RRSIG, RRSIG, A 193.0.6.139, RRSIG, DNSKEY, DNSKEY, DNSKEY, RRSIG, DS, NS manus.authdns.ripe.net., NS a2.verisigndns.com., NS a1.verisigndns.com., NS tinnie.arin.net., NS sns-pb.isc.org., NS sec3.apnic.net., NS a3.verisigndns.com. **(2715)**
Once you’re done, type ‘fg’ and ‘CTRL+C’ to stop the tcpdump.
root@Attacker:~/scripts# tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
fg
tcpdump -i lo
^C
0 packets captured 0 packets received by filter 0 packets dropped by kernel root@Attacker:~/scripts#
This can easily overwhelm the server or overwhelm the outbound network pipe disrupting traffic responses for legitimate requests and/or other applications.
One industry accepted way to mitigate this type of attack is to rate limit the responses on the DNS servers. More information on Response Rate Limiting can be found here:
https://www.isc.org/wp-content/uploads/2014/11/DNS-RRL-LISA14.pdf
Because DoS policies are applied to traffic flows on ingress to the DHD, response rate limiting isn’t currently available. But you still can limit the types of queries that can disproportionately consume bandwidth. The ANY query used in the previous example is one such example.
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page
In the Protected Objects section click DNSServer.
In the DNS row click the + icon, and then click DNS ANY Query.
On the right-side of the page configure using the following information, and then click Update.
Detection Threshold PPS Specify: 50 Detection Threshold Percent Specify: 500 Rate Limit Specify: 75 In the BIG-IP Configuration Utility, open the Security>>DoS Protection>> DoS Overview page and set the Filter type to “Virtual Server (DoS protected) / DNSServer” Set Auto-Refresh to 20 seconds.
In the attacker ssh window issue dns reverse flood attack as follows:
# cd ~/scripts # ./dnsReverseFlood.sh
Observe the DoS Overview as it gradually starts to drop the ANY queries.
In the attacker ssh window stop the attack by hitting CTRL+C many times
Lab 8 – Configuring L7 Attack Protection¶
In this exercise we will use a protected object and enforce mitigation for low and slow/encrypted layer 7 attacks.
Task 1 – Create Protected Object and Launch Attack¶
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Server1 IP Address 10.1.20.11 Port 443 VLAN (Selected) defaultVLAN (uncheck ANY) Protection Settings: Action Log and Mitigate Protection Settings: Silverline Yes (selected) Protection Settings: DDoS Settings IPv4, TCP Launch attacks without any layer 7 protection configured
Open the following in separate tabs in the Hybrid Defender WebUI:
DoS Protection>>Quick Configuration
Security>>Reporting>>DoS>>Analysis
From a Firefox browser go to https://10.1.20.11. Ignore SSL warning and Add Exception.
Note
This bypasses the Hybrid Defender and accesses the server directly, showing the availability and/or performance of the site directly. Click around a few links. This is the site we will launch an attack against and mitigate.
Verify that the configuration is providing no L7 protections by taking the server offline with a slowloris attack. Note that apache will try to clean up the slow flows, but they will do so inefficiently and the server is impacted (which will show as an outage, missing objects and/or slower responsiveness). Run the slowloris attack from the Attacker CLI:
# cd ~/scripts # ./slowloris.sh
The tool will rapidly show the site offline (10-15 seconds, with trivial traffic load):
Refresh https://10.1.20.11 to show the effects of the attack. [Note that since we are running locally from the Win7 system in a virtualized environment, you may be able to access the site, however it will be slower and often the GIFs will not load. An internet user would not be able to “fight through” the attack to get to the server as often as a system on the local LAN.]
Stop the slowloris attack by using CTRL+C.
Start a more effective Slow Read attack.
This attack is harder for DoS mitigation tools to mitigate and can be very effective even with a tiny number of concurrent connections trickling in very slowly to the server to fly below the radar of network detections. In our example we will open 10 connections per second and read the response data at 1 byte / sec. The attack would be effective even at 1 cps, it would just take a bit longer to build up the connections.
From the Attacker CLI/shell start the slowread attack:
# cd ~/scripts # ./slowread.sh
As soon as the site is down (service available: NO), refresh https://10.1.20.11 to show that it is down/slow/intermittent.
Task 2 – Configure Protection/Mitigation, launch attack and view reports¶
In the Hybrid Defender WebUI, access the Server1 Protected Object.
Enable SSL.
Select the default certificate and key. In your environment you would select a valid/cert key for your application.
Enable ‘Encrypt Session to Server’ to avoid any server reconfiguration.
Enable the HTTPS mitigation family.
Click Update.
View the Attacker CLI/shell. The slow read attack is now no longer showing the site as down (service available: YES) because Proactive Bot Detection has mitigated the attack.
Refresh https://10.1.20.11 to see that the site behavior has returned to normal.
You were able to mitigate an encrypted layer 7 attack quickly and with only a few simple steps.
In the Hybrid Defender WebUI, view various reports in the Security>>Reporting>>DoS>>Analysis
HTTP Report (Scroll towards the bottom) shows Proactive Mitigation.
Stop the Slow Read attack by using CTRL+C.
Lab 9 – Configuring L7 Behavioral Attack Protection¶
In this exercise we will use a protected object and show how behavioral DDoS works.
Task 1 – Create Protected Object and Launch Attack¶
In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name Auction IP Address 10.1.20.101 Port / Protocol 80 TCP VLAN (Selected) defaultVLAN (uncheck ANY) Protection Settings: Action Log and Mitigate Protection Settings: DDoS Settings HTTP Make sure Auction is with a capital “A”.
Under the HTTP section make the following adjustments:
- Set Behavioral to Standard Protection.
- Make sure you check “Request Signature Detection”
- Set Proactive Bot Defense to “Disabled”
- Set DOS tool to “Report”
When finished click Create
From the Good Client CLI, issue the following command.
~/scripts/generate_clean_traffic.sh
Note
This will need to run for approximately 10 minutes.
From the DHD CLI issue the following commands:
#/root/scripts/l7bdos-reset.sh #/root/scripts/l7-mon.sh
Monitor the window. When you see the following number go to 100, you will move on.
The health of the Protected Object will be shown. In general, a healthy system will show a value around .45. If the value is .5 consistently, then for some reason no learning is occurring and you should check your configuration and verify that baselining traffic is hitting the protected object in question.
If the system has detected and is mitigating and attack, or not. This will show in the output of ‘info.attack’ signal. The two numbers in brackets indicate if there is an attack (1 = yes, 0 = no) and if the system is mitigating that attack (1 = yes, 0 = no).
The output will also include the ‘info.learning’ signal, which includes 4 comma-separated values that show the status of the admd behavioral dos learning:
- signal values: [baseline_learning_confidence, learned_bins_count , good_table_size , good_table_confidence]
- baseline learning_confidence in % - How confident the system is in the baseline learning.
- This should be between 80% - 90%
- learned_bins_count - number of learned bins
- This should be > 0
- good_table_size - number of learned requests
- This should be > 4000
- good_table_confidence - how confident, as a percentage, the system is in the good table.
- It must be 100% for behavioral signatures.
From the Attacker CLI issue the following command:
~/scripts/http_flood.sh
Choose option 1, “Attack Auction”
You will see the attack start in the DHD SSH window:
In addition you will see the good client start returning a status of 000 as it is unresponsive. It no longer returns a Status 200. Until the DHD starts mitigation.
Once the DHD has enough data a Stable Signature is detected.
Let this run for 2 minutes. Stop the attack by pressing “Enter”” a couple of times in the Attacker window the choosing option “3” to stop the “Attack”
Note
The DHD does not record the end of the attack right away, it is very conservative, therefore you may have to wait 5 minutes to see the results.
You can see in the top-left that a Behavioral Signature was created.
Click on this link, then click on the Signature to see it.
This concludes the DHD Hands on Labs.
Class 4: Mitigating Real World DDoS Attacks¶
Lab Topology & Environment¶
Access and Credential Summary¶
You will be using the Win7 JUMPBOX to access other systems for all labs. You will use Putty that has been preconfigured with appropriate keys to access the Good Client and the Attacker systems. To run scripts, you will need to have root access, requiring you to ‘sudo bash’ before running attacks, baselines, etc.
System | Username | Password |
---|---|---|
Jumpbox | external_user | f5DEMOs4u |
Hybrid Defender – WebUI/TMUI | admin | f5DEMOs4u |
Hybrid Defender – CLI | root | f5DEMOs4u |
Passive BIG-IP – WebUI/TMUI | admin | f5DEMOs4u |
Good Client | ubuntu | Use key |
Attacker | ubuntu | Use key |
Win-ToolsServer | external_user | f5DEMOs4u |
WebServer / Auction Server | root | default |
Helpful Tips and Tricks¶
Here are a few tips that you can use during the labs. Since the environment and all its components are running in a virtualized environment with limited shared resources you may encounter some slow performance.
- When using the Wireshark tool, it will capture a lot of packets. During DDoS attacks the tool will be overwhelmed. Its recommended that you start the capture and then stop it soon so that you can view the data captured easily.
- If you find that you are not seeing any attacks then go back and check if the Attack you launched is still running. If it has stopped, kindly relaunch it.
- If an attack is not being detected on the DHD check the value of your detection threshold EPS. For an attack to be detected this value must be lower than the attack being launched. Similarly, the rate/leak limit value sets the threshold for dropping the packets.
- During automatic/behavioral mitigations labs there is about 10-15 minutes of baseline traffic learning time for the Hybrid Defender. Use that time to ask questions, chat with F5 Engineers and/or your peers about DDoS mitigations, security and what they are doing in their organization. Additionally, browse around the DoS Visibility tool to see some cool graphical reports that were generated.
- Make sure the name of the Protected Objects you create in various labs matches exactly to what is provided in this guide otherwise the scripts/commands for monitoring learning status will not work as they are tied to specific profile names that get created.
- You will notice that the commands “sudo bash” “cd f5agility” are included in each step. If you are already logged in and have root privileges and in the f5agility folder then kindly ignore those steps. If not, then use them. Basically, you need root level access to execute the scripts and be in the f5agility folder/directory.
- Since the WebUI/TMUI will look the same for the BIG-IP Passive and the Hybrid Defender device make sure that all mitigation/changes are being made to the Hybrid Defender only and the Passive device is used only for visibility.
- Don’t forget to use CTRL+C to break and stop the attacks so that you get better responses from various tools once you have enough data.
- When starting a new capture in WireShark always select continue without saving when prompted.
- Use Right click and “Open in new tab” to browse various DHD menus (Overview, Event Logs,etc) so you don’t have to go back and forth.
- STOP all attacks, good traffic baseline scripts after end of each lab before proceeding to the next lab.
- Use the PuTTY shortcuts on the desktops to access various shells. The PuTTY window has a title on top so that you know which shell you are in. If you get a Security Alert for the Servers Host Key just click YES to proceed to connect to the shell.
Accessing the Lab Environment¶
Use RDP client and connect to your Windows Jumpbox IP and the Win-ToolsServer IP
Note: Use the show options to provide
User name: external_user. Password: f5DEMOs4u
Click YES at the warning
All Exercises/Tasks are to be completed from the Windows Jumpbox. There are various shortcuts – Chrome Incognito, Putty shortcuts, on the Jumpbox that you will use through the exercises.
Introduction to DDoS Hybrid Defender¶
F5 DDoS Hybrid Defender (DHD) protects your organization against a wide range of DDoS attacks using a multi-pronged approach. By combining on-premises and cloud technologies, analytics, and advanced methods, DDoS Hybrid Defender is a hybrid solution that detects network and application layer attacks and is easy to deploy and manage.
DDoS Hybrid Defender mitigates against the full spectrum of DDoS attacks including:
- Network capacity attacks
- DNS and SIP protocol volumetric attacks
- HTTP and HTTPS volumetric attacks
- HTTP and HTTPS CPU-based (heavy URL) attacks
You can specify which objects to protect on the network, assigning the appropriate protections to network devices and application servers, and prevent attackers from exhausting network resources and impacting application availability.
Deployments:
The deployment you use for DDoS Hybrid Defender depends on the needs of your organization. For maximum DDoS protection, it is recommended that you deploy DDoS Hybrid Defender inline. However, it can also be deployed out of band, or in locations where symmetric data flows are not guaranteed.
Typical locations for the placement of DDoS Hybrid Defender are at the edge of the network or at the edge of the data center
Inline deployment
DDoS Hybrid Defender provides maximum protection when deployed inline in one of two ways:
- Bridged mode with VLAN groups (This is default and we will use in our labs)
- Routed mode
Out of band deployment
You can deploy DDoS Hybrid Defender out of band in two ways:
- Set up a Layer 2 switch with span ports so that it mirrors traffic onto DDoS Hybrid Defender. (Our passive device is setup this way in our labs)
- Configure network devices so that they send NetFlow data to DDoS Hybrid Defender.
Module 1: Environment Review¶
Lab 1.1 – Review Tools and Environment¶
You are the security engineer for Acme corporation. Your organization has recently seen a lot of outages in your network and applications. Some of these have been due to DDoS attacks and the outages have caused a significant loss of revenue as well as reputational impact. You have made the wise decision to invest in a world class leading edge DDoS mitigation solution and have the F5 DHD installed in your environment. It’s been configured in the Layer 2 inline mode and is now available to you to enforce DDoS mitigations.
Tools:
#. In our lab we have an additional DHD available to you in a passive mode. It’s basically setup on SPAN ports (out of band deployment) to provide you visibility.
#. The Win-ToolsServer is also installed to listen on SPAN port and has Wireshark available for visibility.
Let’s get familiar on how to use these tools.
Note: Not all attacks will be visible in both tools. So, use the tools accordingly. This is done purposefully so that you get into the habit of troubleshooting/fighting attacks in the real world.
Use a web browser (Chrome in incognito mode) to log into the WebUI of the Passive DHD at https://10.1.1.246 or use the bookmarked shortcut. Accept the SSL warning and proceed to connect.
Username: admin
Password: f5DEMOs4u
- Click Security>>Event Logs>>DoS>>Network>>Events
- Click Security>>DoS Protection>>DoS Overview (Tip: Right Click and open link in new tab/window)
- You will use the above two screens on the Passive DHD for visibility of traffic/attacks.
- On the Win-Tools Server launch Wireshark by using the shortcut link on desktop and then click on the blue shark fin on top left corner to start capturing data. (Tip: Use the Red Square button to stop captures when needed)
Lab 1.2 – Launch an attack and view traffic¶
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab1-2.sh
- View Wireshark and notice the ongoing captures.
- What type of traffic do you notice? As you can see these are all ICMP requests/responses and a lot of them. What are the IP addresses involved? Can you identify the attacking IP? (Tip: Did you review the lab network diagram?)
In the Passive DHD Windows what do you notice? (Tip: You may need to click Search button/Refresh button or set Auto Refresh)
As you can see the visibility is better in terms of the Attack Vector and number of packets in/sec on the passive DHD.
It’s up to you on which tool you may want to use for the remaining labs. If you are comfortable with WireShark then use that or use the Passive DHD or both. As noted previously you will have to visit both tools to see where you can gather some visibility to fight a real-world DDoS attack.
Use CTRL+C in the attacker shell to stop the attack.
Module 2: Manual Mitigations¶
Lab 2.1 – Device Level Protection for Mitigating Attacks.¶
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-1.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- On the Passive DHD identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved?
- Let’s mitigate this attack using Device Level mitigation.
Log into the DHD https://10.1.1.245 accept the SSL warning and proceed to connect with credentials provided.
- In the Configuration Utility, go to DoS Protection>>Quick Configuration.
- In the Device Protection section click Device Configuration.
- In the Flood row click the + icon, and then click ICMPv4 flood.
- On the right-side of the page select the drop-down to “Mitigate”
Parameter | Value |
---|---|
Mitigation | Fully Manual |
Detection Threshold EPS | 100 |
Detection Threshold Percent | 500 |
Rate/Leak Limit | 500 |
- On the Hybrid Defender you will now see the attack is being mitigated (Where will you check this? Tip: It’s the same places that you are looking on the Passive device). You have successfully mitigated a network flood single vector attack. Use CTRL+C in the attacker window to stop the attack.
Lab 2.2 – Device Level Protections for Mitigating Attacks¶
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-2.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- On the Passive DHD identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved?
Mitigate this attack using Device Level mitigation steps like those that you did in Lab 2.1 above.
Lab 2.3 – Device Level Protections for Mitigating Attacks¶
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-3.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved? Look closely and you will notice that there is a range of destination IPs that are being targeted and a lot of SYN, Retransmit, Out of Sequence, RST packets. This looks like someone is trying to run a scan against your network. How will you mitigate against this? They are “Sweep”ing your network.
- In the Configuration Utility, in the Device Protection section click Device Configuration.
- In the Single Endpoint row click the + icon, and then click Single Endpoint Sweep.
- On the right-side of the page select the drop-down to “Mitigate”
Parameter | Value |
---|---|
Detection Threshold EPS | 100 |
Rate/Leak Limit | 500 |
Packet Types (Selected) | All IPv4 |
- On the Hybrid Defender you will now see the attack is being mitigated. This attack is short lived so make sure you launch it again if it has stopped to see the mitigation. You have successfully mitigated a sweep flood attack. Use CTRL+C in the attacker window to stop the attack.
Lab 2.4 – Device Level Protections for Mitigating Attacks¶
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-4.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- On the Passive DHD identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved?
- Use the manual mitigations steps you learned in previous tasks to mitigate against all the attack vectors that you have identified.
- Use CTRL+C in the attacker window to stop the attack.
Lab 2.5 – Device Level Protections for Mitigating Attacks¶
You received a call that a lot of users are intermittently getting a page cannot be displayed for various applications. Your Network Operations Center has stated that none of their monitoring systems for those applications are reporting any outages. The NOC tools monitor application health using the application URLs like http://10.1.20.12/index.php and so on. Your users are using the application using the FQDNs. You suspect that there is an ongoing DDoS attack and you need to identify it and mitigate against it.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-5.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- Let’s look at an alternate way to see which vector is being triggered so that you can identify the attack. If in your environment you had no tools like the Wireshark or the Passive DHD device, you can still identify the attack. While the event logs, DoS Overview screens are populated only when an attack is detected based on the threshold values set, if the attack doesn’t trigger the detection threshold you will not see it in the Overview and Event Logs.
- In the Configuration Utility of the Hybrid Defender, go to DoS Protection>>Quick Configuration.
- In the Device Protection section click Device Configuration.
- In the DNS row click the + icon, and then view the Current Device Statistics Section. You can see that we are triggering a vector and registering the packets for that vector even though we have the default detection/mitigation configured for it.
- Alternately there is a CLI command also available to view the attack vector that is being triggered. Open a putty shell to the Hybrid Defender (use shortcut on desktop), login with the credentials: root/f5DEMOs4u and then :
# cd f5agility
# ./show_attackvector_stats.sh
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved? Hint: (Wireshark) Destination IP, Targeted Port and Protocol used.
- Use the manual mitigations steps you learned in previous tasks to mitigate against the attack vector that you have identified.
- Use CTRL+C in the attacker window to stop the attack.
Lab 2.6 – Protected Object Level Protections for Mitigating Attacks¶
You mitigated a DNS vector attack above at device level. You have again received a call that a lot of users are intermittently getting a page cannot be displayed for various applications. Your Network Operations Center has stated that none of their monitoring systems for those applications are reporting any outages. The NOC tools monitor application health using the application URLs like http://10.1.20.12/index.php and so on. Your users are using the application using the FQDNs. You suspect that there is an ongoing DDoS attack and you need to identify it and mitigate against it. You don’t want to implement a mitigation for a vector device wide and want to specifically mitigate the suspected victim server.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-6.sh
- On the WireShark start a capture/stop and identify the ongoing attack.
- On the Passive DHD identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved?
- In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page.
- In the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | DNSServer |
IP Address | 10.1.20.14 |
Port | 53 |
Protocol | UDP |
Protection Settings: Action | Log and Mitigate |
Protection Settings: DDoS Settings | DNS |
- In the DNS row click the + icon, and then click DNS A Query.
- On the right-side of the page configure using the following information, and then click Create.
Parameter | Value |
---|---|
Detection Threshold EPS | Specify: 10 |
Detection Threshold Percent | Specify: 500 |
Mitigation Threshold EPS | Specify: 100 |
- On the Hybrid Defender you will now see the attack is being detected/mitigated. You have successfully mitigated a DNS A Query flood. Use CTRL+C in the attacker window to stop the attack.
Lab 2.7 – Protected Object Level Protections for Mitigating Attacks¶
There has been a high-profile DDoS attack and you must provide Law Enforcement some details on the offending IP addresses. In your environment at any given time you have a few hundred thousands of IP addresses observed on your network. You want to identify a few offending IP addresses and blacklist them so that you can provide the details to Law Enforcement.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-7.sh
- On the WireShark start a capture and identify the ongoing attack.
- Did you identify the attack? What type of attack is it? What Source IPs and Destinations IPs are involved? Make a note of the protocol of attack and the destination IP (target).
- We will build a protected object and use Bad Actor Detection and Black Listing.
- In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page
- In the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | BadActorServer |
IP Address | 10.1.20.12 |
Port | * |
Protocol | All |
Protection Settings: Action | Log and Mitigate |
Protection Settings: DDoS Settings | UDP |
- In the UDP row click the + icon, and then click UDP Flood.
- On the right-side of the page configure using the following information, and then click Create.
Parameter | Value |
---|---|
Detection Threshold PPS | Specify: 100 |
Detection Threshold Percent | Specify: 500 |
Mitigation Threshold EPS | Specify: 200 |
Bad Actor Detection | Checked |
Per Source IP Detection Threshold | 100 |
Per Source IP Mitigation Threshold | 30 |
Blacklist Attacking Address | Checked |
Sustained Attack Detection Time | 15 |
Category Duration Time | 120 |
- On the Hybrid Defender you will now see the attack is being detected/mitigated.
- View the offending IP addresses at Security>>Event Logs>>Network>>IP Intelligence
- View the Shun list / Blacklist at Security>>Event Logs>>Network>>Shun
- You have successfully identified the Bad Actors and put them in a Blacklist. Use CTRL+C in the attacker window to stop the attack.
Lab 2.8 – Whitelisting¶
You get a call from your QA team that is running load runner scripts against your application server 10.1.20.12 that they are seeing packets being dropped. You ask them what’s the source IP address of the server they are running the load runner script from and they provide you with 10.1.17.225.
- Why do you think their packets are being dropped? Hint: Check the blacklist (Event Logs>>Network>>Shun). They have been added to that list. You will now need to maintain the mitigations in place and only allow 10.1.17.225 to not be enforced with any DDoS mitigations going to 10.1.20.12.
- Go to the protected object 10.1.20.12 and add the IP to the whitelist.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab2-7.sh
- View the offending IP addresses at Security>>Event Logs>>Network>>IP Intelligence and Security>>Event Logs>>Network>>Shun and confirm that 10.1.17.225 is not being added to the list.
- You have successfully whitelisted an IP to bypass DDoS mitigations. Use CTRL+C in the attacker window to stop the attack.
Lab 2.9 – BOT Defense for Application Attacks.¶
HTTP DoS attacks are very popular. Some can be in form of HTTP Floods and some can be low and slow attacks (slow loris, slow post, slow read). They have been used by BOTS to bring down a site. Sometimes even though the BOTS don’t bring the site down they demand for you to stand up additional infrastructure to support the traffic they are generating costing your organization a significant spend when it can be mitigated and avoided. Your organization just published a brand-new web application. As soon as it was available to public you started getting calls that the site is sometimes unavailable and slow to respond. Based on the predicted traffic patterns one server was enough to handle the valid user load. The application team viewed the web server logs and noticed that there is 30% additional traffic then predicted from what seems like automated tools. Your IT management has asked you to provide a solution on what’s driving up the traffic to the server and potentially mitigate it. You will now learn how to manually mitigate BOT traffic.
- Open a PuTTY shell to the WebServer (use the shortcut on the desktop). Login with credentials: root/default. You will use the webservers log to monitor the requests coming to the server. Once logged into the WebServer shell:
# cd /usr/local/apache/logs
# tail -f access_log
- Hit the Enter key a few times so that you can see incoming requests clearly in the blank space.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack to simulate BOT traffic:
# sudo bash
# cd f5agility
# ./lab2-9.sh
- We are just simulating 25 requests so that it’s a controlled environment and you can view the requests/logs.
- View the WebServer shell where you have the tail -f access_log running. Do you see the requests come in? What’s the source IP address of the requests?
- As you can see the site is available to everyone including BOTS. You have not set this up on the DHD and hence no BOT protection is applied.
- You will now publish the website through the DHD with needed protections.
- In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | WebServer |
IP Address | 10.1.20.101 |
Port | 80 |
VLAN (Selected) | defaultVLAN (uncheck ANY) |
Protection Settings: Action | Log and Mitigate |
Protection Settings: DDoS Settings | IPv4, TCP, HTTP |
- By simply creating the Protected Object and applying HTTP protections the BOT protections are automatically turned on. Everyone will now access the web application through the DHD with mitigations enforced.
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack to simulate BOT traffic:
# sudo bash
# cd f5agility
# ./lab2-9.sh
View the WebServer log (tail -f access_log) in the shell. You will not see requests come through this time from the attacker.
View the mitigation in Security>>Event Logs>>Bot Defense>>Requests. All the requests from the BOT are blocked.
Open a firefox browser on the Jumpbox and go to http://10.1.20.101. This request will open your web application and its not blocked as it’s not a BOT. You will also see the request in the WebServer log shell.
View the valid request from your browser in the DHD in Security>>Event Logs>>Bot Defense>>Requests. You will notice that valid requests are being challenged and allowed only after a valid response. Note: There is a default grace period of 300s when the mitigation is implemented so some requests are allowed as grace. This is Proactive BOT defense in action.
View the BOT Defense in Security>>Reporting>>DoS>>Analysis and look at the graph under HTTP -> Transaction Outcomes. Please be patient as these graphs are usually populated with a delay.
You have successfully mitigated BOT traffic to your application. CTRL+C in all shell windows and close them all.
Module 3: Automatic Mitigations¶
Lab 3.1 – Auto Thresholding for Mitigating Attacks.¶
Your organization is about to launch a new marketing campaign and there is a website that will host the content. You want to make sure that the application is protected against DDoS attacks but are not sure what traffic patterns are or what values to set for detections/rate limits/mitigations. You will create a Protected Object for the marketing website and use automatic mitigations.
- In the BIG-IP Configuration Utility, open the DoS Protection>>Quick Configuration page and in the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | MarketingServer |
IP Address | 10.1.20.15 |
Port | * |
Protocol | All Protocols |
Protection Settings: Action | Log and Mitigate |
Threshold Sensitivity | High |
Protection Settings: DDoS Settings | IPv4, TCP, |
Generate some good traffic to the marketing server.
- Putty SSH (use the shortcut) to open a shell to the good client system.
- Login as user: ubuntu. The session is preconfigured to authenticate with a certificate.
- Start the auto-threshold baselining script with:
# sudo bash
# cd f5agility
# ./auto_baseline.sh
Let this baseline traffic run for at least 10 minutes before proceeding to the below step.
In our lab we need to roll back the device level protection so that it doesn’t mitigate the stress we are generating for the auto-threshold on the MarketingServer.
- In the Configuration Utility, in the Device Protection section click Device Configuration.
- In the Flood row click the + icon, and then click ICMPv4 flood.
- On the right-side of the page select the drop-down to “Detect-Only”
Parameter | Value |
---|---|
Mitigation | Fully Manual |
Detection Threshold EPS | Infinite |
Detection Threshold Percent | 500 |
Rate/Leak Limit | Infinite |
Click Update at the bottom of the screen. This will allow our attack to pass through to the automatic mitigation profile of the MarketingServer that we are configuring below.
In the Hybrid Defender WebUI, for the MarketingServer Protected Object configuration, enable auto-thresholding for the following vectors: ICMPv4 Flood, TCP SYN Flood, TCP Push Flood, TCP RST Flood, TCP SYN ACK Flood by selecting each vector and clicking the “Fully Automatic” Configuration radio button. When all vectors are configured, click Update at the bottom of the screen.
In the Hybrid Defender WebUI, view the Auto Threshold event log by navigation to Security>>Event Logs>>DoS>>Network>>Auto Threshold.
The system is updating the detection thresholds. With auto-thresholding, the system adjusts the detection thresholds based on observed traffic patterns. However, mitigation rate limits are always dynamic based on detected system or protected object stress. If anomalous levels of traffic are running, but there is no stress, the Hybrid Defender will generate alerts but will not block traffic. Under stress, the rate limits are automatically created and adjusted dynamically.
Generate some stress by launching an attack.
Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack:
# sudo bash
# cd f5agility
# ./lab3-1.sh
Keep on refreshing the Auto Threshold event log Security>>Event Logs>>DoS>>Network>>Auto Threshold and observe how the values are changing dynamically. Even though our attack is ICMPv4 flood the other vectors that are set to Fully Automatically are also being adjusted dynamically.
View Security>>DoS Protection>>DoS Overview. Notice how automatic detection and mitigation is happening as stress varies.
Stop all scripts and attacks using CTRL + C.
Lab 3.2 – Behavioral L4 for Mitigating Attacks¶
In this lab you will use the Hybrid Defender’s network behavioral DoS analysis capabilities and its ability to interpret behavioral history and stress to automatically generate and enforce a precise, dynamic signature. This capability allows the granular filtering of the good from the bad, which is a major challenge in DoS mitigation. The bad must be accurately identified to mitigate the DoS attack, particularly if the attack changes over time. Enforcement of a very precise signature, with enforcement thresholds based on system or network stress signals, dramatically reduces false positives—increasing network and application availability.
- In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page
- In the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | BaDoSL4Server |
IP Address | 10.1.20.13 |
Port | * |
Protocol | All Protocols |
Protection Settings: Action | Log and Mitigate |
Protection Settings: DDoS Settings | IPv4, TCP, L4 Behavioral |
- In the L4 Behavioral row click the + icon.
- Configure under Dynamic Signatures using the following information, and then click Create.
Parameter | Value |
---|---|
Learn Only | Unchecked |
Mitigation Sensitivity | High |
- Putty SSH (use the shortcut) to open a shell to the good client system.
- Login as user: ubuntu. The session is preconfigured to authenticate with a certificate.
- Start the behavioral L4 baselining script with:
# sudo bash
# cd f5agility
# ./baseline_L4.sh
You can monitor the learning progress on the DHD.
- Putty SSH (use the shortcut) to open two shells to the HybridDefender.
- Login as user: root and password provided.
- View the behavioral L4 baselining learning with following in 1st shell. Notice the learning phase In Progress.
# cd f5agility
# ./show_baseline_L4_status.sh
- View the behavioral L4 baselining bins populating in 2nd shell.
# cd f5agility
# ./show_baseline_L4_bins.sh
- While the learning is happening, we need to turn off some manual mitigations at Device Level as they will block our attack that is going to create stress to trigger dynamic signatures.
- In the Configuration Utility, in the Device Protection section click Device Configuration.
- In the Flood row click the + icon, and then change click TCP SYN Flood, TCP SYN Oversize and change the attack vector to “Detect-Only”.
- In the Single Endpoint row click the + icon, and then change click Single Endpoint Sweep and change the attack vector to “Detect-Only”.
Make sure the status is changed from “In Progress” to “Finished” for the learning phase on the DHD before proceeding to the next steps below (about 15 minutes)
- Access the Attacker System CLI/shell and launch the attack:
# sudo bash
# cd f5agility
# ./lab3-2.sh
On the Hybrid Defender you will now see the attack is being detected/mitigated. . Did you notice the dynamic signatures in DoS Overview window? Give it a couple of minutes and it will show up. You can view the signature Security>>DoS Protection>>Signatures under Dynamic Signature section. Click on the “Network” (not the signature hyperlink) to view details of the signature.
Use CTRL+C in all shells - attacker, good traffic, DHD to stop all scripts.
Lab 3.3 – Behavioral L7 for Mitigating Attacks¶
In this lab you will use the Hybrid Defender’s application behavioral DoS analysis capabilities and its ability to interpret behavioral history and stress to automatically generate and enforce a precise, dynamic signature. This capability allows the granular filtering of the good from the bad, which is a major challenge in DoS mitigation. The bad must be accurately identified to mitigate the DoS attack, particularly if the attack changes over time. Enforcement of a very precise signature, with enforcement thresholds based on system, network or application stress signals, dramatically reduces false positives—increasing network and application availability.
- In the BIG-IP Configuration Utility, open the DoS Protection > Quick Configuration page and in the
- In the Protected Objects section click Create.
- Configure a protected object using the following information, and then click Create.
Parameter | Value |
---|---|
Name | BaDoSL7Server |
IP Address | 10.1.20.20 |
Port | 80 |
Protocol | TCP |
Protection Settings: Action | Log and Mitigate |
Protection Settings: DDoS Settings | IPv4, TCP, HTTP |
- In the HTTP row click the + icon.
- Click Behavioral and in the right pane configure using the following information.
Parameter | Value |
---|---|
Mitigation | Standard Protection |
Request Signature Detection | Checked |
- Click Proactive Bot Defense and in the right pane configure using the following information.
Parameter | Value |
---|---|
Mitigate Action | Disabled |
- Click DOS Tool and in the right pane configure using the following information, and then click Create.
Parameter | Value |
---|---|
Mitigate Action | Report |
Putty SSH (use the shortcut) to open two shells to the good client system.
- Login as user: ubuntu. The session is preconfigured to authenticate with a key.
- Start the behavioral L7 baselining script in both shells with:
# sudo bash
# cd f5agility
# ./baseline_L7.sh
Select 1) Increasing in first shell and 2) Alternate in the second shell.
You will see a few 0000 statuses as there are certain bad requests in the script. But majority of status is 200s.
You can monitor the learning progress on the DHD.
- Putty SSH (use the shortcut) to open a shell to the HybridDefender.
- Login as user: root and password provided.
- View the behavioral L7 baseline learning with following. Notice the learning phase In Progress.
# cd f5agility
# ./show_L7BaDoS_learning.sh
The output is like this:
”vs./Common/BaDoSL7Server+/Common/BaDoSL7Server.info.learning:[62.0614, 6, 7061, 100]”
It will be 0.00 for a while (in above example output 62.0614 is the average approximation to the learned baselines)
For this demo, wait until you have reached at least 80.00-90.00 (the first number in the output). This should happen after about 8-10 minutes. Once you see 80.00 and above you can move to next steps.
The longer it runs, the better it is, because the system is self-adjusting permanently.
Make sure the status is “80.00-90.00” range (the first number in the output) for the learning phase on the DHD before proceeding to the next steps (about 10 minutes). Once you see 80.00 and above you can move on.
- Hit CTRL+C in the DHD Shell and stop this learning status. We will now use this Shell window to see the dynamic signature that is generated.
- Keep this shell window easily viewable. Behavioral L7 mitigation is very dynamic and hence based on the environmental conditions, underlying infrastructure for your lab instance some of you may see the Signature quickly appear and vanish, some may not see it and some will see it longer. Basically, the Signature mitigation is triggered and then by default the offending IP is added to Bad Actor/Shun list and the signature disappears if the system identifies it’s no longer needed for mitigation.
# ./show_dos_signature.sh
- Access the Attacker System CLI/shell (use putty shortcut on Jumpbox) and launch the attack. Open TWO shells. In first shell:
# sudo bash
# cd f5agility
# ./lab3-3.sh
Choose 1) Attack Start – Similarity
- In Second shell:
# sudo bash
# cd f5agility
# ./lab3-3.sh
Choose 2) Attack Start – Score
As soon as the attack is started you will see that your baseline traffic status of 200s in the good client is now suddenly going to 0000. Wait for a couple of minutes till it returns to a lot more 200s. (Keep the eye on the DHD Shell for Signature)
On the Hybrid Defender Shell you will now see the attack is being mitigated and a signature may appear (see note above).
View Bot Defense logs. Security>>Event Logs>>Bot Defense>>Requests
View Bad Actor Log/Blacklist and notice the offending IP is added to the list. Security>>Event Logs>>Network>>Shun
Use CTRL+C in all open shell windows (Attacker, Good Client, Hybrid Defender) to STOP all traffic and scripts. Close out all windows
Multilayer DDoS Protection¶
Introduction¶
THE PROBLEM
On-premises DDoS defenses can be very effective for blocking most DDoS attacks locally and, being Always-On, can block most attacks immediately. However, they are useless in the case of large volumetric attacks. On the other hand, while Cloud-based DDoS protection (On-Demand) works well for volumetric attacks, it struggles with much slower mitigation response, increased latency, higher operational complexity and the inability to handle HTTPS encrypted attacks due to its asymmetric nature.
THE SOLUTION
Thoroughly and effectively protect your critical web applications from all types of DDoS attacks with a combination of On-Premises and Cloud-Based DDoS services, leveraging multilayer protection techniques that are able to mitigate volumetric attacks, application-level attacks and HTTPS encrypted attacks while minimizing both application downtime and business impact. Intelligent application attacks, encrypted or not, can be handled on-premises with F5’s DDoS Hybrid Defender (DHD) which provides next-generation DDoS defense to ensure real-time, Always-ON, protection while large volumetric attacks are handled by F5’s Silverline DDoS Protection cloud service which, working On-Demand, detects and mitigates DDoS attacks in real time.
F5 Silverline¶
F5 Silverline is a cloud-based, fully managed security service for WAF and DDoS protection. F5 Silverline provides Enterprise customers proven security technologies coupled with world-class security professionals. F5’s security experts are an extension to the customer’s staff and allow them to defeat the largest and most complex attacks.
The primary customer benefits to F5 Silverline include:
- Minimize the risk of data breach and downtime
- Enhance security visibility to their application state
- Reduced operational expense and capital investment required for application security
- Ensure timely detection and fast restoration of services in the event of an attack
F5 DDoS Hybrid Defender¶
F5® DDoS Hybrid Defender™ (DHD) protects your organization against a wide range of DDoS attacks using a multi-pronged approach. By combining on-premises and cloud technologies, analytics, and advanced methods, DDoS Hybrid Defender is a hybrid solution that detects network and application layer attacks and is easy to deploy and manage.
DDoS Hybrid Defender mitigates against the full spectrum of DDoS attacks including:
- Network capacity attacks
- DNS and SIP protocol volumetric attacks
- HTTP and HTTPS volumetric attacks
- HTTP and HTTPS CPU-based (heavy URL) attacks
You can specify which objects to protect on the network, assigning the appropriate protections to network devices and application servers, and prevent attackers from exhausting network resources and impacting application availability.
Deployments:
The deployment you use for DDoS Hybrid Defender™ depends on the needs of your organization. For maximum DDoS protection, it is recommended that you deploy DDoS Hybrid Defender inline. However, it can also be deployed out of band, or in locations where symmetric data flows are not guaranteed.
Typical locations for the placement of DDoS Hybrid Defender are at the edge of the network or at the edge of the data center
Inline deployment
DDoS Hybrid Defender provides maximum protection when deployed inline in one of two ways:
- Bridged mode with VLAN groups (This is default and we will use in our labs)
- Routed mode
Out of band deployment
You can deploy DDoS Hybrid Defender out of band in two ways:
- Set up a Layer 2 switch with span ports so that it mirrors traffic onto DHD
- Configure network devices so that they send NetFlow data to DDoS Hybrid Defender
Hybrid Defender Setup¶
Getting Started¶
Lab Diagram¶
Note
You may have noticed that although clients (goodclient, attacker) and server (LAMP) are siting at the same network subnet [10.1.20.0/24], they’re in different VLANs actually (internal - ID 20 vs external - ID 10). Those two VLANs will be grouped toghether (VLAN Group) and act like a single Layer-2 broadcast domain.
Networking Info¶
IP addressing, Out of Band management, and credentials for all components:
Component VLAN/IP Address(es) Credentials jumphost
- Management: 10.1.0.51
- internal: 10.1.20.51
f5student
/[will be provided]
attacker
- Management: 10.1.0.52
- internal: 10.1.20.52
f5student
/[will be provided]
goodclient
- Management: 10.1.0.53
- internal: 10.1.20.53
f5student
/[will be provided]
lamp
- Management: 10.1.0.252
- internal: 10.1.20.252
f5
/[will be provided]
F5-DHD
- Management: 10.1.0.244
- internal: 10.1.20.244
root
/[will be provided]
Accessing the lab environment¶
Open a browser and go to http://training.f5agility.com, then enter your Class# and Student# as provided by your instructor.
Look for the jumphost virtual machine. Use the RDP client of your choice and work from there, you are going to use it for all labs.
Hint
You can use either use the PUTTY client provisioned on your jumphost desktop, or native shell prompt in order to access both goodclient and attacker virtual machines. Private keys have been configured in advance so you won’t need passwords. A few scripts require root access. Don’t forget to sudo before running attacks, baselines, etc.
Run the following scripts from both goodclient and attacker hosts. It’s going to sync the tools to be used in the entire lab.
~/update_tools.sh
Re-License your DHD Device¶
Important
For Silverline device registration to function properly the Hybrid Defender device must have a unique device ID, which is comprised of unique attributes like Base MAC and registration key.
For the following steps please use the registration key provided by your instructor.
Go to System->License and then click on Re-activate.
Edit the Base Registration Key, replacing it by the new license key. The Activation Method option must be manual. Then click Next.
Select all in the Dossier frame and copy it. Click on Click here to access F5 Licensing Server.
Agreee with the contract terms, copy the contents in the license frame, then click Next ”.
Go back to your F5 DHD and paste the contents copied from the above. License and click Next.
Hint
The BIG-IP will restart daemons and a window will pop up indicating system configuration has changed. Please wait for it to reconnect and click Continue. Your device is now licensed. Click Next
Perform Initial DHD Network Configuration¶
- In the BIG-IP Configuration Utility, open the DoS Protection-> Quick Configuration page.
- Open the Network Configuration page, then In the Default Network section click defaultVLAN.
- Configure the Default Network settings as follows, the click on Done Editing
Internal VLAN tag: blank Internal Interfaces: 1.2 (Click untagged/Add) External VLAN tag: blank External Interfaces: 1.1 (Click untagged/Add) IP Address/Mask: 10.1.20.244/24
- In the Routes section click Create.
- Configure the route using following information, and then click Done Editing, and then click Update.
Route name: default Destination: 0.0.0.0 Netmask: 0.0.0.0 Gateway Address: 10.1.20.2
- By this time you should be able to reach the LAMP server from both attacker and gooclient machines. Open up a terminal shell with both machines and confirm the can reach out to the LAMP server before moving forward.
f5student@attacker:~$ ping -c 3 server1 PING server1.f5demo.com (10.1.20.11) 56(84) bytes of data. 64 bytes from server1.f5demo.com (10.1.20.11): icmp_seq=1 ttl=64 time=9.73 ms 64 bytes from server1.f5demo.com (10.1.20.11): icmp_seq=2 ttl=64 time=6.21 ms 64 bytes from server1.f5demo.com (10.1.20.11): icmp_seq=3 ttl=64 time=5.88 ms --- server1.f5demo.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 5.880/7.277/9.736/1.744 ms f5student@attacker:~$
Register DHD Device with Silverline¶
For Silverline signaling we will be leveraging both the DHD built-in signaling, as well as bandwidth utilization reporting for Hybrid DDoS protection.
Go to System-> Platform menu and change the hostname as below. This will make easier to identify alerts from your particular device in the Silverline Portal. When finished, click Update.
dhd-[student#].latam.f5demo.com
In Device Management->Devices select the device and then click Change Device Name.
Update the device name to match the hostname you have chosen. Important: Use your student number.
Open a terminal sesson with the Hybrid Defender and restart services:
bigstart restart
Now proceed with the Silverline registration. Go to DoS Protection-> Quick Configuration-> Silverline. Fill out the Authentication Credentials fields as follows, then click Update.
username dhd2018us@f5agility.com password [will be provided] Service URL https://api.f5silverline.com
Hint
That screen provides no feedback when the authentication actually works, so no worries. Go to the next step unless you got an error message here.
From another tab in your browser, access the Silverline Portal https://portal.f5silverline.com using same DHD admin account.
Navigate to Config-> Hybrid Config-> Hybrid Device Management
Enter the hostname of your DHD device in the Search field. Verify that you have both registrations. Approve them and you’re done!
Module - Network Level DoS Protection¶
In this module you will learn how the F5 Hybrid Defender protects from several network level DDoS vectors.
Lab – Launching Network-Level Flood Attacks¶
The idea in this lab is to observe how poorly the application performs when the network is under attack.
Configure DHD Device Bandwidth Thresholds¶
In the Configuration Utility, open the Protected Objects page.
In the Network Protection section click Create.
Configure as follows then click Save.
Maximum Bandwidth: Specify 100 Scrubbing Threshold: Type Percentage Scrubbing Threshold: Value 60 Advertisement Method Silveline Scrubber Details: Type Advertise All
Turning Device-Level Protection off¶
In the Configuration Utility, in the Device Protection section click Device Configuration.
In the Bad Headers row click the + icon, and then click Bad Source.
On the right-side of the page configure using the following information.
Detection Threshold PPS Infinite Detection Threshold Percent Infinite Rate/Leak Limit Infinite
Now In the Flood row, click the + icon, and then click ICMPv4 flood.
On the right-side of the page configure using the following information.
Detection Threshold PPS Infinite Detection Threshold Percent Infinite Rate/Leak Limit Infinite
Apply the settings above for TCP SYN flood and UDP Flood.
In the Behavioral row click on Learn Only, then click Update.
On the goodclient, start the network baselining (Let it running for the entire lab)
sudo ~/tools_agility_183/baseline_l4.sh
Important
In order to assure best performance and good lab results, always use the management network ip addresses/hostnames for remote access (goodclient-mgmt, attacker-mgmt and lamp-mgmt)
f5student@goodclient:~$ cd ~/tools_agility_183/ f5student@goodclient:~/tools_agility_183$ ./baseline_l4.sh / status: 200 bytes: 3952 time: 0.016 / status: 200 bytes: 3952 time: 0.019 / status: 200 bytes: 3952 time: 0.014 / status: 200 bytes: 3952 time: 0.014 / status: 200 bytes: 3952 time: 0.018 / status: 200 bytes: 3952 time: 0.221 /httprequest.php status: 200 bytes: 699 time: 0.014 /httprequest.php status: 200 bytes: 699 time: 0.014
Launch an ICMP flood Attack on the LAMP Server¶
Hint
The pentest tool can be used to send several types of DoS Attacks for the most part of the lab, few free to try it out. For some specific exercises there will be custom shell scrtips though.
sudo ~/tools_agility_183/pentest
________ _______ _ _________ _______ ________ _ | ____ || ____ \| \ /|\__ __/| || ____ \| \ /||\ /| | | | || | \/| \ | | | | | || || || | \/| \ | || | | | | |____| || |__ | \ | | | | | || || || |__ | \ | || | | | | ______|| __) | |\ \| | | | | ||_|| || __) | |\ \| || | | | | | | | | | \ | | | | | | || | | | \ || | | | | | | |____/\| | \ | | | | | | || |____/\| | \ || |___| | |/ (_______/|/ \_| |_| |/ \||_______/|/ \_||_______| Welcome to pentmenu! Please report all bugs, improvements and suggestions to https://github.com/GinjaChris/pentmenu/issues This software is only for responsible, authorised use. YOU are responsible for your own actions! Please review the readme at https://raw.githubusercontent.com/GinjaChris/pentmenu/master/README.md before proceeding 1) Recon 2) DOS 3) Extraction 4) View Readme 5) Quit Pentmenu>
Hit option 2 (DOS), then 1 (ICMP Echo Flood)
Use Attack options as follows:
Enter target IP/hostname: server1 Enter Source IP: r (random) Now open two more terminal sessions with attacker and lamp servers respectively. On each screen open the bmon util for instant traffic stats.
eth1 Interfaces │ RX bps pps %│ TX bps pps % lo │ 0 0 │ 0 0 eth0 │ 66B 1 │ 545B 1 qdisc none (pfifo_fast) │ 0 0 │ 525B 1 ->eth1 │ 77B 1 │ 1.59MiB 39.63K qdisc none (pfifo_fast) │ 0 0 │ 1.59MiB 39.63K ───────────────────────────────┴───────────────────────┴──────────────────────────────────────────────────────────── (RX Packtes/second) 5.00 ....|..|.........|......|........................|.......... 4.17 ...|||||||...|...|.||.|||...........|||.......|..||.|.|...|| 3.33 ...||||||||..|..||||||||||..||.....|||||.....|||||||||||.||| 2.50 ...||||||||..|..||||||||||..||.....|||||.....|||||||||||.||| 1.67 .|||||||||||||.|||||||||||||||....|||||||...|||||||||||||||| 0.83 ||||||||||||||||||||||||||||||||.|||||||||..|||||||||||||||| 1 5 10 15 20 25 30 35 40 45 50 55 60 K (TX Packtes/second) 52.32 ..............||||....|.||.................................. 43.60 ||.|||||||||||||||||||||||||||||||||..||||||||.||||||||||||| 34.88 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 26.16 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 17.44 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 8.72 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 1 5 10 15 20 25 30 35 40 45 50 55 60Hint
Use either the RIGHT and LEFT arrow keys to move between Bps and pps metrics. Don’t forget selecting the right inteface using the UP/DOWN arrow keys. Attacker uses eth1 and Lamp uses eth4 for data traffic.
Open a terminal session with the DHD and use the tcpdump util to verify that ICMP attack traffic is passing through the device.
[root@dhd:Active:Standalone] config # tcpdump -i defaultVLAN
Observe the baseline running on goodclient. Since the flood attack is hitting the server hard, the legitimate client sessions are being degraded. Look at the statude code 000 for most requests.
In the Configuration Utility, open the Statistics-> Performance-> Performance page. As you can see, there is a drastic spike in the traffic.
Open the Security-> DoS Protection-> DoS Overview page.
In the Filter Type field select Device DoS. Then on the left corner search for ICMP.
Review the statistics for Current, 1 min. Average, and 1 hr Average.
- Open the Security-> Event Logs-> DoS-> Network-> Events page.
The log file is empty as we disabled device-level flood protection on BIG-IP DHD.
From the attacker terminal session type Ctrl + C to stop the ICMP flood.
Lab - Configure Hybrid Defender Flood Protections¶
This lab teachs you on how to configure DoS protection for common network-level DoS vectors.
Configure Protected Object-Level IPv4 Flood DHD DoS Protection¶
Configure object-level IPv4 ICMP flood protection, and then issue an ICMP DoS flood and review the results.
On the Protect Objects page, in the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create:
Name: ServerNet IP Address: 10.1.20.0/24 Port: any Protocol All Protocols Protec. Settings Action: Advertise All Protec. Settings DDoS: IPv4 In the IPv4 row click the + icon, and then click ICMPv4 flood
On the right-side of the page configure using the following information, and then click Create.
Detection Threshold PPS: Specify: 1000 Detection Threshold Percent: Infinite Rate/Leak Limit: Specify: 1000 Important
From now on, make sure you have an always-on terminal session with both the attacker and LAMP servers. Let them running the bmon utility, or a tcpdump. Those will provide instant and detailed visibility of the ammount of packets comming in/out of both virtual machines.
From the attacker terminal session launch an ICMPv4 DoS attack using the Pentmenu tool (Options 2, 1) as follows:
target IP/hostname: server1 source IP: r[random] Check out the LAMP terminal session and observe how many ICMP packets are hitting this server.
Before moving on, wait the attack to run for about 30 seconds or so
In the Configuration Utility go to Security-> DoS Protection-> DoS Overview. You should be able to see the DHD stopping the Attack.
Now stop the Attack with Ctrl + C.
Open the Security-> Event Logs-> DoS-> Network-> Events page.
The DoS Source is Volumetric, Aggregated across all SrcIP’s, VS-Specific attack, metric:PPS.
- The virtual server column displays /Common/ServerNet, identifying this is a protected object.
- The type is ICMPv4 flood.
- The action is Drop.
Now check out the Security-> Event Logs-> DoS-> Network-> Events Page.
The DHD was able to detect the moment the attack started and stopped, along with all volumetric info.
Configure Protected Object-Level UDP Flood Attack Protection¶
Configure object-level DoS UDP flood protection, and then issue an UPD flood and review the results.
From the attacker terminal session launch an UDP flood attack using the Pentmenu tool (Options 2, 7) as follows:
target IP/hostname: server2 target port (defaults to 80): default [ENTER] random string (data to send): F5Agility2018 source IP: r[random] Let the attack run for about 30 seconds before moving on.
In the Configuration Utility, open the Statistics-> Performance-> Performance page. There is a spike in connections and throughput. The BIG-IP system is being hit with the UDP flood attack.
Open the DoS Protection-> Quick Configuration page and in the Protected Objects section click ServerNet.
In the DDoS Settings row click the UDP checkbox. In the UDP row click the + icon, and then click UDP Flood.
On the right-side of the page configure using the following information, and then click Update.
Detection Threshold PPS: Specify: 1000 Detection Threshold Percent: Infinite Rate/Leak Limit: Specify: 3000 From the Attacker terminal session launch a new UDP flood attack using the same options and values as previously in this task.
Let the attack run for about 30 seconds before moving on.
In the Configuration Utility, click Security-> DoS Protection-> DoS Overview. You should be able to see the DHD stopping the DNS Attack.
Now stop the Attack with Ctrl + C.
Open the Security-> Event Logs-> DoS-> Network-> Events page.
- In one minute or so, the virtual server column displays /Common/ServerNet, identifying this is protected object.
- The type is UDP flood.
- The action is Drop.
Configure Bad Actor Detection¶
Add bad actor detection for the UDP flood protection
In the Configuration Utility, open the DoS Protection-> Quick Configuration page and in the Protected Objects section click ServerNet.
In the UDP row click the + icon, and then click UDP Flood.
On the right-side of the page configure using the following information, and then click Update.
Bad Actor Detection: Yes (selected) Per Source IP Detection (PPS): Specify: 100 Per Source IP Rate Limit (PPS): Specify: 30 Blacklist Attacking Address: Yes (selected) Detection Time: 30 Duration: 60 From the attacker virtual machine launch an UDP flood attack using a single IP address [Pentmenu tool - Options 2, 7]:
target IP/hostname: server4 target port (defaults to 80): 53 random string (data to send): F5Agility2018 source IP: i[interface] Let the attack run for like 30s seconds before moving on.
Stop the attack with Ctrl + C.
Now try to ping the server4. Try to ping the same address from the goodclient virtual machine. Does it work ???
Stop the Attack with Ctrl + C and move to the next exercise.
Configure Protected Object-Based Sweep Protection¶
In the Configuration Utility, open the DoS Protection-> Quick Configuration page and in the Protected Objects section click ServerNet.
In the DDoS Settings row click the Sweep checkbox.
In the Sweep row click the + icon, and then click Sweep.
On the right-side of the page configure using the following information, and then click Update.
Detection Threshold PPS: Specify: 1000 Rate/Leak Limit: Specify: 3000 Packet Types: Move All IPv4 to the Selected field On the attacker machine type (or copy and paste) the following command:
sudo ./sweep.sh
Let the attack run for like 30s seconds before moving on.
Stop the attack with Ctrl + C.
In the Configuration Utility, click Security-> DoS Protection-> DoS Overview. You should be able to see the DHD stopping the Sweep attack.
Check out the DoS Visibility Page¶
Use the DoS Visibility page to view statistics about the DoS attacks you submitted during this exercise.
Mouse over several of the attacks to get additional details of each attack.
Scroll down in the left-side of the page to view the Attacks section.
You can see the number of high, moderate, and low attacks in addition to the types of attacks (HTTP, ICMP, etc.) and the severity levels.
Check out the Silverline Portal¶
Use the Silverline portal to view details about the attacks launched in this exercise.
Access the Silverline Portal https://portal.f5silverline.com
Open the Audit-> API Activity log page.
Enter the hostname of your DHD device in the Search field and then check out the activity your Hybrid Defender device has reported back to the Silverline Scrubing Center.
Lab - Preventing DNS DoS Attacks¶
Use a protected object to mitigate DNS query floods.
Use a Protected Object to Mitigate a DNS Query Flood¶
In the Protected Objects section click Create.
Configure a protected object using the following information, and then click Create.
Name: DNS_Server IP Address: 10.1.20.14/32 Port: 53 Protocol UDP Protec. Settings Action: Log and Mitigate Protec. Settings DDoS: DNS In the DNS row click the + icon, and then click DNS A Query.
On the right-side of the page configure using the following information, and then click Create.
Detection Threshold PPS: Specify: 75 Rate Limit Specify: 100
Establish a DNS Baseline¶
Use a script to establish a DNS baseline on the BIG-IP DHD.
From the goodclient terminal session run the following commands:
sudo ~/tools_agility_183/dnsbaseline.sh
Let the baseline run until you get the following results:
[Status] Testing complete (time limit) Statistics: Queries sent: 6000 Queries completed: 6000 (100.00%) Queries lost: 0 (0.00%) Response codes: NXDOMAIN 6000 (100.00%) Average packet size: request 41, response 116 Run time (s): 120.000552 Queries per second: 49.999770 Average Latency (s): 0.005793 (min 0.003970, max 0.020681) Latency StdDev (s): 0.001383In the Configuration Utility, go to Security-> DoS Protection-> DoS Overview.
In the Filter Type select Virtual Server with DNS_Server protected object, then examine the a statistics for DNS A Query.
Initiate a DNS Attack¶
Run a script to generate a DNS DoS alert. This script will send 80 pps of “A” queries just above our detection threshold PPS setting of 75. This is just the threshold that we are alerting at. It has not reached a high enough threshold to determine that we should do something about it.
From the attacker terminal session run the following commands:
sudo ~/tools_agility_183/dnsdosattack.sh
Wait for the attack to run for about 30 seconds before moving on.
In the Configuration Utility, open the Security-> DoS Protection-> DoS Overview page.
In the Filter Type select DoS Attack.
Note
The A query DOS attack vector will be detected, but not yet blocked. It will take up to a couple minutes to display as Detected.
Wait for the attack to complete (if not done yet). Verify the results of the DNS attack from the attacker terminal session:
[Status] Testing complete (time limit) Statistics: Queries sent: 28800 Queries completed: 27217 (94.50%) Queries lost: 1583 (5.50%) Response codes: NXDOMAIN 27217 (100.00%) Average packet size: request 41, response 116 Run time (s): 360.000538 Queries per second: 75.602665 Average Latency (s): 0.004487 (min 0.002909, max 0.036921) Latency StdDev (s): 0.001372
Initiate a DNS Attack that Exceeds the Rate Limit¶
Run another script that initiates a DNS DoS attack that exceeds the rate limit we set earlier.
From the attacker terminal session run the following commands:
sudo ~/tools_agility_183/dnsdosrate.sh
Wait for the attack to run for about 30 seconds before moving on.
In the Configuration Utility Review the DoS Overview page -> Security-> DoS Protection-> DoS Overview.
Note
The A query DOS attack vector is now dropping attack traffic.
Also take a look at the script which will record the number of drops if any as a result of the attack rate limit being hit. You should be able to correlate the drops registered with the script with the drops recorded by the Hybrid Defender.
Statistics: Queries sent: 5899 Queries completed: 3504 (59.40%) Queries lost: 2395 (40.60%) Response codes: NXDOMAIN 3504 (100.00%) Average packet size: request 41, response 116 Run time (s): 120.000642 Queries per second: 29.199844 Average Latency (s): 0.006696 (min 0.002080, max 0.087619) Latency StdDev (s): 0.003606In the Configuration Utility open the Statistics-> DoS Visibility page.
View the attack details in the Attacks section.
Module - Application Layer DoS Protection¶
In this module you will learn how the F5 Hybrid Defender can effectively protect from DoS Attacks at the Application Level.
Lab – Configure Application Layer DoS Defenses¶
Check out how to detect and mitigate application layer attacks, not matter if it’s encrypted or behavioral based.
Create Protected Object for Behavioral DoS Protection¶
In the BIG-IP Configuration Utility, open the DoS Protection-> Quick Configuration page and in the Protected Objects section click Create.
Configure the protected object Server1-http using the following information:
Name: Server1-http IP Address: 10.1.20.11/32 Port: 80 VLAN: defaultVLAN Protec. Settings Action: Log and Mitigate Protec. Settings Silverline: Yes (selected) Protec. Settings DDoS: IPv4, TCP, HTTP In the HTTP row click the + icon, and then click Behavioral, then from the Mitigation list select Standard Protection.
In the HTTP section click Proactive Bot Defense, then from the Mitigate Action list select Disabled, finally click Create
Note
Both the good and bad (attack) traffic are generated with tools that would be blocked by Proactive Bot Defense. Please note that by default, the Hybrid Defender will set Proactive Bot Defense to always’. That’s the reason why we’re disabling it, only to allow the scripts to work and generate sample traffic.
In the Protected Objects section click Create.
Open the Security-> DoS Protection-> DoS Profiles page and click Server1-http.
Open the Application Security page.
Click Behavioral & Stress-based Detection, and then for Behavioral Detection and Mitigation click Edit.
Select the Request signatures detection checkbox, and then click Update.
Generate L7 Behavioral baseline for Server1-http¶
Use a script to generate an L7 behavioral DoS baseline for the Hybrid Defender.
In the goodclient terminal session, type (or copy and paste) the following command:
sudo ~/tools_agility_183/generate_clean_traffic.sh
Note
This will generate traffic. Please note that it will take at least 15 minutes.
f5student@goodclient:~/tools_agility_183$ ./generate_clean_traffic.sh welcome.php status: 200 bytes: 1045 time: 0.017 welcome.php status: 200 bytes: 1045 time: 0.014 welcome.php status: 200 bytes: 1045 time: 0.014 welcome.php status: 200 bytes: 1045 time: 0.015 headers.php status: 200 bytes: 1847 time: 0.014 headers.php status: 200 bytes: 1847 time: 0.014 httprequest.php status: 200 bytes: 710 time: 0.013 httprequest.php status: 200 bytes: 710 time: 0.014 httprequest.php status: 200 bytes: 710 time: 0.014 httprequest.php status: 200 bytes: 710 time: 0.013 badlinks.html status: 200 bytes: 1270 time: 0.014 badlinks.html status: 200 bytes: 1270 time: 0.014 F5_building.jpg status: 200 bytes: 33447 time: 0.019 F5_building.jpg status: 200 bytes: 33447 time: 0.021 bigip4200.jpg status: 200 bytes: 9753 time: 0.016 bigip4200.jpg status: 200 bytes: 9753 time: 0.017 viprion2400.jpg status: 200 bytes: 13009 time: 0.016 viprion4800.jpg status: 200 bytes: 10078 time: 0.018 viprion4800.jpg status: 200 bytes: 10078 time: 0.017
Move on in the exercises while the baseline is being generated.
Open a terminal session with the DHD and run the following command:
admd -s vs./Common/Server1-http.info -s vs./Common/Server1-http.sig.health
[root@dhd-01:Active:Standalone]# admd -s vs./Common/Server1-http.info -s vs./Common/Server1-http.sig.health vs./Common/Server2-http.sig.health:[0.452373] vs./Common/Server2-http.sig.health:[0.453407] vs./Common/Server2-http.sig.health:[0.451726] vs./Common/Server2-http.sig.health:[0.45372] vs./Common/Server2-http.sig.health:[0.452021] vs./Common/Server2-http.sig.health:[0.45349]Important
The results for each health check should not be 0.5, otherwise the system ins’t learning. Let both terminal sessions opened for the rest of this lab.
Configure DoS Protection for L7 Encrypted Traffic¶
Launch an encrypted Slowloris attack to the web server and view the results, then configure proper mitigation on the Hybrid Defender.
Go to DoS Protection-> Quick Configuration page and in the Protected Objects section click Create.
Configure another Protected Object using the following information, and then click Create.
Name: Server2-http IP Address: 10.1.20.12/32 Port: 80 VLAN: defaultVLAN Protec. Settings Action: Log and Mitigate Protec. Settings Silverline: Yes (selected) Protec. Settings DDoS: IPv4, TCP, HTTP Now repeat the steps for disabling the Proactive Bot Defense which allows the HTTP request scripts to work.
Go to the HTTP section and click Proactive Bot Defense, then from the Mitigate Action list select Disabled.
In the HTTP section click DoS Tool, then from the Mitigate Action list select Report, and then click Create.
Now run the monitor script on server2 as follows. It will be usefull for server health monitoring.
~/tools_agility_183/server2_monitor.sh
Before launching the application layer attack, observe server2 is currently healthy.
welcome.php status: 200 bytes: 1045 time: 0.018 bigtext.html status: 200 bytes: 634965 time: 0.136 httprequest.php status: 200 bytes: 710 time: 0.017Note
The system is healthy since the web server returns HTTP Status Code 200 for every request.
Now from the attacker terminal session run the following command:
~/tools_agility_183/slowloris.sh
Mon Aug 13 11:26:54 2018: slowhttptest version 1.6 - https://code.google.com/p/slowhttptest/ - test type: SLOW HEADERS number of connections: 4090 URL: https://server2.f5demo.com/ verb: GET Content-Length header value: 4096 follow up data max size: 68 interval between follow up data: 10 seconds connections per seconds: 200 probe connection timeout: 5 seconds test duration: 240 seconds using proxy: no proxy Mon Aug 13 11:26:54 2018: slow HTTP test status on 30th second: initializing: 0 pending: 1790 connected: 150 error: 0 closed: 2092 service available: **NO**Observe how the service is impacted as the slowloris attack hits the server2.f5demo.com.
welcome.php status: 000 bytes: 0 time: 1.002 bigtext.html status: 000 bytes: 0 time: 1.002 httprequest.php status: 000 bytes: 0 time: 1.002Note
Since the slowloris attack is being encrypted (https://server2.f5demo.com) we need to setup the certificate and private keys so the traffic can be inspected by the Hybrid Defender..
Configure SSL on the protected object to in order to inspect HTTPS traffic.
Go to DDoS Protection-> Quick Configuration-> Protected Objects, then click Server2-http. Configure the SSL as follows:
Port: 443 SSL: Enabled SSL Certificate: default Key: default Encrypt Connection to Server: Yes (selected) Disable bot protections so the scripts can be used for testing the server health.
On the Server2-http Protected Object section go to the HTTP row, click the + icon, click Behavioral
Now from the Mitigation list select Standard Protection.
In the HTTPS section click Proactive Bot Defense, then from the Mitigate Action list select Disabled.
Now that SSL is also being inspected for this Protected Object, let’s run the slowloris script once again and verify if the attack still works.
– Behavioral L7 DoS Mitigation¶
Once the L7 behavioral baseline has been established, launch an L7 DoS attack and view the results.
Now get back to the DHD terminal session.
You will need to observe the info.learning signature to ensure that the system has accumulated enough learning details.
This signature has 4 comma-separated values for monitoring the learning progress:
- Value #1: baseline-learning_confidence
This should be between 80 - 90%
- Value #2: learned_bins_count (the number of learned bins)
This should be > 0
- Value #3: good_table_size (the number of learned requests)
This should be > 4000
- Value #4: good_table_confidence (how confident, as a percentage, the system is)
It must be 100% for behavioral signatures
vs./Common/Server1-http.info.learning:[96.3163, 78, 5355, 100]If you see the pattern such as that described, it indicates the traffic baseline was already established, then you can move forward with the lab.
Once the info.learning values are acceptable based on the details above, from the attacker terminal session run the following command:
~/tools_agility_183/http_flood.sh
Select option “1”
Now take a look at the goodclient terminal session, you should start seeing the effects of the HTTP DoS attack, as requests are starting to fail (HTTP Status Code 000). If you were to examine the Lamp server at this time, you would see that it is under severe stress.
welcome.php status: 200 bytes: 1045 time: 0.017 welcome.php status: 200 bytes: 1045 time: 0.029 welcome.php status: 000 bytes: 0 time: 1.000 headers.php status: 000 bytes: 0 time: 1.001 headers.php status: 200 bytes: 1847 time: 0.204 headers.php status: 200 bytes: 1847 time: 0.258 headers.php status: 200 bytes: 1847 time: 0.218 badlinks.html status: 000 bytes: 0 time: 1.001 badlinks.html status: 200 bytes: 1270 time: 0.242 badlinks.html status: 200 bytes: 1270 time: 0.272 badlinks.html status: 000 bytes: 0 time: 1.002 bigip4200.jpg status: 200 bytes: 9318 time: 0.247Also from the DHD terminal session watch the health signal feed. You should see it climb from ~.5, which is optimal health, to values over 1, indicating an increase in server stress. You will also be able to watch as the system responds and mitigations are engaged.
When the system has analyzed the attack traffic, dynamic signatures are created and engaged:
vs./Common/Server1-http.sig.health:[0.768427] vs./Common/Server1-http.info.attack:[1, 1] vs./Common/Server1-http.sig.health:[0.746648] vs./Common/Server1-http.info.signature:["Stable signature detected: (http.f5_filename_bin == 21) and (http.request.method eq \"GET\") and (!(http.user_agent matches \"(MSIE|Chrome|Firefox|Opera|Safari|Maxthon|Seamonkey)\")) and (!http.content_type) and ((http.hdr_len->= 128) and (http.hdr_len < 256)) and (http.request.uri matches \"^[^\\\\?]*$\") and (http.f5_headers_count == 5) and (http.f5_cache_control_bin == 0) and (http.accept) and (http.request.line matches \"Accept-Charset:.*\") and (http.f5_host_bin == 4) and (http.f5_referer_bin == 0) and (http.f5_uri_len_bin == 0) and (!(http.accept matches \"(application|audio|message|text|image|multipart)\")) and (http.connection) and (http.host) and (!(http.request.line matches \"Accept-Charset\")) and (http.user_agent)"] vs./Common/Server1-http.info.attack:[1, 1] vs./Common/Server1-http.sig.health:[0.726608] vs./Common/Server1-http.info.attack:[1, 1] vs./Common/Server1-http.sig.health:[0.709827] vs./Common/Server1-http.info.attack:[1, 1] vs./Common/Server1-http.sig.health:[0.691779]In the Configuration Utility, notice the indicator at the top-left side of the page.
As you watch the feed, you should see HTTP requests being served again after the dynamic signature kicks in.
In the Configuration Utility open the Security-> DoS Protection-> Behavioral Signatures page.
You will see a signature that was created (as seen in the output of the
admd
command earlier). Note the system reports metrics such as Accuracy (an estimate of the percentage of traffic that will be blocked that is definitely hostile) and Efficiency (a measure of how much of the observed DoS traffic is mitigated by that signature). In our lab these values are both at or near 100%. In a real environment the Accuracy should be very high, but sometimes Efficiency will be lower (in a mutating attack) and the system may have to create additional signatures or refine the current one based on effectiveness.Click the new signature.
Note the Wireshark filter at the bottom which can be used in conjunction with the Record Traffic feature of F5’s L7 DoS to identify exactly which requests the signature matches/will match. This can be helpful if using the “Approved Only” in the DoS profile setting to allow a risk-averse administrator to approve signatures before they begin to filter traffic.
Change the Alias value to Agility2018, and then click Finished.
– View Silverline Signals¶
Use the Silverline portal to view details about the L7 DoS attacks that were launched in this exercise.
Click Alerts for Hybrid Defender.
Open the Audit-> API Activity Log page.
Click the + icon to expand one of the entries to view additional attack details.
That completes the hands-on exercise for BIG-IP DDoS Hybrid Defender.
F5 Agility 2018: DDoS Attack Protection¶
F5® DDoS Hybrid Defender™, a hybrid DDoS solution that offers comprehensive protection, high availability, and is easy to deploy and manage. It guards against aggressive volumetric and targeted DDoS attacks, includes hardware-assisted DDoS mitigation, and optionally, connects with Silverline, a cloud-based scrubbing service.
This class covers the following topics:
- Initial Set-up, Device Configuration and working with basic device-level DDoS vectors to mitigate the most commonly encountered attacks. Then we will cover Auto-thresholding, and Mitigation of L7 Behavioral Attacks time permitting.
Getting Started¶
Please follow the instructions provided by the instructor to start your lab and access your jump host.
Note
All work for this lab will be performed exclusively from the Windows jumphost. No installation or interaction with your local system is required. You will use Putty that has been preconfigured with appropriate keys in order to access the DHD CLI, Good Client, and the Attacker systems. The shortcuts are on the desktop. You will log in as “root” or “ubuntu”.
Lab Topology¶
The following components have been included in your lab environment:
- 1 x F5 BIG-IP VE (v14.0) Provisioned as DHD
- 1 x Linux Attacker (Ubuntu 14.04)
- 1 x Linux Good Client (Ubuntu 14.04)
- 1 x Linux LAMP Webserver (xubuntu 14.04)
- 1 x Windows Jumphost
Lab Components¶
System | Username | Password |
---|---|---|
Ravello | Given at site | Given at site |
Win7 Jumpbox | external_user | f5DEMOs4u |
Hybrid Defender - WebUI | admin | f5DEMOs4u |
Hybrid Defender - CLI | root | f5DEMOs4u |
Good Client | ubuntu | Use key |
Attacker | ubuntu | Use key |
Lamp CLI | root | default |
Lamp X-Server Shell | xubuntu | <no password> |
Accessing the Lab Environment¶
Task 1 – Open your RDP client and connect to your Windows Jumpbox¶
- A URL will be provided by your Instructor at the training site that will access the training portal.
- In the training portal you will enter the given class number and student number.
- Login
- Click the Jumpbox RDP link.
This will RDP to the Jumpbox where you will work all the labs from.
Note
Use the show options to provide details.
- Login to the Jumpbox
- User name: Jumpbox external_user. Password: f5DEMOs4u
- Click YES at the warning
Note
We need to ensure the Jumpbox and the DDoS Hybrid Defender are in time sync. Please run the following commands from an Elevated Command Prompt. (Administrator)
- net start w32time
- w32tm /config /update /manualpeerlist:10.1.1.245
- net stop w32time && net start w32time
DDoS Hybrid Defender Setup¶
In this module you will learn how to complete the setup of F5 Networks DDoS Hybrid Defender and the initial configuration related to Device Protection.
Lab 1 – DDoS Hybrid Defender Setup¶
Estimated completion time: 20 minutes
Task 1 – Initial Set-up¶
- Open the Chrome web browser and access the DHD from the toolbar shortcut.
- Login to the BIG-IP Configuration Utility using the ”admin” account.
Note
When you first power up a F5 DHD device you would normally go through the steps of licensing, provisioning and basic set-up. We have licensed, assigned the management IP, hostname, NTP and DNS servers for you. Verify DHD and Jumpbox are showing same time.
Note
If you are familiar with the BIG-IP UI, You will notice the menus on the left are consolidated. This is an indication you are working with a DDoS Hybrid defender device.
Expand each panel section to see the components available in each section.
- Dos Configuration: Where most day-to-day configuration takes place.
- Dos Setup: Where one-time or infrequent system Dos configuration is performed.
- Network: The new simplified Security Network Configuration utility to add new network topologies to the system.
- Visibility: Were the Analyst will spend a majority of the time looking at the GUI and logs.
- System: Shows a subset of the system utilities found in the traditional TMUI System menu. (Available in Advanced View on the DHD)
- If you need to access more options, there is a shortcut at the bottom of the Menu page. Show Advanced Menu
- Explore the Resource Provisioning page
Note
The above task ensures that you are using a purpose built DDoS Hybrid Defender. If you are familiar with other F5 Modules/Technology that you have used in the past, you will notice that we have none of those provisioned.
- When done click Submit.
Task 2 – DDoS Hybrid Defender Base Configuration¶
The architecture and design decisions should have been made already. Based on F5 recommendations we are going to deploy this device in L2 Transparent Mode.
- Click Network in the left hand menu. Then Select Topology.
- Click Create on the upper right side.
- You will notice the various options you can select based on the prior architecture decisions.
- For this classes purpose Click on the VLAN Group image.
- Fill out the information from the table below. Then Click Done Editing within that section.
VLAN Group Name: | defaultVlan |
Internal: VLAN Tag | 20 |
Internal: Interfaces | 1.2 Untagged (Click Add) |
External: VLAN Tag | 10 |
External: Interfaces | 1.1 Untagged (Click Add) |
- At the bottom of the page click Finished to create the default network.
This completes the initial Network Set-Up of DHD.
Lab 2 – Configuring Hybrid Defender DDoS Device Protection¶
Task 1 – Verify Communication Through the DHD Device.¶
PuTTY to the BIG-IP CLI (10.1.1.245) from your jumpbox desktop shortcut and resize window by making it wider. You will be logged on as
root
.At the config prompt, type (or copy and paste) the following command:
tcpdump -i 0.0 host 10.1.20.12
PuTTY to the Attacker host from your jumpbox desktop shortcut. Accept the Warning. Enter “ubuntu” as the user. It will use a pre-loaded public key as the credentials.
At the config prompt, type (or copy and paste) the following command:
ping 10.1.20.12
Examine the tcpdump window and verify ICMP packets are flowing through the BIG-IP DHD.
The attacker can successfully communicate with a back-end resource behind the BIG-IP DHD.
Note
The listener for the ICMP packets is the VLAN group.
- Cancel the
ping
command, then verify thetcpdump
stops receiving ICMP packets, and then press Enter several times to clear the recent log entries.
Task 2 – Disable Device-Level DHD DoS Protection¶
- In the Configuration Utility, in the DoS Configuration >> Device Protection section click Network.
- On the left side of the page select the checkbox for ICMPv4 flood and UDP Flood.
- At the bottom just below the last vector, choose the drop down Set State and then select Disabled.
Hint
This is the new method for selecting and changing multiple items at one-time. This will be how we will Set State and Set Threshold.
- Navigate back to the top of the window and Select Commit Changes to System
- On the Jumpbox in the Attacker PuTTY window type (or copy and paste) the following:
# sudo su
# cd scripts
# ls
Note
Ignore the “sudo: unable to resolve host” error.
These are some of the different scripts we’ll be using during the exercises to simulate DoS attacks.
Type (or copy and paste) the following command:
for i in {1..10}; do ./icmpflood.sh; done
This script launches the Attack and then repeats for a total of ten occurrences.
- View the
tcpdump
window and verify that ICMP attack traffic is reaching the back-end server. - Let the attack run for about 15 seconds before moving on.
- In the Configuration Utility, open the DoS Configuration >> DoS Overview (non HTTP) page.
- Make sure the Filter Type is “Dos Attack”.
- View the Protection Profile column in the display and notice no results are returned, you disabled those vectors.
- Navigate to Visibility >> Event Logs >> DoS >> Network >> Events.
- Go back to the Attacker and stop the script. CTRL+C (This needs to be hit several times to break out of the script)
- Notice no logs are captured. We could have chosen Learn Only or Detect Only and had different results. If you want to test, feel free.
Note
If you want to run the other attacks, use the format above. ./synflood.sh and udp_flood.sh behave similar. If you are not seeing the traffic on the DHD CLI, Stop and Re-Start the tcpdump.
Both of these locations we will return to throughout this course to see how our DHD is viewing these attacks.
Task 3 – Re-enable Device-Level DHD DoS Protection¶
In this task you will re-configure device-level DoS protection and then issue the same command and review the results.
- In the Configuration Utility, in the DoS Configuration >> Device Protection under Log Publisher select “local-db-publisher”.
- Next click the Network section.
- On the left side of the page select the checkbox for ICMPv4 flood and UDP Flood.
- At the bottom just below the last vector, chose the drop down Set State and then select Mitigate.
Note
You have the option of Learn Only and Detect Only as well.
- Navigate back to the top of the window and Select Commit Changes to System
Note
This returns the configuration back to factory supplied device level enforcement.
Task 4 – Attack the DDoS Hybrid Defender again and see what you can tell.¶
Type (or copy and paste) the following command:
for i in {1..10}; do ./icmpflood.sh; done
In the Configuration Utility, open the DoS Configuration >> DoS Overview (non HTTP) page.
Make sure the Filter Type is “Device Dos”.
This page will show the preset vectors for the Device and the Current Attack Status, Average EPS, Current Dropped EPS and the Detection Thresholds including the Threshold Mode.
Scroll down until you see ICMPv4 Flood.
Attention
Why is the DHD not dropping packets?
Hint
Look at the Manual Thresholds set and the current rate of packets. We are not generating enough traffic.
- We need to set a lower threshold Manually.
- In the Configuration Utility, open the DoS Configuration >> Device Protection page. Scroll down in the Network section to ICMPv4 flood. Click ICMPv4 flood.
Note
The new fly out page.
- Manually Set The Detection Threshold PPS to 100 and the Mitigation Threshold EPS to 500. Scroll up and Commit Changes to System
- Relaunch the Attack from the Attacker CLI.
- In the Configuration Utility, open the DoS Configuration >> DoS Overview (non HTTP) page.
- Make sure the Filter Type is “Dos Attack”. See the Dropped traffic with the new thresholds. Alternatively, you can go “Device DoS”, scroll down to ICMPv4 Flood and see the same information.”
- Look at the Protection Profile: dos-device, attack status and various rates.
- You can terminate the Attack with Ctrl+C when finished.
This concludes this section where we looked at setting manual thresholds to mitigate attacks that might not have been mitigated with the default settings.
Note
We did this to only one vector. These same procedure can be applied to all the vectors or selected vectors, depending on your environment.
DDoS Hybrid Defender Attacks and Mitigations¶
In this module you will create Protected Objects, Set Mitigation Thresholds Manually, and then launch various attacks against the F5 Networks DDoS Hybrid Defender and view the results in the GUI and logs. Then you will allow the DDoS Hybrid Defender to Automatically detect and set Threshold for detection and mitigation, easing the burden on Administartors. Finally, time permitting, we will explore Behavioral mitigations. (Covered in Detail in the Advanced class)
Lab 1 – Quick GUI Overview of the Visibility and Reporting Available¶
Task 1 – View the New Visibility Page¶
You can now use the new DHD Visibility page to view the Dashboard, Analysis, Event Logs and Debugging info.
- Take advantage of the expandable window feature to give more screen space to the GUI.
- In the Hybrid Defender Web UI, go to the Visibility >> Dashboard overview.
Note
DoS Visibility Dashboard defaults to not Auto-Refresh. Click the Button to set Real-Time to ON.
- You should see categories as: Attack Duration, Attacks, Virtual Severs, System Health and Countries.
Scroll through the Left Pane and explore the windows.
- You can use the slider to shorten the time frame, or filter on the protocol, if desired when viewing attacks if needed.
- Later when we have data and attacks, you will see the different attacks in the Attack Duration window. You will be able to hover over for more details.
- Scroll down in the left-side of the page to view the Attacks section.
- View the details at the bottom of the Attacks section.
This table displays details of each attack that has occurred.
- Examples are; Attack ID, Severity, Vector, Trigger Virtual Server, Start Time, Stop Time…etc
- Scroll down in the left-side of the page to view the Virtual Servers section.
- You can see the details of protected object-level attacks.
- Examples are; Virtual Server, Server Latency, Health, Current Connections, Blocked IP’s…etc
- Scroll down to the System Health section. This table displays the current health of the system.
- Scroll down to the Countries section. This table displays the attack details from each country.
Now focus on the Right Panel.
- View the various widgets in the panel on the right-side of the page. The top can be expanded and contracted visa the slider bar.
- Click Network to filter out only the network-level attacks (all the attacks so far have been network-level).
- If it’s not already expanded, expand the Virtual Servers widget, and then select /Common/Server.
- This filters the results to only attacks at this protected object-level. Notice the changes to the map on in the Countries section.
- Continue to Explore and Scroll down the right side. Notice each widget supplies greater detail.
Lab 2 - Multi-vector Attack Demo¶
In this simple demo you will launch a small number of network attacks and show the configuration, logging and reporting capabilities of the F5® DDoS Hybrid Defender™. The point of this demo is to provide context for a UI walk-through with more live data and viewing and setting manual thresholds.
Task 1 – Create a Protected Object that the Attacker will be targeting¶
The DHD device wide protection is enforced for all traffic flowing through the device. For more granular control, we use Protected Objects and configure mitigation settings for those objects to be enforced.
In this task you will configure Object-Level DoS protection for a network (L4), simulating your Server Network and then issue an attack and review the results.
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protected Objects page and in the Protected Objects section click the Create dropdown and select Protected Object.
- Configure the Protected Object using the following information, and then click Create.
Name | ServerNet |
Destination Address | 10.1.20.0/24 |
Port | *All Ports |
Protocol | All Protocols |
Protection Profile: | dos |
Eviction Policy: | Leave Blank |
VLAN(s): | defaultVLAN |
Logging Profiles: | local-dos |
- Click Save
This protected object is defending all ports/protocols for 10.1.20.0/24, which is the network behind the Hybrid Defender. Attacks will be launched at 10.1.20.12, which is an interface on the LAMP server.
In the default dos profile no sections are selected or enabled for protected objects in the default configuration.
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protection Profiles page. Click dos, Then Check the Network box under the Families Heading.
- Click the Network Section. Notice all vectors are disabled. Check the top box to select all the vectors, Scroll to the bottom and Select Mitigate. Scroll to the top and Commit Changes to System.
- Navigate to DoS Configuration >> Device Protection. Under Log Publisher select “local-db-publisher” from the drop down. Select Commit Changes to System. This publishes our logs to the appropriate location for analysis.
You will now launch the attacks and show the behavior
- Open the following tabs in the DHD UI (Duplicate Tabs to make it easier):
- DoS Configuration >> DoS Overview >> Filter Type >> Try Both DoS Attack and Device Dos
- Visibility >> Dashboard change Dashboard to Real Time which is centered on the timeline.
- Visibility >> Event Logs >> DoS >> Network >> Events
- Access the Attacker shell and run the following commands/attack (if already in the folder just issue the command)
# sudo su
# cd ~/scripts
# ./multivector.sh
Note
Ignore the “sudo: unable to resolve host” error.
- Click Refresh on the DoS Overview page. Look at and explore both DoS Attack and Device Dos filters to refine your results.
Note
The screens show different info, why? Device Dos shows the status of all vectors for that profile and the current status and rates. Use the last lesson to adjust thresholds of the current attacks to see different results.
Hint
Manual thresholds under Dos Overview >> Filter Type >> Device DoS. Scroll down and see all the vectors and rates. Adjust if you desire.
- Change the View Filter and see how you get different Views of some of the same data in a different context.
- Make sure you adjust the filter to Protected Object and select ServerNet. This will show the status of the protected object, not the device level protection.
- Navigate to Visibility >> Dashboard. Explore the amount of rich data returned. Hover over the attacks. Scroll down and see what information is supplied.
- Notice under Attack Duration the red heart symbol. Signifies an ongoing attack. If you don’t see it. Use Ctrl - to shrink your screen view. Or use the arrow at the top to expand.
Note
Why is there no data in the Virtual Server Section?
Hint
We only have Device Protection and the Server Network /24 protection set. We will see VS when we configure the next exercise.
- Navigate to Visibility >> Event Logs >> DoS >> Network >> Events
- Further explore the DoS Event logs. For example, clear the search and identify the “Stop” and “Start” times for an attack, type, action, PPS and Dropped Packets etc.
- Clean-up: On the Attacker CLI, if the attack is still running be certain to end it with Ctrl-C.
- Clean-up: After stopping the attack, delete the ServerNet Protected Object.
Lab 3 – Using Auto Thresholding¶
This exercise will simulate a newly configured Protected Object where the Security Administrator is unsure what values to assign to a few common vectors. Note that auto-thresholding is useful at both the Device and Protected Object levels.
Note
This demo may place significant stress on the demo environment. This may make the DHD UI less responsive. This is unavoidable since for auto-thresholding to block, the attack must be damaging enough to cause stress, which will push the CPU on the Virtual Environment very high. Remember that this is a virtual environment with minimal resources for lab under high stress and that the Hybrid Defender appliances mitigate these attacks in dedicated hardware.
Task 1 – Create Protected Objects that the baseline traffic will be targeting¶
The DHD device wide protection is enforced for all traffic flowing through the device. For more granular control, we use Protected Objects and configure mitigation settings for those objects to be enforced.
In this task you will configure object-level DoS protection, and then issue an attack and review the results.
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protected Objects page and in the Protected Objects section click the Create dropdown and select Protected Object
Configure the Protected Object using the following information, and then click Create.
Name Server15 Destination Address 10.1.20.15 Port *All Ports Protocol TCP Protection Profile: dos Eviction Policy: Blank VLAN(s): defaultVLAN Logging Profiles: local-dos Click Save
This Protected Object will be used for the Auto-Thresholding lab.
Task 2 – Run Scripts to start L4 traffic generation – Good Traffic¶
- Putty SSH (use the desktop shortcut) to open a shell to the good client system.
- Accept the SSH Warning.
- Enter “ubuntu” as the user. The session is preconfigured to authenticate with a certificate.
- This script will generate baseline traffic against both 10.1.20.14 and 10.1.20.15 (Your Protected Object)
- Start the auto-threshold base-lining script with:
# sudo su
# cd ~/scripts
# ./baseline_l4.sh
- In the Hybrid Defender UI, in Dos Configuration >> Device Protection, Click in the AutoThreshold Section Start Relearning
In the Hybrid Defender Web UI, Navigate to Dos Configuration >> Protection Profiles Select the dos profile and Click the Network box. We will enable auto-thresholding for the following vectors: ICMPv4 Flood, TCP SYN Flood, TCP Push Flood, TCP RST Flood, TCP SYN ACK Flood. If not set to Fully Automatic select each vector and clicking the Set Threshold Mode drop down and selecting Fully Automatic. When all vectors are configured, Go back to the top and Select Commit Changes to System.
- In the Hybrid Defender Web UI, view the Auto Threshold event log by navigating to Visibility >> Event Logs >> DoS >> Network >> Auto Threshold.
Note
The system is updating the detection thresholds. With auto-thresholding, the system adjusts the detection thresholds based on observed traffic patterns.
However, mitigation rate limits are always dynamic based on detected system or protected object stress. If anomalous levels of traffic are running, but there is no stress, the Hybrid Defender will generate alerts but will not block traffic. Under stress, the rate limits are automatically created and adjusted dynamically.
- In the Hybrid Defender UI, navigate to Dos Configuration >> Dos Overview, view in Dos Attack or Device Dos, the device sees no attacks.
Task 3 – Create Stress to trigger Auto Thresholding and view Reports¶
- Let’s create some stress with a Flood attack. In the Attacker CLI start the auto-threshold flood:
# sudo su
# cd ~/scripts
# ./autot_flood.sh
This is a long duration attack. You can terminate it with Ctrl+C when finished.
- In the Hybrid Defender Web UI, view the Dos Configuration >> DoS Overview. Note that the ICMP Flood attack is being mitigated and the rate limit thresholds for each of the auto-threshold vectors have been adjusted based on stress, including vectors that are not detecting or blocking an attack.
- Select the filter type to Protected Object and then Select the Virtual Server Server15 and view how various thresholds are dynamically adjusted based on the stress. But all the blocking is still being handled by the device-dos.
- Terminate the attack in the Attacker CLI with Ctrl+C.
- After the attack has ended, in the Hybrid Defender Web UI, navigate to the DoS Visibility page. Click the Network filter. Under Vectors, select ICMPv4 Flood. View the various details.
Attention
If you want to run other attacks and see the UI and logging, adjust settings so you can mitigate attacks. Please do so. This will also be done in the Advanced Class.
- Clean-up: On the Attacker CLI, if the attack is still running be certain to end it with Ctrl-C.
- Clean-up: After stopping the attack, clear the learning on the Hybrid Defender CLI with:
# tmsh run security dos device-config auto-threshold-relearn
# tmsh run security dos virtual name Server15 auto-threshold-relearn
- Clean-up: Stop the baseline traffic generation from the good-client if still running using CTRL+C
Lab 4 – Configuring L7 Attack Protection¶
In this exercise we will use a protected object and enforce mitigation for low and slow/encrypted layer 7 attacks.
Note
We will first launch attacks with no protection to see the results. Then enable protection and compare the results.
Task 1 – Use Firefox to access Website and use Attacker to bring it down.¶
- Open the following in separate tabs in the Hybrid Defender Web UI:
- DoS Configuration >> Dos Overview
- Visibility >> Event Logs >> DoS >> Application Events
- From a the Firefox browser on the jumphost go to https://10.1.20.11. Ignore SSL warning and Add Exception.
Note
This bypasses the Hybrid Defender and accesses the server directly, showing the availability and/or performance of the site directly.
Click around a few links. This is the site we will launch an attack against and mitigate.
- Verify that the configuration is providing no L7 protections by taking the server offline with a slowloris attack.
Note
Apache will try to clean up the slow flows, but they will do so inefficiently and the server is impacted (which will show as an outage, missing objects and/or slower responsiveness).
- Run the slowloris attack from the Attacker CLI:
# cd ~/scripts
# ./slowloris.sh
- The tool will rapidly show the site offline (10-15 seconds, with trivial traffic load)
- Refresh https://10.1.20.11 to show the effects of the attack. Click links on the page.
Note
Since we are running locally from the Win7 system in a virtualized environment, you may be able to access the site, however it will be slower and often the GIFs will not load. An Internet user would not be able to “fight through” the attack to get to the server as often as a system on the local LAN.
- Stop the slowloris attack by using CTRL+C.
Start a more effective Slow Read attack.
Note
This attack is harder for DoS mitigation tools to mitigate and can be very effective even with a tiny number of concurrent connections trickling in very slowly to the server to fly below the radar of network detections. In our example we will open 10 connections per second and read the response data at 1 byte / sec. The attack would be effective even at 1 cps, it would just take a bit longer to build up the connections.
- From the Attacker CLI/shell start the slowread attack:
# cd ~/scripts
# ./slowread.sh
As soon as the site is down (service available: NO) in the Attacker CLI, refresh https://10.1.20.11 to show that it is down/slow/intermittent.
- In the DDoS Hybrid Defender GUI access the tabs you opened previously and notice no attacks were detected.
- Stop the slowread attack by using CTRL+C.
Task 2 – Create Protection Profile for Dos https Object¶
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protection Profiles page and click the Create button.
- Name the profile dos_HTTPS and select the HTTP Families Vectors.
Change the settings depicted in the image below.
- Hover in the HTTP box and Click in the “”White Space”“
- Click “Per Source IP requests”
- Click the HTTP Group Configuration Link. On the Right Side.
- Under Behavioral and Stress Based Attributes, Set the Operation Mode to Blocking
- Leave Threshold Mode in Manual.
- Under Behavioral Based, Set the Mitigation to Standard Mitigation
- Ensure Signature Detection is Selected.
- Under Mitigation select Request Blocking “Rate Limit”
- Commit Changes to System
Task 3 – Modify Default Eviction Policy¶
Important
When making a Slow-Read attack, a client establishes a connection to the Server and sends an appropriate HTTP request, However, the client reads the response at a very slow speed. Some Slow-Read attack clients don’t read the response at all for long time and then starts reading data one byte at a time just before the idle connection timeout. The clients sends a Zero window to the server which makes the Server to assume that the client is busy reading the data. As a result, the server to keeps the connection opened for long period of time. Such multiple connections to the Server will consume the resources of the server and can make the server unresponsive to the new and genuine requests.
In order to mitigate such an attack we need to make adjustments to the default-eviction-policy.
- Navigate to Dos Configuration >> Eviction Policy and Click on the default-eviction-policy.
- Under “Slow Flow Monitoring” choose “enable” and change the value to 1024.
- Under the “Grace Period” change the default value to 5 Seconds.
- Under “Slow Flow Throttling” change the value to “absolute” and 50 connections as the value.
- Click Update when finished.
What we are doing here is setting up the policy to recognize and then evict slow flows through the DDoS Hybrid Defender.
Task 3 – Create Protected Object¶
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protected Objects page and in the Protected Objects section click the Create dropdown and select Protected Object.
- Configure a protected object using the following information, and then click Save.
Name: | Server_HTTPS |
Destination Address: | 10.1.20.11 |
Service Port: | 443 |
Protocol: | TCP |
Service Profile: | http |
Protection Profile: | dos_HTTPS |
Eviction Policy: | default-eviction-policy |
VLAN(s): | default_VLAN |
Logging Profile(s): | local-dos |
Task 4 – Configure Protection/Mitigation¶
Next we need to modify the VS we created to pass traffic.
- At the bottom of the Menu Click the “Show Advanced Menu”” >> Local Traffic >> Virtual Servers >> Virtual Server List >> Select the Server_HTTPS VS.
- Under “”Configuration”” Select Advanced
- Ensure the following are Set:
- SSL Profile (Client) to clientssl
- SSL Profile (Server) to serverssl
- Source Address translation to none
- Uncheck Address translation
- Uncheck Port translation
- Set Transparent Next Hop to the Internal Interface Bridge Member of the VLAN. If you have followed along, it will be the interface associated with 1.2
- To figure out interface type “tmsh list net vlan” You want the next hop to be the internal interface.
- Click Update
Next we need to modify the Virtual Server Address List Address
- At the bottom of the Menu Click the “Show Advanced Menu”” >> Local Traffic >> Virtual Servers >> Virtual Address List >> Select the address 10.1.20.11
- Under Configuration disable/ uncheck ARP.
- Click Update
Task 5 – Attack Website notice Mitigation/Protection¶
- From the Attacker CLI/shell start the slowread attack:
# cd ~/scripts
# ./slowread.sh
- From Firefox access the website and click around. You will notice although the website is being DoS’d via slow read, the website remains available.
- If you look in the command window of the Attacker the tool even reports the site off-line, although the site remains available.
- On the DHD CLI run the following command.
#tmctl -w 200 virtual_server_stat -s name,clientside.cur_conns,clientside.slow_conns,clientside.slow_killed,serverside.cur_conns,serverside.slow_conns,serverside.slow_killed
- Notice as the slow connections increase, the DDoS Hybrid Defender will start killing them.
- Clean-up: On the Attacker CLI, if the attack is still running be certain to end it with Ctrl-C.
- Clean-up: After stopping the attack, delete the Server Protected Object.
Lab 5 – Configuring L7 Behavioral Attack Protection¶
In this exercise we will use a protected object and analyze how the DDoS Hybrid Defender reacts and mitigates L7 attacks based on Behavioral Analysis.
Task 1 – Create Protection Profile for Dos Behavioral Object¶
- In the BIG-IP Configuration Utility, open the DoS Configuration >> Protection Profiles page and click the Create button.
- Name the profile dos_behavioral and select the “Network” and “HTTP Families”.
- Hover over the Network Box. Click the Pencil in the right corner.
- Ensure Dynamic Signature Enforcement is “enabled”.
- Hover in the HTTP box and Click in the “”White Space”“
- Click “Per Source IP Requests” Under Behavioral and Stress Based.
- Click the HTTP Group Configuration Link. On the Right Side.
- Under Behavioral and Stress Based Attributes, Set the Operation Mode to Blocking
- Leave Threshold Mode in Manual.
- Under Behavioral Based, Set the Mitigation to Standard Mitigation
- Ensure Signature Detection is Selected.
- Commit Changes to System
- Go back and click in HTTP again.
- Select “Per Source IP Requests” Under Behavioral and Stress Based, Select Request Blocking (Near the bottom, right).
- Commit Changes to System
This places this profile into a behavioral based detection profile. No vectors are used in this demo.
Task 2 – Create Protected Object and Launch Attack¶
- In the BIG-IP Configuration Utility, open the DoS Protection >> Quick Configuration page and in the Protected Objects section click
- Create.
- Configure a protected object using the following information, and then click Save.
Name | Auction |
Destination Address | 10.1.20.101 |
Service Port | 80 |
Protocol | TCP |
Service Profile: | http |
Protection Profile: | dos_behavioral |
VLAN(s) | default_VLAN |
Logging Profile(s) | local-dos |
- Click in the whitespace of the Protected Object to get additional info that will be useful for detection and mitigation.
Warning
Name needs to be exact or demo will fail.
- Next we need to modify the VS we created earlier to pass traffic.
- At the bottom of the Menu Click the “Show Advanced Menu”” >> Local Traffic >> Virtual Servers >> Virtual Server List >> Select the Auction Server.
- Under “”Configuration”” Select Advanced
- Ensure the following are Set:
- Source Address translation to none
- Uncheck Address translation
- Uncheck Port translation
- Set Transparent Next Hop to the Internal Interface Bridge Member of the VLAN.
- To figure out interface type “tmsh list net vlan” You want the next hop to be the internal interface.
- Click Update
- Next we need to adjust for ARP.
- Go to >> Local Traffic >> Virtual Servers >> Virtual Address List >> Select the Server 10.1.20.101
- Under Configuration Un-Select ARP.
- Click Update
- From the Good Client CLI, issue the following command.
#sudo su
# cd scripts
#./generate_clean_traffic_101.sh
Make sure you are receiving Status Code 200. If you are not receiving a 200, ask for assistance.
Note
This will need to run for approximately 10 minutes.
- From the DHD CLI issue the following commands:
#/root/scripts/l7bdos-reset.sh
#admd -s vs. | grep -e learning -e health -e attack
You can use variations of the filters in grep if you are familiar.
- Monitor the window. When you see the following number go to 100, you will move on.
- The health of the Protected Object will be shown. In general, a healthy system will show a value around .45. If the value is .5 consistently, then for some reason no learning is occurring and you should check your configuration and verify that baselining traffic is hitting the protected object in question.
- If the system has detected and is mitigating and attack, or not. This will show in the output of ‘info.attack’ signal. The two numbers in brackets indicate if there is an attack (1 = yes, 0 = no) and if the system is mitigating that attack (1 = yes, 0 = no).
- The output will also include the ‘info.learning’ signal, which includes 4 comma-separated values that show the status of the admd behavioral dos learning:
- signal values: [baseline_learning_confidence, learned_bins_count , good_table_size , good_table_confidence]
- baseline learning_confidence in % - How confident the system is in the baseline learning.
- This should be between 80% - 90%
- learned_bins_count - number of learned bins
- This should be > 0
- good_table_size - number of learned requests
- This should be > 4000
- good_table_confidence - how confident, as a percentage, the system is in the good table.
- It must be 100% for behavioral signatures.
- From the Attacker CLI issue the following command:
~/scripts/http_flood_101.sh
- Choose option 1, “Attack Auction”
- You will see the attack start in the DDoS Hybrid Defender SSH window:
- In addition you will see the good client start returning a status of 000 as it is unresponsive. It no longer returns a Status 200. Until the DHD starts mitigation.
- You will see the DDoS Hybrid Defender issue a reset when it mitigates the attack.
- Explore Dos Configuration >> Protected Objects. Click on the “Attack Status” to expand.
- Let this run for 2 minutes. Stop the attack by pressing “Enter”” a couple of times in the Attacker window the choosing option “3” to stop the “Attack”
Note
The DDoS Hybrid Defender does not record the end of the attack right away, it is very conservative, therefore you may have to wait 5 minutes to see the results.
- Look at the Event Logs.
- Look at the Signature created. Advanced Menu >> Security >> Dos Protection >> signatures
- This concludes the DHD Hands on Labs.
Introduction to L7 Behavioral DoS¶
F5’s Application Security Manager, Advanced Web Application Firewall, and DDoS Hybrid Defender products all include advanced functionality for defending L7DoS attacks. In this self-paced lab, attendees will have an opportunity to explore L7 Behavioral DoS (BaDOS), leverage BaDOS to mitigate various L7DoS attacks, and examine the built-in reporting and monitoring functions provided by Advanced Web Application Firewall. At the conclusion of the lab, the attendee will have comfort in the basics of BaDOS, how the feature is deployed, and the types of attacks it can be used to mitigate.
Expected time to complete: 45-60 minutes
Getting Started¶
Please follow the instructions provided by the instructor to start your lab and access your jump host.
Note
All work for this lab will be performed exclusively from the Linux Workstation jumphost. No installation or interaction with your local system is required.
Lab Topology¶
Lab Components¶
The following table lists VLANS, IP Addresses and Credentials for all components:
Component | VLAN/IP Address(es) | Credentials |
---|---|---|
bigip01 |
|
admin /admin |
bigip02 |
|
admin /admin |
Ubuntu Linux Workstation |
|
f5student /f5DEMOs4u |
Kali Linux Workstation |
|
root /f5DEMOs4u |
Accessing Lab Environment¶
Please follow the instructions below to access the lab environment.
- Open a browser and go to http://training.f5agililty.com/
Use the class number and student number included on the class survey to login to the training portal. Once logged in:
Base Configuration and Traffic Baseline¶
In this module, we will configure the base DoS profile and Local Traffic Manager objects used in the remaining modules. Additionally, you will generate traffic needed for Advanced Web Application Firewall Behavioral DoS engine to build a learning baseline.
- Objectives:
- Create DoS Profile
- Create Logging Profile and attach to virtual server
- Create iRule for inserting X-Forwarded-For headers and attach to virtual server
- Generate good traffic to establish BaDOS baseline
- Verify BaDOS learning status
Attention
In this lab, you will configure a number of options to get the lab started. In modules 3 and 4 we will spend time examining the configuration options in more detail. For now, just configure the options as outlined, and we will examine further in later modules.
Set up the DoS profile¶
In the section you will create a DoS profile with Behavioral Detection and Analysis enabled, and attach the DoS profile to the virtual server.
Using Chromium Browser on the Xubuntu Jumpbox, open a tab to the GUI on bigip01 (https://10.1.1.245).
Navigate to Security ›› DoS Protection : DoS Profiles
Select Create. Name your profile hackazon_bados and select Finished. Open your hackazon_bados DoS profile.
Select the Application Security tab from DoS Profile navigation bar.
Click General Settings, select Edit to the right of Application Security in the rightmost panel, and check the Enable box.
This will activate the other sections of the DoS profile.
Tip
At any point you can save your changes by hitting the Update button in the lower left-hand corner
Select the Bot Signatures section, then select the Edit link to the right of Bot Signature Check, and check the Enabled box.
Select Edit next to Bot Signature Categories then change both the Malicious Categories and Benign Categories to Report. This step is necessary because the tools used to generate baseline and attack traffic in this lab will both be categorized as bots.
Attention
The message in red below the Enabled box indicates a DNS Resolver has not been set up. The DNS resolver is used to perform DNS reverse lookups as part of bot identity validation, but is not relevant for this lab exercise.
Select TPS-base DoS Detection and change Operation Mode to Off.
Select Behavioral & Stress-based Detection and change Operation Mode to Blocking.
- Set the Thresholds Mode to Automatic.
- Under Stress-based Detection and Mitigation edit By SourceIP and uncheck Request Blocking. Under By URL uncheck Heavy URL Protection and Request Blocking.
- Under Behavioral Detection and Mitigation check the Request signatures detection and set the Mitigation to Standard. For now, please leave bad actors detection unchecked.
- Click Update in the lower left-hand corner. Collapse all the sections, and Behavioral & Stress-based Detection should match the figure below.
Create a DoS Logging Profile¶
Logging profiles are required to enable local and remote logging for Application DoS and Bot events. In this lab, we will use local logging to review events. Below are the steps to configure the logging profile and attach to your test virtual server.
Go to Security ›› Event Logs : Logging Profiles and click Create on right-hand side of the configuration screen. Name your profile l7_dos_bot_logger then check the DoS Protection and Bot Defense enable boxes.
From the DoS Protection tab enable the Local Publisher.
From the Bot Defense tab check ALL the boxes.
Click Finished.
Add the DoS profile to a virtual server¶
Below are the steps to associate this profile with the Local Traffic Manager virtual server processing the application traffic in this lab.
Navigate to Local Traffic > Virtual Servers > Virtual Server List and select vs_hackazon_http. Under the Security tab on the top bar select Policies.
Enable the DoS Protection Profile and select the hackazon_bados profile.
Add l7_dos_bot_logger to the Log Profile and Update
For purposes of this lab, Disable the Application Security Policy and remove asm_allrequests from the Log Profile.
Create XFF-Mixed_Attacker iRule¶
Because we do not have dozens of good and bad source IPs available for clients and attackers in this environment, we simulate them by adding an iRule to the virtual server. The iRule adds a randomized X-Forwarded-For (XFF) header to each request.
Navigate to Local Traffic ›› iRules : iRule List and select Create. Name a new iRule named XFF_mixed_Attacker_Good_iRule. Copy and paste the iRule below.
1 2 3 4 5 6 7 8 9 10 11 12 13
when HTTP_REQUEST { # Good traffic if { [IP::addr [IP::client_addr] equals 10.1.10.52] } { set xff 153.172.223.[expr int(rand()*100)] HTTP::header insert X-Forwarded-For $xff } # Attack traffic if { [IP::addr [IP::client_addr] equals 10.1.10.53] } { set xff 132.173.99.[expr int(rand()*25)] HTTP::header insert X-Forwarded-For $xff } }
Advanced Web Application Firewall/Application Security Manager will honor the X-Forwarded-For header by enabling this in the http profile.
Create HTTP Profile to Accept X-Forwarded-For HTTP Header¶
- Navigate to Local Traffic ›› Profiles : Services : HTTP and click Create. Name the new http profile xff_http, and click the rightmost checkbox in the row Accept XFF to enable a custom setting, then click the checkbox to the immediate right of Accept XFF to enable processing of an inbound X-Forwarded-For header.
- Click Finished button at bottom of configuration page.
Tip
Due to a large number of service profiles, occasionally part of the Services menu will get stuck under the browser menu. If that happens, click on Profiles on the side-bar, then click Services in the top navigation bar to get to the HTTP profile.
Attach iRule and HTTP Profile to Local Traffic Manager Virtual Server¶
- Navigate to the vs_hackazon_http virtual server. In the Properties tab, under Configuration section, select xff_http for the HTTP Profile.
- Click the Resources tab in the virtual server navigation bar, in the iRules section select the Manage button, and move the XFF_mixed_Attacker_Good_iRule from the Available to the Enabled box.
- Click Finished button at bottom of the Resource Management page.
Generate Traffic to Establish Baseline¶
Advanced Web Application Firewall’s Behavioral DoS feature is based on learning and analyzing all traffic to the web application, building baselines, and then idenitifying anamolies when server stress is detected. As a result, in this lab, we need to generate normal traffic allowing Advanced Web Application Firewall to build a baseline.
You will use the Xubuntu Jumpbox to generate legitimate traffic and bad traffic, eth1 has 10.1.10.51-55 configured and 10.1.10.52 will be the source-IP used for the good traffic script. The source IP will match XFF_mixed_Attacker_Good_iRule created above, and an X-Forwarded-For header will be placed in the HTTP request in the 153.172.223.0/24 IP address range.
In the home directory (/home/f5student) on the Xubuntu Jumpbox, you will find the two scripts used for this lab:
- baseline_menu.sh - is used to create baseline traffic
- AB_DOS.sh - is used to launch L7 DOS attacks
Start baseline traffic, using Xubuntu Jumpbox Terminal application, navigate to the home directory, then type:
f5student@xjumpbox~$ ./baseline_menu.sh - Select option 2 **alternate** and keep it running in the window
Tip
This is your valid traffic, and the number of requests will change over time. The requests also change as the script continuously alters the User-Agent header and the requested URI. Both values are randomly taken from files in the “source” directory in the home directory.
Next, validate you are seeing the traffic, and Advanced Web Application Firewall is actively building learning baselines. From a separate Terminal window type:
f5student@xjumpbox$~ ssh root@10.1.1.245
Then, run the following command:
[root@bigipo01:Active:Standalone] config # admd -s vs./Common/vs_hackazon_http+/Common/hackazon_bados.info.learning - /Common/vs_hackazon_http – is the name of the virtual server - /Common/hackazon_bados – is the name of the DoS profile. **It may take several minutes for baseline numbers to be generated**Screenshot of sample output below:
Tip
If your aren’t getting any output, or seeing no signs of accumulated signals, verify the name of the virtual server and profile in the admd command are accurate.
- baseline_learning_confidence:
- Description: in % how confident the system is in the baseline learning.
- Desired Value: > 90%
- learned_bins_count:
- Description: number of learned bins
- Desired Value: > 0
- good_table_size:
- Description: number of learned requests
- Desired Value: > 2000
- good_table_confidence:
- Description: how confident, as %, the system is in the good table
- Desired Value: Must be 100 for signatures
Note
It may take 5 or more minutes before you begin to get learned baseline numbers. Also, the desired values are the minimum values we would like to see prior to triggering attacks as part of this lab exercise. You can, however, move onto module 3 and 4 in this lab while baselines are being established. Do not stop baseline traffic script
To see all of the values available and wide range of interesting statistics, enter the following command from BIG-IP console:
admd -s vs./Common/vs_hackazon_http
To view Advanced Web Application Firewall layer 7 DoS log, enter the following command from BIG-IP console:
tail -f /var/log/dosl7/dosl7d.log
Note
The goal of this module is to explain DoS profile configuration options. The module does not contain any exercises. If you are already familar with a the settings in an Application Security DoS profile you can skip to module 4.
Application Security DoS Profiles¶
In this module, we will review the various settings and options that make up a layer 7 DoS profile. We will not review each and every setting, leaving that exercise up to the reader, but instead will focus on key settings which will most likely require attention during a production deployment. More detail on each individual setting can be found by viewing the Help on left side of the BIG-IP Configuration Utility (GUI).
Review DoS Profile General Settings¶
Navigate to Security ›› DoS Protection ›› DoS Profiles and click the DoS profile hackazon_bados created earlier for this module.
Settings in this screen are profile wide, and can affect all aspects of the dos configuration.
- Application Security
This setting enables or disables the DoS profile.
- Heavy URL Protection
Heavy URL’s are application resources which may consume more backend resources with each client request. Additionally, URLs which are not generally considered heavy may become heavy under significant load or attack. As a result, low rate requests targeting these URLs can cause significant DoS attacks, and be difficult to differentiate from legitimate requirements based on rate alone. Advanced Web Application Firewall automatically detects heavy URLs by measuring the latency tail ratio, which is the number of transactions whose latency is consistently greater than the latency threshold defined in this configuration option. A URL is considered heavy if its latency is more than two times the site global average over a 24 hour (default) period.
Checkbox, enables or disables, automatic detection of the heavy URLs profile-wide. The text box allows for configuration of the baseline threshold that URLs must exceed before being considered for heavy URL determination.
This section of the DoS Profile Heavy URL configuration allows an administrator to explictly configure a URL(s) as heavy, whether it is detected as heavy by Advanced Web Application Firewall or not. Use this section to define application resources which are known to be heavier in terms of resource consumption, or known to be less resilient to higher volumes of traffic than the rest of the application.
This section of the Dos Profile Heavy URL configuration allows an administrator to explictly configure URL(s) and wildcard URL patterns to be excluded from automatic heavy URL detection. Use this section, to identify URL’s which you know may perform slower than average under normal conditions, or URLs you do not wish to have Advanced Web Application Firewall offering heavy URL protection.
Note
To provide mitigation for heavy URLs, you must enable at least one of the URL-based prevention policy methods in the TPS or Stress-based Anomaly sections of the DoS profile.
Geolocations Geolocations provides options to override the dos profile geolocation detection criteria by explicity whitelisting or blacklisting specific geolocations.
Single Page Application Single Page Applications (SPA) represent a change in application architecture that moves much of the content rendering and routing to client-side code. Application requests which require server-side processing are sent as AJAX requests towards server, and the response is typically JSON/XML; this is different from traditional web applications that send HTTP requests, and generally levergage HTML as the predominant response content type. As a result, Advanced Web Application Firewall needs to modify the way it challenges clients for features like Proactive Bot Defense and capturing Device ID in the TPS/Stress based anomaly detections. Enabling this option modifies Advanced Web Application Firewall’s challenge and challenge validation mechanisms. When deploying L7 DoS protections it is important to understand the application architecture, and if protecting a SPA, enabling this option is critical for proper operation.
Note
The goal of this module is to explain the Stress-Based and Behavioral DoS configuration options. The module does not contain any exercises. If you are already familar with a the settings you can skip to module 5.
Stress-Based and Behavioral DoS Profile Settings¶
In this module, we will review the various settings for configuring Stress-based and Behavioral DoS protections in more detail. We will not review each and every setting, leaving that exercise up to the reader, but instead will focus on key settings which will most likely require attention during a production deployment. More detail on each individual setting can be found by viewing the Help on left side of the BIG-IP Configuration Utility (GUI).
Review Stress-Based Dos Profile Settings¶
To appreciate the powerful nature of Advanced Web Application Firewall’s Behavioral DoS feature, it first makes sense to analyze one of the other L7 DoS protection mechanisms. For this exercise, we will examine the options and behaviors of the Stress-based DoS protections available in an Application Security DoS profile.
To review the settings below, navigate to Security ›› DoS Protection ›› DoS Profiles, click the DoS profile hackazon_bados created earlier for this module, then click Behavioral & Stress-based Detection in the Application Security navigation menu, and set the Operation Mode to Transparent.
Operation Mode Defines the operational mode for the stress-based dos protection feature. Available options include: Blocking, Transparent, Off. Blocking means feature will detect, report, and mitigate. Transparent means feature will detect, report, but will not mitigate. Off means the feature is disabled.
Threshold Mode Defines how Advanced Web Application Firewall derives thresholds to be used in detecting the TPS component of a stress-based attack. Options include:
- Manual: Administrator explicity configures TPS and percentage thresholds based on their knowledge of the environment or specific requirements.
- Automatic: Advanced Web Application Firewall monitors traffic rates automatically and calculates the thresholds based on normal traffic volume to the application.
Stress-based Detection Options Advanced Web Application Firewall can trigger an attack if any/all of the following detection methods exceed the thresholds defined or calculated for the detection method:
- By Source IP: A specific source IP has exceeded the thresholds defined in the detection thresholds.
- By Device ID: A specific device has exceeded the thresholds defined in the detection thresholds. Device ID is ASM calculating a fingerprint for a given device. The feature requires Javascript injection for proper operation. However, the feature offers the benefit of detecting a specific device, even if the attack varies its source IP address.
- By Geolocation: A country/geolocation has exceeded the thresholds defined in the detection thresholds.
- By URL: Request traffic to a specific (or set of URL’s identified in URL patterns section of the DoS Profile General Properties) has exceeded the thresholds defined in the detection thresholds.
- Site Wide: Request traffic to the entire web site has exceeded the thresholds defined in the detection thresholds, and an attack has not been detected using any of the other detection criteria. Site-wide is considered last resort.
Note
It is important to understand that while stress-based protections are monitoring server latency, and tracking application request volume in short and long term intervals, the detection methods listed above are the only ways to identify when an attack is on-going. This, as you will see, is quite a bit different than they way Advanced Web Application Firewall Behavioral DoS feature identifies attacks and attackers!
Review Behavioral DoS Settings¶
Having reviewed the options for configuring Stress-based dos mitigation, now let’s examine the options required for configuring Advanced Web Application Firewall’s Behavioral DOS mitigations.
Bad Actors Behavior Detection Determines whether Behavioral DoS engine tracks and attempts to identify the bad actors contributing to a given set of malicious traffic. When Bad Actor Behavior Detection is enabled, once Advanced Web Application Firewall detects server stress and identifies a set of malicious traffic contributing to the server stress, the Behavioral DoS engine then attempts to identify what source IP addresses are generating the malicious traffic, and what percentage of malicious traffic a given bad actor is contributing. Bad actors, are mitigated at transport layer via slowdown mitigation techniques, and the rate at which they are mitigated is directly related to their percentage of contribution to the malicious traffic set, and the migitation mode selected.
Request Signature Detection Determines whether Behavioral DoS engine will attempt to generate a traffic signature to block anamolous traffic. Advanced Web Application Firewall Behavioral DoS feature is in a permanent learning state, always tracking application requests, and the construction of these requests, and then comparing to an evolving baseline. When Request Signatures Detection is enabled, once Advanced Web Application Firewall detects server stress, it looks to identify traffic characteristics which have deviated from the baseline. If there are deviating characteristics, the Behavioral DoS engine, then dynamically generates a signature based on these deviating characteristics to block anamolous traffic.
Note
In addition to generating signatures the Behavioral DoS Engine also continually evaluates the signature for efficacy, minimizing the risk of signature becoming false positive and blocking known good traffic.
Use Approved Signatures Only By default, when Request Signatures Detection is enabled, Advanced Web Application Firewall will generate and use dynamically generated attack signatures as defined by the mitigation mode selection. By enabling this option, the administrator overrides this behavior, and forces a manual step to review and approve the signature prior to any mitigations taking effect. Signatures can be reviewed from Advanced Web Application Firewall GUI via Security -> DoS Protection -> Signatures.
Mitigation Defines the mitigation mode for Advanced Web Application Firewall Behavioral DoS. Options include:
- No Mitigation:
- Monitors traffic, generates signatures, and identifies bad actors, but does not perform any mitigation.
- Conservative Protection:
- If Bad Actors Behavior Detection is enabled, slows down bad identified bad actors.
- If Request Signatures Detection is enabled, blocks requests that match attack signatures
- Standard Protection:
- If Bad Actors Behavior Detection is enabled, slows down bad identified bad actors.
- If Request Signatures Detection is enabled, blocks requests that match attack signatures
- Rate limits all requests based on server health
- Limits the number of concurrent connections from bad actor IP addresses
- If necessary, limits the number of all concurrent connections based on server health
- Aggressive Protection:
- If Bad Actors Behavior Detection is enabled, slows down bad identified bad actors.
- If Request Signatures Detection is enabled, blocks requests that match attack signatures
- Rate limits all requests based on server health
- Limits the number of concurrent connections from bad actor IP addresses
- If necessary, limits the number of all concurrent connections based on server health
- Proactively performs all protection actions, even before attack detection, increasing impact of protection techniques.
Advanced Web Application Firewall mitigates DoS with the most effective and efficient method available, and as quickly as possible to restore server health. Meaning, the mitigation method will often change over time as more data is learned and analyzed. For example, at the onset of an attack, Advanced Web Application Firewall may apply global rate limiting in an attempt to mitigate an onslaught of traffic. Then, as the signature engine has observed enough traffic to identify malicious traffic and generate a signature, the Behavioral DoS engine will begin mitigating with request signatures and discontinue global rate limiting. Finally, as bad actors are identified, traffic from those sources is mitigated using layer four slowdown mechanisms, and request signatures are only used for traffic matching the signature and not in the bad actor list. This approach allows Advanced Web Application Firewall to perform better under attack, and mimimizes the risk of blocking good traffic while mitigating DoS.
Summarizing Key Points¶
After reviewing several options for both Stress-based and Behavioral DoS features, the goal of this section is to call out some key points which might be overlooked when reviewing configuration options:
- All DoS features are complementary to Advanced Web Application Firewall web application firewall (WAF) and bot protection features. DoS features mitigate traffic that exceeds a certain rate or induces server-side stress. This traffic is, many times, completely legitimate traffic which will not trigger a WAF block.
- Heavy URL, TPS-based DoS, Stress-based DoS, and Behavioral DoS features can all be configured concurrently, complementing one another, or separate and independent of one another.
- Both Stress-based and Behavioral DoS protection features continually monitor application server performance for signs of server stress. Both features will consider server stress as a key component in detecting an attack, and neither will trigger a mitigation if the server is perceived to be healthy.
- Stress-based and TPS based DoS features can detect DoS attacks across a pre-defined set of detection criteria (source IP, URL, device ID, geolocation, site). Behavioral DoS is not constrained to a pre-defined set of detection criteria, but instead is a self adjusting dynamic DoS defense system which can detect DoS across hundreds of traffic predicates. As a result, Behavioral DoS, is much more effective in mitigating multi-vector layer seven DoS attacks which mutate over time. Conversely, TPS and/or Stress-based DoS features are much better at defining specific rate limits for traffic entering your application.
Request Signatures¶
In this module you will be initiating a L7 DDoS attack on the hackazon virtual server, from eth1, using 10.1.10.53 as the source IP address. This source IP will match XFF_mixed_Attacker_Good_iRule, and an X-Forward-For header will be inserted in the HTTP request in the 132.173.99.0/24 IP address range.
Once the attack begins the BIG-IP WAF (ASM) will immediately switch into attack mode due to the server health deteriorating almost immediately. As the server gets totally overwhelmed, you may at first notice the good script dropping requests. That’s why BaDoS first mitigates with a global rate limit just to protect the server. In a short time, the good script will go back to all 200 OK responses. During this time Behavioral DoS identifies anamolous traffic and generates Dynamic Signatures matching only the malicious traffic. Once mitigation is in effect, the server health will rapidly improve and application performance will return to normal.
Using Chromium Browser on the Xubuntu Jumpbox, open tab to the GUI on bigip01 (https://10.1.1.245)
Navigate to Security ›› DoS Protection:Signatures and click on the Dynamic box, then set the Refresh value to 20 secs.
Open another tab/window in Chromium Browser, and go to Security ››Reporting : DoS : Dashboard. The dashboard is NOT real time in may take up to 10 minutes for traffic to display.
Revisit the Terminal window you opened earlier which is monitoring behavioral DoS learning signals. Verify the first number (baseline_learning_confidence) is at or above 80%. Normally, above 90% would be ideal, but for the purposes of this lab over 80% will suffice.
Revisit the Terminal window you opened earlier which is still running the baseline traffic generation script. Make note of the normal, pre-attack, response time for each request.
From Xubuntu Jumpbox open a NEW Terminal window. From your home directory enter:
f5student@xjumpbox~$ ./AB_DOS.sh - Select **2** – Attack start - score
Using Chromium Browser on the Xubuntu Jumpbox, open another tab to the GUI on bigip01, and navigate to Security ›› Event Logs ›› DoS ›› Application Events
Almost immediately you should see an attack has started, and Advanced Web Application Firewall has assigned an Attack ID to the event. You will see something similar to the screenshot below:
- Review the Dyanmic Signatures UI page opened in step #2. It might take a few moments for a dynamic signature(s) to generate, but shortly after the attack has been detected a signature should be created. Once a signature(s) is generated, if you click on the signature (NOT on the blue link, but somewhere on the signature bar), you will get the details about the signature in Wireshark format. Also, you can examine the current status of the signature (mitigating or not), and statistics on recent attacks which used the signature.
Signature ID: Signature ID generated for this signature. You can use the signature ID in DoS Analysis/Dashboard views (explored in module 6) to get more details on actions taken by this signature.
Deployment State: current state of the signature. Options include:
- Mitigate - Collect stats, learn, alert, and mitigate. All thresholds and threshold actions are applied, and rate limiting occurs if the device is under high stress.
- Detect Only - Collects stats, learn, and alert. Develops dynamic signatures without enforcing any thresholds or limits.
- Learn Only - Collect stats and learn. Develops dynamic signatures without enforcing any thresholds or limits
- Disabled - No stat collection or mitigation, totally disables the signature.
Attack Status - the state of the signature with respect to ongoing attacks. Specifically, defines whether this particular signature is being used to mitigate an on-going attack.
Attack ID - the attack ID for the attack that generated this signature. Clicking the attack ID will take you to the DoS Analysis views filtered on this attack ID.
Predicates List - the conditions for the request to be associated with this signature. Includes one or more match ,expresssions, joined by logical operators, which the system uses to match traffic causing a DoS attack.
Attack History - provides an account of all attacks in which this signature has been used to mitigate.
Note
Dynamic Attack signatures generated will remain in the list up to the max number of signatures supported, and will be will re-used whenever an attack is detected, and traffic matches the conditions defined in the signature
- With the attack script still running, examine the output of the baseline script. You should be getting HTTP 200 OK responses, and the response time should be inline with pre-attack response times. Also, verify you can use browse to http://hackazon.f5demo.com without issue.
- In the window where you are running the attack script, enter CTRL-C, then type 4 to kill the attack script cleanly.
- Using Chromium Browser, navigate to Security ›› DoS Protection:Signatures and click on the Dynamic box. Then click the check box next to the Name column to select all signatures, and click delete to remove all attack signatures created during this module.
- Leave baseline_menu.sh script running.
Bad Actor Detection¶
In the last module, you used request signature detection to mitigate an application layer DoS attack. You also saw the Behavioral DoS engine deploy global rate limiting to bring the servers back to health while signatures were being generated, then mitigate targeted attack traffic with the newly generated signature. In this module, we will leverage Bad Actor Detection to throttle known bad actors.
- Navigate to Security ›› DoS Protection : DoS Profiles and click the hackazon_bados profile we created earlier.
- Click the Application Security tab, and then click Behavioral & Stress-based Detection button in the Application Security panel.
- Click the Edit link to the right of the Behavioral Detection and Mitigation section, then check the checkbox for Bad actors behavior detection, and uncheck the box next to request signatures detection
- Scroll down, and click the Update button.
- From the Xubuntu Jumpbox open another Terminal window. Then:
f5student@xjumpbox$~ ssh root@10.1.1.245
- From the SSH session, run the following command:
[root@bigipo01:Active:Standalone] config # watch ipidr -l /Common/vs_hackazon_http+/Common/hackazon_bados
Initially, because no attack is active, the IP list will be empty. Keep this command running in one of the Terminal windows. Things are about to change!
- Using the Terminal window on the Xubuntu Jumpbox from the previous module, or a new one, re-run the attack script using the following command:
f5student@xjumpbox~$ ./AB_DOS.sh - Select **2** – Attack start - score
- Using Chromium Browser on the Xubuntu Jumpbox, open another tab to the GUI on bigip01, and navigate to Security ›› Event Logs ›› DoS ›› Application Events
- Almost immediately you should see an attack has started, and Advanced Web Application Firewall has assigned an Attack ID to the event. You will see something similar to the screenshot below:
- From the Terminal window started in step #6, monitor the output of the ipidr command, and the status of the IP greylist. You should see something similar to the image below:
- IP: IP address that is member of the greylist
- Rate: Probability of drop for an ingress packet. Higher number equals higher drop rate at the TCP layer. As drop rate goes up, retransmit rates increase, and subsequently TCP window sizes adjust closer to zero. Also, note this behavior will be different if the client IP is learned through a layer 7 header. If so, the behavior will be an HTTP rate limit versus TCP based mitigations.
- Prod: Number of stat producers. In this environment, this should always be 1.
- Tout: Time-out/TTL. Prior to releasing an IP address from the greylist, Advanced Web Application Firewall will quarantine the IP address for a period of time. During this time, TCP slowdown methods will discontinue, and HTTP rate limiting will take over. If during the quarantine period, the IP address triggers more attack traffic, the IP will be removed from quarantine and placed back in greylist. Quarantined IP addresses are visible in the DoS Dashboard/Analytics views in the Mitigation panel.
- With the attack script still running, examine the output of the baseline script. You should be getting HTTP 200 OK responses, and the response time should be inline with pre-attack response times. Also, verify you can use browse to http://hackazon.f5demo.com without issue.
- In the window where you are running the attack script, enter CTRL-C, then type 4 to kill the attack script cleanly.
- Leave baseline_menu.sh script running.
Bad Actor Detection and Request Signatures¶
In the previous modules, we examined both request signature detection and bad actor detection mitigations individually. In this module, we will enable both mitigations together, and explore how they operate in tandem to mitigate a DoS attack. Additionally, we will use Advanced Web Application Firewall’s DoS Reporting tools to further inspect the details of each attack.
- Using Chromium Browser on the Xubuntu Jumpbox, open another tab to the GUI on bigip01
- Navigate to Security ›› DoS Protection : DoS Profiles and click the hackazon_bados profile we created earlier.
- Click the Application Security tab, and then click the Behavioral & Stress-based Detection button in the Application Security panel.
- Click the Edit link to the right of the Behavioral Detection and Mitigation section, then uncheck the checkbox next to Bad actors behavior detection, and check the box next to Request signatures detection
- Scroll down, and click Update button.
- Navigate to Security ›› Reporting ›› DoS ›› Dashboard
- From the DoS Dashboard select the refresh drop down and set value to 1 min, and grab the slider bar at the top and drag it as far right as possible.
- On the right side of the DoS Dashboard, grab the handle just to the right of the HTTP and Network filter labels, and pull left to the midway point of the screen.
- Using the inner-most vertical scroller on the right-hand side of the screen, scroll down until you see the Transaction Outcomes dynamic panel. Click the panel to expand, then click the three vertical lines to the left of the Transaction Outcomes label. Click on Columns, and click the green icon to remove all row labels except the following:
- Transactions
- Attacks
- Valid Transactions
- Mitigated Transactions
- Blocked Transactions
- Imcomplete Transactions
- Repeat the same process to filter the Behavioral Signatures dynamic panel.
- With the baseline traffic still running, examine both the Transaction Outcomes and Behavior Signatures panels. You should see all transactions have an outcome of Passthrough. Also, the center column of the main dashboard view should show no current attacks in progress. Keep this window open.
- From the Xubuntu Jumpbox open another Terminal window, or return to a previously opened window. Then:
f5student@xjumpbox$~ ssh root@10.1.1.245
- From the SSH session, run the following command:
[root@bigipo01:Active:Standalone] config # watch ipidr -l /Common/vs_hackazon_http+/Common/hackazon_bados
- From the Xubuntu Jumpbox open another Terminal window, or return to a previously opened window. Then, re-run the attack script using the following command:
f5student@xjumpbox~$ ./AB_DOS.sh - Select **2** – Attack start - score
- Open another tab to the GUI on bigip01, and navigate to Security ›› Event Logs ›› DoS ›› Application Events
- Almost immediately you should see an attack has started, and Advanced Web Application Firewall has assigned an Attack ID to the event. You will see something similar to the screenshot below:
- Open another tab to the GUI on bigip01, and navigate to Security ›› DoS Protection : Signatures, and click on the Dynamic box, then set the Refresh value to 20 secs. In a few moments, you should see request signatures being generated.
- Return to the browser tab opened to the DoS Reporting Dashboard. Monitor the Transaction Outcomes and Behavioral Signatures dynamic panels. After a few minutes, you will begin to see signature based mitigations, and your dashboard should like similar to the image below:
- DoS Dashboard view shows an attack has been triggered. Select the attack, click the filter icon in upper right hand corner of Attacks table, and you can adjust the columns to view.
- This attack was initially mitigated with HTTP global rate limiting before a signature can be generated, accounted for in the DoS Blocked row. Then, as an attack signature is generated, all attack traffic should begin to be blocked with the request signature(s), evident by looking at the Blocked Bad Request row in transaction outcomes. At this point, if you refresh the dashboard, DoS Blocked counts should remain static, and Blocked Bad Request counters should be incrementing.
- Behavioral DoS will generate and adjust signatures as the traffic changes. This panel shows the signatures, referenced by signature name, that have been used to mitigate this attack.
- Look back at the browser tab showing the Dynamic Request Signatures. You should now see that not only have signatures been generated, but they are active in mitigating a current attack. See below:
- Using a different browser tab, navigate to Security ›› DoS Protection : DoS Profiles and click the hackazon_bados profile. As you did earlier, edit the Behavioral Detection and Mitigation section. This time, check the checkbox next to Bad actors behavior detection, then click Update.
- Return to the browser tab monitoring the DoS event logs. Soon, you will see Advanced Web Application Firewall ends the current attack, and immediately triggers a new attack. Your DoS Application Events log should look similar to the below image:
- Return to the browser tab opened to the DoS Reporting Dashboard. Monitor the Transaction Outcomes and Behavioral Signatures dynamic panels. After a few minutes, you will begin to see transactions being mitigated with Blocked Bad Actor. Shortly after you begin seeing transactions being mitigated via bad actor detection the Blocked Bad Request row should stop incrementing blocked transactions. Also, you should now see another attack has been triggered in the Attacks table. Your DoS dashboard should look similar to below image:
- Return to the Terminal window from step #13 above. You should see the IP greylist again adding attacking IP addresses.
- Return to the browser tab monitoring the Dyamic Request Signatures, and examine the attack status for the attack signatures and EPS counter. You should see the attack status as Detected, not mitigating, and EPS should be 0. This attack is now being mitigated excusively by bad actors as in the previous module.
Bonus¶
The exercise above shows Request Signatures and Bad Actor Detection working in tandem to mitigate an attack. However, we have a relatively small set of attackers, so almost immediately Advanced Web Application Firewall will identify all the bad actors, and the attack will be 100% mitigated with bad actor detection. In the real world, it is highly likely the set of attackers will be very large and dynamic. So, it is quite possible, that as soon as bad actors are detected, the attacking sources will change. At that point, you will see an attack being mitigated by both request signatures and bad actors. Try the below steps to simulate this activity.
- Return to the iRule configured in module 1 (Create XFF-Mixed_Attacker iRule)
- Modify line #10 to match below and click Update
1 2 3 4 5 6 7 8 9 10 11 12 13 when HTTP_REQUEST { # Good traffic if { [IP::addr [IP::client_addr] equals 10.1.10.52] } { set xff 153.172.223.[expr int(rand()*100)] HTTP::header insert X-Forwarded-For $xff } # Attack traffic if { [IP::addr [IP::client_addr] equals 10.1.10.53] } { set xff 112.173.99.[expr int(rand()*1000)] HTTP::header insert X-Forwarded-For $xff } }
- Return to the browser tab monitoring the DoS Dashboard. Shortly, after the iRule change you should now see the Blocked Bad Request counter incrementing again. In time, Advanced Web Application Firewall will begin to learn all the new IP’s as well, but you should have enough time to see both mitigations active concurrently.
- Return to the browser tab monitoring the Dynamic Request Signatures. You should now see the attack signatures are again active and mitigating the attack until all new sources have been learned by bad actor detection.
This completes the Introduction to L7 Behavioral DoS Self Guided Lab. Thanks for attending the session, and have a great week at F5 Agility 2018!