Framework for Preprocessing and Feature Extraction from Weblogs for Identification of HTTP Flood Request Attacks

Abstract

The HTTP flood attacks are carried out through enormous HTTP requests generated by automated software agents within a short period. The application layer is more vulnerable to HTTP flood attacks and exhausted computing and communication resources of the web server to disrupt the different web services. All HTTP requests are stored at the server as a web log file. However, malicious automated software agents camouflage their behavior on the web server logs and pose a great challenge to detect their HTTP requests. It is assumed that navigational behavior of actual visitors and automated software agents are fundamentally different. In this paper, a framework for weblog preprocessing and extracting various predefined features from raw web server logs is implemented. The most effective features are identified which are potentially useful in differentiating legitimate users and automated software agents. The sessionized HTTP feature vectors are also labeled as an actual visitor or possible web robots. The experiments are performed on raw weblogs of a commercial web portal.