Systems and methods for processing data flows

Abstract

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit of the following provisional applications, each of which is hereby incorporated by reference in its entirety: U.S. application Ser. No. 60 / 749,915 filed on Dec. 13, 2005 and entitled “HIGH SPEED PATTERN MATCHING”; U.S. application Ser. No. 60 / 750,664 filed on Dec. 14, 2005 and entitled “USING NEURAL NETWORKS TO DETECT ANOMALOUS COMMUNICATIONS FLOWS”; U.S. application Ser. No. 60 / 795,886 filed on Apr. 27, 2006 and entitled “SYSTEM AND METHODS OF FLOW PROCESSING FOR UNIFIED THREAT MANAGEMENT”; U.S. application Ser. No. 60 / 795,885 filed on Apr. 27, 2006 and entitled “SYSTEM AND METHODS OF FLOW PROCESSING FOR VIRUS PROTECTION”; U.S. application Ser. No. 60 / 795,708 filed on Apr. 27, 2006 and entitled “SYSTEMS AND METHODS FOR FLOW PROCESSING”; U.S. application Ser. No. 60 / 795,712 filed on Apr. 27, 2006 and entitled “SYSTEM AND METHODS OF FLOW PROCESSING WITH MACHINE LEARNING”; and U.S. application Ser. No. ...

AI Technical Summary

Benefits of technology

[0020] Since all, or nearly all of the data accessed and used by internal users, external users, clients, servers, vendors, and the like passes through an organization's network, segmenting the network to address the various needs of the network participants can be costly because of the substantial expense associated with hardware security facilities. Also, segmenting may not relieve the constraints sufficiently to justify this expense. In addition, management of a myriad of segmented, network management devices increases complexity which may create new opportunities for segments being vulnerable to intrusion.

[0083] In another aspect of the invention, method and systems of network security may include providing a flow processing facility for processing a data flow; learning a network activity baseline; processing a data flow to calculate a rate of network activity; comparing the learned baseline to the calculated rate to detect one or more anomalies in the data flow; and preventing an anomalous data flow from propagating an intrusion to the network.

Problems solved by technology

The latter approach, known as unified threat management, offers more comprehensive protection against threats; however, the protection comes at the expense of processing resources, as each application in a unified threat management suite must use such resources.

One type of standalone products, known as firewalls, addresses and protects against these kinds of threats; however, this protection comes either at the expense of processing resources (in cases where a software firewall product must be installed on a server) or at the expense of operational complexity (in cases where the firewall product is embodied in a dedicated network device).

Network security is also being threatened from ever increasingly sophisticated threats that attack any and all vulnerabilities of network communication systems.

Packet switched network communication systems remain vulnerable to security threats in part due to their layered protocol schemes.

While this helps boost productivity, each Internet-based endeavor potentially opens another door to outside hackers and malicious code attacks.

Companies are also faced with legal and ethical responsibility of their information and network security.

Systems that provide only intrusion detection may have substantial drawbacks in this environment including false alarms, low manageability, high maintenance, and no prevention of attacks.

False alarms may manifest as large quantities of records that require manual filtering, a costly and error prone process.

An intrusion detection system that requires substantial time and effort to maintain detection sensors, security policies, and intrusion lists may contribute to poor intrusion detection.

Critical threats include, for example, viruses, network security holes, network communications, content inspection, intrusions, and other attacks that can be blocked by firewalls.

Providing a network security solution that effectively delivers all of one participant's access needs may impose constraints on one or many other participants' needs such as making critical aspects of the network vulnerable to intrusions.

Since all, or nearly all of the data accessed and used by internal users, external users, clients, servers, vendors, and the like passes through an organization's network, segmenting the network to address the various needs of the network participants can be costly because of the substantial expense associated with hardware security facilities.

Also, segmenting may not relieve the constraints sufficiently to justify this expense.

In addition, management of a myriad of segmented, network management devices increases complexity which may create new opportunities for segments being vulnerable to intrusion.

Security violations may include intrusion of applications, databases, file systems, operating systems, network communications, and security policies.

Images