Hundreds of Columns Sorted by Topic
Real-Time Traffic Intelligence for Defending Networks in an Age of CyberWar
by Antonio Nucci, CTO
10/6/2008
Managing and securing large IP networks has become nothing short of a nightmare for network operators due to their increasing complexity. Defending against a gamut of innovative and sophisticated network attacks and the prospect of cyberwarfare add to the complexity, making it harder for operators to effectively deliver value-added services to increase business revenue. Operators tend to install silo applications to address specific network problems, resulting in inefficient business operations.
However, operators are turning to a new concept in which their existing infrastructure investments are leveraged and combined with security and traffic management solutions to create a complete real-time traffic intelligence system.
Common Solutions for Managing and Protecting IP Traffic
Historically, operators have purchased siloed applications and installed them incrementally to address specific needs, each of them deployed to solve a specific problem. This practice led to a dispersion of information across many products that do not interact with each other, and a large operational investment to manage and maintain this complex infrastructure.
Operators most often use deep packet inspection (DPI) for traffic management, and a wide range of solutions for traffic security, including firewalls, intrusion detection systems (IDSs), security event managers (SEMs) and network behavior anomaly detection (NBAD).
Each of these solutions brings something novel and important from an operational perspective, either as a useful tool to better manage the traffic itself or as a fundamental security shield against an ever-growing number of threats. Although each of these products is needed to carry out a specific type of analysis and function, a system that leverages the strengths of each can dramatically improve operational efficiencies. A system that can correlate and analyze all the information captured and processed, interpret and cluster associated alerts, and manage the overall infrastructure as a whole (monitor, diagnose, act on the data collected from a large pool of such solutions) from a single console is even more powerful. We call this a real-time traffic intelligence system.
Converge! One Minute Videos
Characteristics of Real-Time Traffic Intelligence Systems
Real-time traffic intelligence is the ability to understand all IP traffic across the entire network, from the lowest layers in the network -- layer 2 -- to the application layer in the network -- layer 7. The system is designed to offer a series of fundamental operational values that ensure a secure, scalable and high-performance network. Firstly, it offers deep insight into the behavior of network protocols, applications and services from a network-wide perspective. With a real-time traffic intelligence system in place, the operator has the ability to understand which services, applications and even end users consume the most bandwidth, along with the performance metrics with which services are delivered. This function is typically provided by today's DPI products at a network link level. With a real-time traffic intelligence system, the operator is able to extend this knowledge to many links at the same time, thus gaining the global "network-wide" perspective.
The system also offers flexible normalization, scalable correlation and sophisticated statistical analysis of multi-typed information. It leverages the network infrastructure to provide the operator with 24/7 traffic monitoring and a prompt detection of traffic abnormalities. Such events are displayed with enriched records of information to enable the operator to carry out a thorough, easy and guided troubleshooting process.
A comprehensive real-time traffic intelligence system provides extensive forensic analysis of traffic abnormalities, facilitated by close interaction with the underlying network infrastructure. It enables the operator to understand the nature of the anomaly; the life-cycle of the anomaly; the impact of such anomaly to protocol, services and applications being delivered (in terms of QoS) and customers affected (in terms of service-level agreements, or SLAs); the packet-payload; and the data, all by providing a fast query engine and extensive reporting to organize and distill data as required.
Powerful contextualization of information for easy identification of the cause of the problem is essential to a real-time traffic intelligence system as well. Usually, a problem manifests itself in many different shapes and forms. One problem can generate tens or even hundreds of alerts, making the troubleshooting process time-consuming for the operational personnel. The real-time traffic intelligence system distills the vast amount of information, clusters alerts associated to the same problem and pinpoints the cause of the problem for the operator. The operator is then able to take prompt action against the cause of the problem, thus saving precious time and diminishing the negative impact of the problem to the network and the associated customer perception.
A comprehensive real-time traffic intelligence system offers the operator a complete view of the anomaly and provides a vast set of actions from which to choose. The system has an inherent ability to identify which actions can be executed on a given network element, which elements the operator should act on, and guides the operator as to what kind of actions to take.
The system is also able to scale depending on the size of the network. It has the ability to process large volumes of data captured from many network elements in real-time.
Finally, a real-time traffic intelligence system is highly modular, easy to manage to accommodate fast integration with third-party network infrastructure, and substantially cuts operational costs. It provides open south- and north-bound APIs to facilitate the collection and policy enforcement from and to a variety of different network elements.
Real-Time Traffic Intelligence Demystified: Breakdown of Sources
Operators must collect and analyze data from a wide variety of sources in order to keep their networks secure and operating efficiently, including packet and flow statistics, SNMP statistics, firewall/NAT/AAA events, routing and topology events, and IP-SLA metrics. Each source of data brings immense value to a real-time traffic intelligence system.
Telemetry and SNMP are two fundamental and rich sources of data for gaining a good understanding of the health of the traffic and network elements. They constitute the basic foundation of traffic intelligence. Telemetry from routers is a powerful source of information used today to gain a global view of the network activity at the Layer-4, or flow, level. Since operators can enable sampling, telemetry is the de-facto source of data used to monitor traffic activity across the entire network. The system that consumes telemetry data can provide the operators with details on the nature of the traffic flowing across the entire network and its overall composition. Only very recently, routers have been equipped with more powerful functionalities that go beyond the Layer-4 information. Indeed, such routers can export packet level records on demand for forensic analysis. SNMP statistics captured from routers and router interfaces enable a more accurate assessment of the impact of traffic abnormalities to network elements in terms of volume and element health.
Layer-7 data from DPI appliances is used for traffic management. It is indispensable for a very accurate breakdown of traffic into network protocols, services and applications. When Layer-7 information is collected from many links, correlated and analyzed in a central location, the operators gain a unique network-wide perspective into their services and applications. DPI appliances can be used as intelligent and targeted mitigation devices in case the operator is willing to take surgical actions on a per-packet basis.
Routing (BGP, IGP) and topology information is fundamental to understanding how packets traveled into the network and which network elements they have traversed. Operators can pinpoint the network element that caused the problem and act on it. Routing information is essential to monitor the stability of the routing infrastructure, and to detect router misconfiguration and threats targeting the overall routing infrastructure.
SLAs determine the type of service the provider guarantees, and the monetary impact when the service level is violated. A real-time traffic intelligence system can quantify the impact of security events on customers and service using the existing technology in network equipment.
Firewalls/NAT/AAA provide broad coverage of a variety of attacks either from legitimate sessions with unauthorized users or legitimate sessions that violate a customer's policies. A real-time traffic intelligence system combined with a firewall/NAT/AAA will provide additional visibility into end-host and user credentials, beyond public IP to private IP (NAT translation) and user credentials. The integrated solution will enable the operator to understand the threat impact and intent.
Contextualization of Information
A side-effect of the increasing complexity and size of today's networks is the increase in the volume of alerts associated with anomalous or malicious traffic. In fact, a single malicious anomaly can generate up to 40 individual alerts, all of which are related to the same cause. An effective real-time traffic intelligence system offers a way to group these related alerts into "meta-events" in an effort to slim down that mass of information into a manageable form. The individual alerts are still available, but with a real-time traffic intelligence system, NOC/SOC personnel can maximize resources by focusing on the associated cause. This same logic is also applied to BGP updates, which are commonly reported in terms of their volumes over time. With a real-time traffic intelligence system, BGP updates associated to the same cause of the problem are turned into "BGP events" that are addressed in groups and linked to the cause of the problem. As a result, the operator will have only tens of BGP events per day to review, rather than hundreds or thousands. At the same time, real-time traffic intelligence provides deep insight into the cause of those events, alerting the operator as to which of those changes might affect the normal operation of their network. Overall, the ability to summarize information while still allowing for drill-down capability ensures that security groups are efficient in their analysis and effective in their mitigation practices.
Measure Impact to Network Protocols, Services and Applications
A real-time traffic intelligence system must provide visibility into QoS metrics for SLA compliance. A real-time traffic intelligence system captures, creates and profiles IPSLA metrics used to monitor the correct behavior of network protocols, services and applications. Those metrics such as RTT, jitter, packet losses and Layer-7 SLA metrics specific to the most used protocols are generated either using information collected from DPI appliances or by a close interaction with network routers whose IOS supports such functionality. When any of the metrics being baselined violate a specific criteria being configured by the operator, an alert is triggered and detailed reports are displayed. Operators might decide to prioritize their tasks by using this metric as an example (since it is the one they measure as a source of revenue with their customers).
Real-Time Forensic Analysis
Forensic analysis is another key feature of a real-time traffic intelligence system. With cyberwarfare on the horizon, as indicated by a number of recent overseas and domestic network attacks, forensic analysis is critical to real-time traffic analysis. Operators must have a converged operational view across the network traffic, routing, topology, service and application behavior. The operator must access tabular and graphical reports before, during and after a problem has occurred and corrective action has been taken. A real-time traffic intelligence system allows an operator to dig deeper into an alert detected by close interaction with DPI boxes or routers that have seen the malicious stream. Raw flow and packet information can be captured and extensively analyzed by security personnel.
Suggest Where and How to Take an Action
As part of their duties, operators must be in a position to react to a problem quickly and precisely. This means that the real-time traffic intelligence system has to pinpoint to the operator which network elements have seen the anomaly being detected and suggest which network element to act on in order to resolve the problem promptly and with minimal network intrusion. It must monitor in real-time the effectiveness of the action put in place in the network and create reports for the operator. The real-time traffic intelligence system provides a vast pool of actions that can be taken, ranging from policy enforcement to specific appliances, or automated generation of ACL, or blackholing and sinkholing, or integration with third-party mitigation devices.
Scalability
The real-time traffic intelligence system is designed to meet scalability requirements of operators. It can incrementally scale to support changes in traffic volume, number of events and network coverage. While each collector can collect up to 150,000 events per second, the system can load balance across additional modules to collect and process traffic from as many points in the network as required. The real-time traffic intelligence system can process millions of events per second in real-time by using sophisticated load-balancing algorithms to spread the load across the available servers.
In Conclusion
Operators now regard the ability to see every bit of information traversing their network not only as a potential source of revenue, but also as a key differentiator in the market to deliver advanced services in the most reliable manner. As a consequence, from a service provider's perspective, efficient traffic management is imperative. In other words, the reliability and efficiency with which ISPs deliver content to their customers and the protection of every single bit of information are major differentiators that enable them to attract new customers and decrease operational cost. Using a real-time traffic intelligence system will make achieving this differentiator easily attainable.
About the Author
In his career, Antonio has published more than 30 papers for international conferences and professional journals, served on several Technical Program Committees (IEEE Infocom 2005/2006, ACM Sigmetrics 2005, IP-QoS 2005, LSNI 2005), and was referee for more than 130 papers from five different professional journals and 10 international conferences. He is a member of the IEEE where he was elected to the grade of Senior for contributions in the area of modeling and analysis of network performance.
Antonio received the Dr. Ing (B.S. and M.S.) and Ph.D. degrees in Electrical Engineering from Politecnico di Torino. He was also a scholar visitor at the Computer Science Department of the University of Montreal leading a project in optical networks.
About Narus
Narus is the leader in real-time traffic intelligence for large IP networks, and is the only company that provides security, intercept and traffic management solutions within a single, flexible system. With Narus, service providers, governments and large enterprises around the world can immediately detect, analyze, mitigate and target any unwanted, unwarranted or malicious traffic. Narus provides its customers with complete, real-time insight into all of their IP traffic from the network to the applications. Combined with the ability to enable numerous actions, Narus customers have the ability to take the most appropriate actions quickly. Narus' system protects and manages the largest IP networks around the world including AT&T, KT (Korea), KDDI (Japan), Telecom Egypt, Reliance (India), Saudi Telecom, US Cellular and Pakistan Telecom Authority. Narus is headquartered in Mountain View, California with regional offices around the world. For more information, please visit www.narus.com.
Send us your response to this article.
Learn How to Get Your Column Published on this Site
