"We propose a working group to define a basic set of monitoring event descriptions. These descriptions, or schemas, will describe the information (attributes) associated with a particular data element and will describe conventions for the representation of the value associated with it. The development of this sort of shared vocabulary is essential for interoperability among a growing number of testbeds and projects. This view is supported by the recent proliferation of independent efforts along these lines, both inside and outside the GGF.
The aim of this group is to develop standard representations of the most widely used measurement values (the "top N".) From this we envision the emergence of a set of conventions and recommendations that will ease the task of defining richer, domain-specific schemas. "
The Network Calculus book by Jean-Yves Le Boudec and Patrick Thiran is available for free download:
Folks,
Do we have a solid definition for the following terms? If so, I'd like to update the Glossary page:
1) Reflector
2) Beacon (is this identical to Reflector?)
3) Discard Server
Are there other terms we are beginning to use that we should add to the Glossary?
Tools based on hop-by-hop network analysis are increasingly critical to network troubleshooting on the rapidly growing Internet. The network characterization service (NCS) provides the ability to diagnose and troubleshoot networks hop-by-hop in an easy and timely fashion. Using NCS makes applications capable of fully using the high-speed networks, e.g., saturating 1Gbps local network from a single x86 platform
"netest-2 is the automated version of netest, which is for network problem analysis and diagnosis. netest has many options for analyzing different problems. However, it has too many options to use. In this revision 2, I have simplified it for all users with or without network knowledge. netest-2 can be run without parameters and do all testing automatically. The revision 2 also has all original (revision 1) features that can be used at command line.
"netest is an end-to-end network analysis tool. It needs to run on both ends of measured network path, one server and one or more clients. Generally, this test runs about 1 minute when the network condition is healthy. It may take longer if the network has some type of problems.
-=-=-
This is different from "nettest". netest (network test, although I like "network estimation") sends some test traffic, reports TCP and UDP throughputs, and reports if it feels that multiple TCP streams will help, or if there seems to be a TCP problem because the TCP throughput is radically different from the UDP throughput.
url above: http://www-didc.lbl.gov/NCS/netest-2.html
also see that there are a bunch of tools under the Network Characterization Service,
http://www-didc.lbl.gov/NCS/
[originally submitted by Matt, edited for technical reasons by George.]
Here is NetTest, the vendor.
NetTest is a leading worldwide provider of network testing, monitoring and management systems across both the optical and network layers.
Here is Nettest from LBNL
Nettest is a secure, real-time network monitoring utility. The nettest framework is designed to incorporate existing and new network tests, and be run as a daemon or an interactive process. Requests for network tests are received via a SSL connection or the user interface and are authorized using an ACL list (in the future authorization using Akenti will also be supported).
Here's the Self-Configuring Network Monitor that MattZ just referenced. [NB: at the bottom of the page is a report from Feb 2002]
The ultimate goal of this infrastructure is to provide accurate, comprehensive, and on-demand, application-to-application monitoring capabilities throughout the interior of the interconnecting network domains. In this project we are designing and implementing a self-configuring monitoring system that uses special request packets to automatically activate monitoring along the network path between communicating endpoints. Archived monitoring data will help point the way beyond the handcrafted systems of network testbeds to a production environment that can routinely support high performance distributed applications. This passive monitoring system will integrate with active monitoring efforts and provide an essential component in a complete end-to-end network test and monitoring capability. It will complement the existing network operation efforts.
Here is a beta of Pathprobe version 0.2.0 a MIB tool that uses TCP and web100 to probe and characterize the path between two hosts. The goal of this tool is to run hop-by-hop tests to determine if the paths along the way are capable of supporting the desired end-to-end target bandwidth between the sender and receiver. Please follow the links in the sections below for additional information on the methodology used by this tool in characterizing and debugging a network path.
Check the Net100 work at LBL
"The Data Intensive Distributed Computing Group at Berkeley Labs has been working with high speed networks for a number of years and has extensive experience in the integration of applications and networks. The net100 project brings together our knowledge of networks and our experiences with applications running on high speed networks. We are leveraging off the web100 project, which is providing the kernel instrumentation, for doing application level network optimizations. "
You'll also find "a modified version iperf (version 1.2) that has netlogger and web100 incorporated into it".
The document for "Application of Advanced Network Infrastructure Technology in Health and Disaster Management" RFP -- BAA RFP NLM 02-103/VMS is now available online in PDF.
Item 7 of Toru Ohira's list is a postscript version of "Phase transition in computer network traffic" (T. Ohira and R. Sawatari) Physical Review E, vol.58 (1998), pp.193-195. (SCSL-TR-98-009)
Here is the abstract of another article from Sole' et al. for "Information transfer and phase transitions in a model of internet traffic"
In a recent study, Ohira and Sawatari presented a simple model of computer network traffic dynamics. These authors showed that a phase transition point is present separating the low-traffic phase with no congestion from the congestion phase as the packet creation rate increases. We further investigated this model by relaxing the network topology using a random location of routers. It is shown that the model exhibits nontrivial scaling properties close to the critical point, which reproduce some of the observed real Internet features. At criticality, the net shows maximum information transfer and efficiency. It is shown that some of the key properties of this model are shared by highway traffic models, as previously conjectured by some authors. The relevance to Internet dynamics and to the performance of parallel arrays of processors is discussed.
Another article: On the Propagation of Long-Range Dependence in the Internet - Veres, Kenesi, ar, Vattay (ResearchIndex)
" This paper analyzes how TCP congestion control can propagate self-similarity between distant areas of the Internet. This property of TCP is due to its congestion control algorithm, which adapts to self-similar uctuations on several timescales. The mechanisms and limitations of this propagation are investigated, and it is demonstrated that if a TCP connection shares a bottleneck link with a self-similar background trac ow, it propagates the correlation structure of the background trac ow above a characteristic timescale. The cut-o timescale depends on the end-to-end path properties, e.g., round-trip time and average window size. It is also demonstrated that even short TCP connections can... "
Another article: Santa Fe Institute Working Paper Abstract: Self Organized Critical Traffic in Parallel Computer Networks
". . . In this paper we further extend our analysis by considering a generalization of the previous model in which the rate of packet emission is regulated by the local congestion perceived by each node. "
Valverde
Here is a Publications List from K. Fukuda which includes the 1999 paper, "Phase Transition Phenomena in Internet Traffic [PDF]
From Rich Carlsson the ANL Network tester Development Server applet
ANL Web100 based Network Configuration Tester
This java applet was developed to test the reliablity and operational status of your network link. It does this by sending data between your desktop computer and this remote Web server. This tester can determine the speed (10 or 100 Mbps) and duplex (full or half) settings of your network link. It can also determine if your performance is being limited due to network congestion (e.g., traffic between buildings). Finally 2 serious error conditions, Duplex Mismatch and Faulty cables, are looked for and reported if found. This java applet was originally developed by ORNL and extensively modified at ANL. Each test takes about 10 seconds. Click on "start" to start tests.
Yesterday I asked Chris Heermann if he knew of any visual tools. He suggested I have a look at AdventNet.
One of their products is AdventNet Web NMS Platform. I found this page to be very interesting. There are some good diagrams on down the page.
Does this page illustrate some of the processes we are talking about? Let me know. Thanks
I setup Internet2: E2Epi : Tool Presentation
as a page to capture publicly the weather maps, dashboards, and other visual network tools.
Industrial Networking Solutions sells the Industrial SNMP Suite v2.1. One component is the Network DashBoard a stand-alone graphical network management and monitoring interface that allows industrial network users to view their industrial Ethernet network without having to be connected to an HMI.
have a look at the TracePlus/Ethernet from SST. Scroll down the page to the image of the Realtime Network Dashboard
Here is a page titled "Network Dashboard Indicators" from Abilene Christian University. Not much text, but it must have the info these folks need.
By the way, I'm finding this information from Google Searches
Here's one way of using a visual model of a Dashboard with regard to public policy.
click on the thumbnail for a larger image.
Here's a series of Network Problem Cases from Les Cottrell. They might serve as models for what kind of info would we need to solve different problems.
Have a look at this document with the idea of the measurement dashboard / toolkit in mind. While much of this pointed at scientific stuff, I think it might serve as a framework for us.
". . . A new class of advanced network-based applications is emerging, distinguished from today's Web browsers and other standard Internet applications by a coordinated use of not only networks but also endsystem computers, data archives, various sensors, and advanced human computer interfaces. These applications require services not provided by today%u2019s Internet and Web: they need a "Grid" that both integrates new types of resource into the network fabric and provides enhanced "middleware" services such as quality of service within the fabric itself. "
From cflowd - Frequently Asked Questions - CAIDA : TOOLS : measurement : cflowd
"... The problem is that the data set grows large very fast. Since I'm looking at 8-10 million records a day, I'm figuring 1 month's worth of data will be about 40Gbytes. Although it seems that keeping the raw flow data will allow me to generate whatever reports I need, is there a more efficient approach? Can anyone recommend an economical hardware/ operating system/ database to use? (I've tried Flowscan, but it doesn't give me all that I need.) ..."
There are a number of responses... including one reference to Mark Fullmer's (Ohio ITEC) flowtools.
Have a look at the NCDM
The National Center for Data Mining (NCDM) at the University of Illinois at Chicago (UIC) was established in 1998 to serve as a national resource for high performance and distributed data mining. The Center sponsors:
> research projects
> standards
> testbeds
> outreach
From Friday's The Chronicle of Higher Ed Info Tech:
"Case Western Reserve University has begun a $25-million project to build a data, video, and voice network that will be 10 times as fast as most campus networks. The effort, which entails innovative financing arrangements as well as the latest network hardware, will give every computer on the university's campus a connection that moves a billion bits of data per second.
The university network already runs on fiber-optic cable -- some 5,000 miles of it -- so most of the $25-million will be spent on such components as routers, switches, and network cards. The infrastructure project, which could take as long as 18 months to complete, involves establishing 17,000 gigabit-Ethernet-over-fiber connections in 87 buildings across the university's 150-acre campus."