More Web Proxy on the site http://driver.im/

Article

Properties and prediction of flow statistics from sampled packet streams

Authors:

Mikkel ThorupAuthors Info & Claims

IMW '02: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment

Pages 159 - 171

https://doi.org/10.1145/637201.637225

Published: 06 November 2002 Publication History

Abstract

Many routers can generate and export statistics on flows of packets that traverse them. Increasingly, high end routers form flow statistics from only a sampled packet stream in order to manage resource consumption involved.This paper addresses three questions. Firstly: what are the downstream consequences for the measurement infrastructure? Long traffic flows will be split up if the time between sampled packets exceeds the flow timeout. Using packet header traces we show that flows generated by increasingly prevalent peer-to-peer applicalions are vulnerable to this effect.Secondly: can the volume of packet-sampled flow statistics be easily determined? We develop a simple model that predicts both the export rate of flow packet-sampled flow statistics and the number of active flows. It uses unsampled flow statistics---those commonly currently collected--as its data, i.e., it does not rely on having packet header traces available.Thirdly: what properties of the original traffic stream can be inferred from the packet sampled flow statistics? We show that as well as estimating total bytes and packets, one can also infer more detail, specifically the number and average length of flows in the unsampled traffic stream, even though some flows will have no packets sampled. We believe that this information is useful, both for understanding source traffic, e.g. the dependence of flow lengths on application type, and also monitoring changes in the composition of the traffic, e.g., a flood of short flows during a DoS attack. In all cases, we evaluate our approach using packet header traces gathered in backbone and campus networks.

References

[1]

J. Apisdorf, K. Claffy, K. Thompson, and R. Wilder, "OC3MON: Flexible, Affordable, High Performance Statistics Collection," For further information see http://www.nlanr.net/NA/Oc3mon

[2]

R. Cáceres, N.G. Duffield, A. Feldmann, J. Friedmann, A. Greenberg, R. Greer, T. Johnson, C. Kalmanek, B. Krishnamurthy, D. Lavelle, P.P. Mishra, K.K. Ramakrishnan, J. Rexford, F. True, and J.E. van der Merwe, "Measurement and Analysis of IP Network Usage and Behavior", IEEE Communications Magazine, vol. 38, no. 5, pp. 144--151, May 2000.

Digital Library

[3]

Cisco NetFlow; for further information see http://www.cisco.com/warp/public/732/netflow/index.html and http://www.cisco.com/univercd/cc/td/doc/product.software/ios120/120newfff120limit/120s/120s11/12s_sanf.htm

[4]

K.C. Claffy, H.-W. Braun, and G.C. Polyzos. "Parameterizable methodology for internet traffic flow profiling", IEEE Journal on Selected Areas in Communications, vol. 13, no. 8, pp. 1481--1494, October 1995.

Digital Library

[5]

K.C. Claffy, G.C. Polyzos, and H.-W. Braun. "Application of Sampling Methodologies to Network Traffic Characterization", Computer Communication Review, 23(4):194--203, October 1993, appeared in Proceedings ACM SIGCOMM'93, San Francisco, CA, September pp. 13--17, 1993.

Digital Library

[6]

D. Comer, "Internetworking with TCP/IP, Volume 1: Principles, Protocols, and Architecture", Third Edition, Prentice Hall, NJ, 1995.

Digital Library

[7]

N.G. Duffield and M. Grossglauser, "Trajectory Sampling for Direct Traffic Observation", IEEE/ACM Transactions on Networking, v. 9 no. 3 (June 2001) pp. 280--292. Abridged version appeared in Proc. ACM Sigcomm 2000, Computer Communications Review, vol. 30, no. 4, October 2000, pp. 271--282.

Digital Library

[8]

C. Estan and G. Varghese, "New Directions in Traffic Measurement and Accounting", ACM SIGCOMM Internet Measurement Workshop 2001, San Francisco, CA, November 1--2, 2001.

Digital Library

[9]

A. Feldmann, J. Rexford, and R. Cáceres, "Efficient Policies for Carrying Web Traffic over Flow-Switched Networks," IEEE/ACM Transactions on Networking, vol. 6, no.6, pp. 673--685, December 1998.

Digital Library

[10]

Inmon Corporation, "sFlow accuracy and billing", see: http://www.inmon.com/PDF/sFlowBilling.pdf.

[11]

"Internet Protocol Flow Information eXport" (IPFIX). IETF Working Group. See: http://net.doit.wisc.edu/ipfix/

[12]

P. L'Ecuyer, "Efficient and portable combined random number generators", Communications of the ACM, vol. 31, pp. 742--749 and 774, 1988.

Digital Library

[13]

V. Paxson, G. Ames, J. Mahdavi, M. Mathis, "Framework for IP Performance Metrics", RFC 2330, available from: ftp://ftp.isi.edu/in-notes/rfc2330.txt, May 1998.

Digital Library

[14]

J. Postel, "Transmission Control Protocol," RFC 793, September 1981.

[15]

Qosient, "Argus": http://www.qosient.com/argus/index.htm

[16]

Real Time Flow Measurement, see: http://www.auckland.ac.nz/net/Internet/rtfmL

[17]

Riverstone Networks, Inc., see: http://www.riverstonenet.com/technology/

[18]

M.J. Schervish, "Theory of Statistics", Springer, New York, 1995.

[19]

XACCT Technologies, Inc., see: http://www.xacct.com

Cited By

Fejrskov MVasilomanolakis EPedersen J(2022)A Study on the Use of 3rd Party DNS Resolvers for Malware Filtering or Censorship CircumventionICT Systems Security and Privacy Protection10.1007/978-3-031-06975-8_7(109-125)Online publication date: 3-Jun-2022
https://doi.org/10.1007/978-3-031-06975-8_7
Fejrskov MPedersen JVasilomanolakis E(2021)Using NetFlow to Measure the Impact of Deploying DNS-based BlacklistsSecurity and Privacy in Communication Networks10.1007/978-3-030-90019-9_24(476-496)Online publication date: 3-Nov-2021
https://doi.org/10.1007/978-3-030-90019-9_24
Qin TLiu ZWang PLi SGuan XGao L(2020)Symmetry Degree Measurement and its Applications to Anomaly DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2019.293373115(1040-1055)Online publication date: 2020
https://doi.org/10.1109/TIFS.2019.2933731
Show More Cited By

Index Terms

Properties and prediction of flow statistics from sampled packet streams

Recommendations

Estimating flow distributions from sampled flow statistics
SIGCOMM '03: Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications

Passive traffic measurement increasingly employs sampling at the packet level. Many high-end routers form flow statistics from a sampled substream of packets. Sampling is necessary in order to control the consumption of resources by the measurement ...
Estimating flow distributions from sampled flow statistics

Passive traffic measurement increasingly employs sampling at the packet level. Many high-end routers form flow statistics from a sampled substream of packets. Sampling controls the consumption of resources by the measurement operations. However, ...
Packet arrival and buffer statistics in a packet switching node
DATACOMM '73: Proceedings of the third ACM symposium on Data communications and Data networks: Analysis and design

Packet multiplexing or switching systems split incoming customer messages into fixed length packets. Within a packet switching system information is transferred packet by packet. In this paper a model for a packet switching node is investigated. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IMW '02: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment

November 2002

334 pages

ISBN:158113603X

DOI:10.1145/637201

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

IMW02

Sponsor:

SIGCOMM

IMW02: Internet Measurement Workshop

November 6 - 8, 2002

Marseille, France

Acceptance Rates

Overall Acceptance Rate 29 of 80 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

160
Total Citations
View Citations
964
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)9

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fejrskov MVasilomanolakis EPedersen J(2022)A Study on the Use of 3rd Party DNS Resolvers for Malware Filtering or Censorship CircumventionICT Systems Security and Privacy Protection10.1007/978-3-031-06975-8_7(109-125)Online publication date: 3-Jun-2022
https://doi.org/10.1007/978-3-031-06975-8_7
Fejrskov MPedersen JVasilomanolakis E(2021)Using NetFlow to Measure the Impact of Deploying DNS-based BlacklistsSecurity and Privacy in Communication Networks10.1007/978-3-030-90019-9_24(476-496)Online publication date: 3-Nov-2021
https://doi.org/10.1007/978-3-030-90019-9_24
Qin TLiu ZWang PLi SGuan XGao L(2020)Symmetry Degree Measurement and its Applications to Anomaly DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2019.293373115(1040-1055)Online publication date: 2020
https://doi.org/10.1109/TIFS.2019.2933731
Liang DHe Y(2020)Obfs4 Traffic Identification Based on Multiple-feature Fusion2020 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS)10.1109/ICPICS50287.2020.9202018(323-327)Online publication date: Jul-2020
https://doi.org/10.1109/ICPICS50287.2020.9202018
Vakilinia SElbiaze H(2019)Latency Control of ICN Enabled 5G NetworksJournal of Network and Systems Management10.1007/s10922-019-09497-wOnline publication date: 22-Apr-2019
https://doi.org/10.1007/s10922-019-09497-w
Wallschlager MGulenko ASchmidt FAcker AKao O(2018)Anomaly Detection for Black Box Services in Edge Clouds Using Packet Size Distribution2018 IEEE 7th International Conference on Cloud Networking (CloudNet)10.1109/CloudNet.2018.8549546(1-6)Online publication date: Oct-2018
https://doi.org/10.1109/CloudNet.2018.8549546
Sehery WClancy C(2017)Flow Optimization in Data Centers With Clos Networks in Support of Cloud ApplicationsIEEE Transactions on Network and Service Management10.1109/TNSM.2017.276132114:4(847-859)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1109/TNSM.2017.2761321
Zhang XQi Li Jianping Wu Jiahai Yang (2017)Generic and agile service function chain verification on cloud2017 IEEE/ACM 25th International Symposium on Quality of Service (IWQoS)10.1109/IWQoS.2017.7969150(1-10)Online publication date: Jun-2017
https://doi.org/10.1109/IWQoS.2017.7969150
Zhang YLiu YXuanzhe Liu Li Q(2017)Enabling accurate and efficient modeling-based CPU power estimation for smartphones2017 IEEE/ACM 25th International Symposium on Quality of Service (IWQoS)10.1109/IWQoS.2017.7969112(1-10)Online publication date: Jun-2017
https://doi.org/10.1109/IWQoS.2017.7969112
He JYang YWang XTan Z(2017)Adaptive traffic sampling for P2P botnet detectionInternational Journal of Network Management10.1002/nem.199227:5Online publication date: 4-Aug-2017
https://doi.org/10.1002/nem.1992
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents