Overview

Scamper is designed to actively probe destinations in the Internet in parallel (at a specified packets-per-second rate) so that bulk data can be collected in a timely fashion. Scamper’s native output file format is called warts: a warts file contains substantial meta data surrounding each individual measurement conducted, as well as substantial detail of responses received. The measurements conducted can range from simple to complex. An example of a simple measurement is where a single measurement method (e.g. traceroute) is used on a list of IP addresses to conduct a bulk measurement. A more complex measurement might be where the outcome of a previous test influences what happens next: for example, for each hop in a traceroute path, infer the address of the outgoing interface for the previous hop. Complex measurements are conducted by connecting to a running scamper process with a driver program which contains the logic. The driver program can use the Python API provided by the scamper python module, or the low-level C APIs provided by libscamperctrl and libscamperfile.

Scamper Availability

• The current snapshot of scamper’s source code is cvs-20241112, released November 12th 2024.
• All releases of scamper are licensed under the GPL v2.
• Scamper is available in FreeBSD ports, OpenBSD ports, an Ubuntu PPA, a Brew formula, an openSUSE package, NetBSD pkgsrc, Macports, and Debian/Ubuntu packages. The FreeBSD, OpenBSD, Ubuntu PPA, and openSUSE packages should be up to date with the latest version of scamper.
• To use the Ubuntu PPA, add the PPA to your system as described in the PPA documentation. The PPA provides different scamper components in different packages, named as follows: scamper, python3-scamper, scamper-utils, scamper-remoted, scamper-hoiho, libscamperfile9, libscamperfile9-dev, libscamperctrl2, and libscamperctrl2-dev.

Building Scamper

• We recommend that you use one of the available packages listed above, where possible.

• Scamper should compile and run under FreeBSD, OpenBSD, NetBSD, Linux, MacOS X, Solaris, Windows, and DragonFly. Not all of scamper will run on all systems: for example, the sting and tbit modules require IPFW or PF. To build scamper:

  ./configure
make
make install

• The scamper source distribution includes many optional features that can be controlled at build time. You can obtain an overview of these optional features by reviewing the output of ./configure --help.

• The Python interface requires Python 3 to build. Depending on your compilation environment, you might need to provide the path to a Python binary to the configure script, using something similar to PYTHON=/path/to/python3 ./configure --with-python.

• To build sc_hoiho, pass --enable-sc_hoiho and one of --with-pcre or --with-pcre2 to configure. You might need to pass the path to the pcre headers and library via CFLAGS and LDFLAGS if these are not in a location that the compiler will consider by default.

• To build sc_minrtt, pass --enable-sc_minrtt and one of --with-pcre or --with-pcre2 to configure, and ensure that the sqlite3 headers and library are in a location that the compiler will consider, via CFLAGS and LDFLAGS. You also might need to pass the path to the pcre headers and library via CFLAGS and LDFLAGS if these are not in a location that the compiler will consider by default.

• To build sc_uptime, pass --enable-sc_uptime to configure, and ensure that the sqlite3 headers and library are in a location that the compiler will consider, via CFLAGS and LDFLAGS.

• You can also disable measurement primitives in the compiled scamper binary (but not libscamperfile) by using --disable-scamper-foo, where foo is the name of a scamper measurement primitive to disable.

Usage

usage: scamper [-?Dv] [-c command] [-p pps] [-w window]
               [-M monitorname] [-l listname] [-L listid] [-C cycleid]
               [-o outfile] [-O options] [-F firewall] [-e pidfile]
               [-n nameserver]
               [-d debugfile]
               [-i IPs | -I cmds | -f file | -P [ip:]port | -R name:port |
                -U unix-dom]
            -? give an overview of the usage of scamper
            -c command string (default: trace)
            -C cycle id
            -d write debugging information to the specified file
            -D start as a daemon listening for commands on a port
            -e write process ID to specified file
            -f list of files provided on the command line
            -F use the system firewall to install rules as necessary
            -i list of IP addresses provided on the command line
            -I list of scamper commands provided on the command line
            -l name to assign to default list
            -L list id for default list
            -M specify the canonical name of the monitor
            -o specify the file to write output to
            -O specify options to use:
               text: output results in plain text for interactive use
               warts: output results in warts format for science
               warts.gz: output results in gzipped warts format
               warts.bz2: output results in bzip2 warts format
               warts.xz: output results in xz warts format
               cmdfile: input file specifies whole commands
               json: output results in json format, better to use warts
               planetlab: necessary to use safe raw sockets on planetlab
               noinitndc: do not initialise neighbour discovery cache
               outcopy: output copy of all results collected to file
               rawtcp: use raw socket to send IPv4 TCP probes
               icmp-rxerr: use recverr cmsg to receive ICMP responses
               notls: do not use TLS anywhere in scamper
               notls-remote: do not use TLS on remote control sockets
               cafile=file: use the CA certs in file for remote auth
               client-certfile=file: use cert in file for remote auth
               client-privfile=file: use privkey in file for remote auth
               select: use select(2)
               poll: use poll(2)
               epoll: use epoll(7)
               kqueue: use kqueue(2)
               debugfileappend: append to debugfile, rather than truncate
               ring: use PACKET_RX_RING to receive datalink packets
               dl-any: open a cooked any datalink interface
               dyn-filter: use dynamic BPF filters on datalink interfaces
            -p number of packets per second to send (1 <= pps <= 10000)
            -P [ip:]port for control socket, default to loopback
            -R name and port of remote host to receive commands from
            -U name of control socket in the file system
            -v output the version of scamper this binary is
            -w limit the window of actively probing tasks

Documentation

The paper Scamper: a Scalable and Extensible Packet Prober for Active Measurement of the Internet describes scamper’s motivation and architecture. The cite for the paper is:

M. Luckie. Scamper: a Scalable and Extensible Packet Prober for Active Measurement of the Internet.
Proceedings of the 10th ACM SIGCOMM conference on Internet measurement (IMC), Melbourne, Australia, 1-3 Nov 2010, p. 239-245.

The Python module is documented in web pages provided on the CAIDA website, found in the scamper python module documentation.

Most of scamper, and its associated libraries and utilities, is documented in man pages included in the source code package. PDF files of these man pages are:

• scamper: main data collection program.
• sc_ally: driver for Ally implementation.
• sc_analysis_dump: convert scamper traces to something easily parsed.
• sc_attach: connect to scamper daemon and execute series of commands, collecting the results in a warts file.
• sc_bdrmap: driver to map first hop border routers of networks.
• sc_erosprober: driver to periodically probe addresses and rotate output files.
• sc_filterpolicy: driver to test systems for congruent filtering policy.
• sc_hoiho: holistic orthography of Internet hostname observations.
• sc_ipiddump: dump IP-ID values embedded in ping and dealias objects in warts files.
• sc_minrtt: manage RTT input to hoiho
• sc_pinger: driver to run ping with different probe methods on a list of addresses.
• sc_prefixprober: driver to probe addresses in specified prefixes.
• sc_prefixscan: driver to test if a set of IPv4 links are point-to-point.
• sc_radargun: driver to run radargun on a list of candidate aliases.
• sc_remoted: interact with a collection of remotely controlled scamper instances.
• sc_speedtrap: driver to resolve aliases for a set of IPv6 interfaces.
• sc_tbitblind: driver to test systems for resilience to blind TCP attacks.
• sc_tbitpmtud: driver to test systems for responsiveness to ICMP packet too big messages.
• sc_tracediff: display traceroute paths that have changed.
• sc_ttlexp: dump source addresses from ICMP TTL expired messages in warts files.
• sc_uptime: driver to infer reboot windows for systems with IPv6 addresses.
• sc_warts2csv: dump traceroutes collected by scamper in csv format.
• sc_warts2json: print a JSON object representing each warts object in a file.
• sc_warts2pcap: generate pcap files from tbit and sting data.
• sc_warts2text: generate simple text for human parsing.
• sc_wartscat: concatenate warts files.
• sc_wartsdump: detailed dump of scamper data that was collected in warts format.
• sc_wartsfilter: select specific records from a warts file.
• sc_wartsfix: truncate damaged warts files.
• libscamperctrl: a library to execute measurements on a set of scamper instances.
• libscamperfile: a library to read and write warts files, as well as read CAIDA’s ARTS files.
• warts: documentation on the internal format of a warts file.

Release Announcements

If you would like to receive notifications of future releases of scamper, you may subscribe to the receive only mailing list scamper-announce by filling out the scamper-announce mailman form.

Network Research

Here is a list of research done by the author using scamper.

• On the Latency Impact of Remote Peering
  F. Mazzola, P. Marcos, I. Castro, M. Luckie, and M. Barcellos
  Proceedings of the 23rd Passive and Active Measurement (PAM) Conference, March 2022.
• Learning Regexes to Extract Router Names from Hostnames
  M. Luckie, A. Marder, B. Huffaker, and k. claffy
  Proceedings of the Asian Internet Engineering Conference (AINTEC), December 2021.
• Learning to Extract Geographic Information from Internet Router Hostnames
  M. Luckie, B. Huffaker, A. Marder, Z. Bischof, M. Fletcher, and k. claffy
  Proceedings of the 17th ACM Conference on emerging Networking EXperiments and Technologies (CoNEXT), December 2021.
• Inferring Regional Access Network Topologies: Methods and Applications
  Z. Zhang, A. Marder, R. Mok, B. Huffaker, M Luckie, k. claffy, and A. Schulman
  Proceedings of the 21st ACM Internet measurement Conference (IMC), November 2021.
• Learning to Extract and Use ASNs in Hostnames
  M. Luckie, A. Marder, M. Fletcher, B. Huffaker, and k. claffy
  Proceedings of the 20th ACM Internet measurement Conference (IMC), October 2020.
• Learning to Extract Router Names from Hostnames
  M. Luckie, B. Huffaker, and k. claffy
  Proceedings of the 19th ACM Internet measurement Conference (IMC), October 2019.
• The Impact of Router Outages on the AS-level Internet
  M. Luckie and R. Beverly
  Proceedings of ACM SIGCOMM, August 2017
• bdrmap: Inference of Borders Between IP Networks
  M. Luckie, A. Dhamdhere, B. Huffaker, D. Clark, and k. claffy
  Proceedings of the 16th ACM SIGCOMM conference on Internet Measurement (IMC), November 2016.
• Don’t Forget to Lock the Back Door! A Characterization of IPv6 Network Security Policy
  J. Czyz, M. Luckie, M. Allman, M. Bailey
  Proceedings of Network and Distributed Systems Security (NDSS) Conference, February 2016.
• Resilience of Deployed TCP to Blind Attacks
  M. Luckie, R. Beverly, T. Wu, M. Allman, k claffy
  Proceedings of the 15th ACM SIGCOMM conference on Internet measurement (IMC), October 2015.
• Measuring and Characterizing IPv6 Router Availability
  R. Beverly, M. Luckie, L. Mosley, k claffy
  Proceedings of the 16th Passive and Active Measurement (PAM 2015) Conference, New York, March 2015.
• Challenges in Inferring Internet Interdomain Congestion
  M. Luckie, A. Dhamdhere, D. Clark, B. Huffaker, k claffy
  Proceedings of the 14th ACM SIGCOMM conference on Internet measurement (IMC), November 2014.
• Speedtrap: Internet-scale IPv6 Alias Resolution
  M. Luckie, R. Beverly, W. Brinkmeyer, and k claffy
  Proceedings of the 13th ACM SIGCOMM Internet Measurement Conference (IMC), October 2013.
• Measuring the Deployment of IPv6: Topology, Routing and Performance
  A. Dhamdhere, M. Luckie, B. Huffaker, k.c. claffy, A. Elmokashfi, E. Aben
  Proceedings of the 12th ACM SIGCOMM Internet Measurement Conference (IMC), November 2012.
• Revealing MPLS tunnels obscured from traceroute
  B. Donnet, M. Luckie, P. Merindol, J-J Pansiot
  ACM SIGCOMM Computer Communication Review, 42 (2), pp. 87-93, April 2012.
• Measured Impact of Crooked Traceroute
  M. Luckie, A. Dhamdhere, k.c. claffy, D. Murrell
  ACM SIGCOMM Computer Communication Review, 41 (1), pp. 14-21, January 2011.
• Measuring Path MTU Discovery Behaviour
  M. Luckie and B. Stasiewicz
  Proceedings of the 10th ACM SIGCOMM Internet Measurement Conference (IMC), Melbourne, Australia, pp. 102-108, November 2010
• Traceroute probe method and forward IP path inference
  M. Luckie, Y. Hyun, and B. Huffaker
  Proceedings of the 8th ACM SIGCOMM Internet Measurement Conference (IMC), Vouliagmeni, Greece, pp. 311-324, October 2008
• Inferring and Debugging Path MTU Discovery Failures
  M. Luckie, K. Cho, and B. Owens
  Proceedings of the 5th ACM SIGCOMM Internet Measurement Conference (IMC), pp. 193-198, October 2005
• Identifying IPv6 Network Problems in the Dual-Stack World
  K. Cho, M. Luckie, and B. Huffaker
  Proceedings of ACM/SIGCOMM Network Troubleshooting Workshop 2004, pp. 283-288, August 2004

Bonus Material

• A library written by Young Hyun at CAIDA for reading warts files with Ruby is available at RubyForge.
• Robert Beverly (NPS) wrote native Python implementations of sc_attach and sc_wartsdump without any scamper dependencies. His code is available at CMAND’s GitHub.
• Baptiste Jonglez wrote a native Python warts library for traceroute and ping records, also without any scamper dependencies. His code is available at the Drakkar group’s GitHub.
• Ovidiu Dan wrote a C# library for parsing traceroute output stored in warts files. His code is available on GitHub.

Acknowledgments

The development of scamper was funded by the WIDE Project in association with CAIDA for the 12 month period ending April 2005.