The contents of this legacy page are no longer maintained nor supported, and are made available only for historical purposes.

The 2002 and 2003 Macroscopic Internet Topology Data Kit (ITDK)

CAIDA's Macroscopic Internet Topology Data Kit (ITDK) contains data for researchers about connectivity, routing and latency gathered from a large cross-section of the global Internet.

This page introduces the first two ITDKs made available in 2002 and 2003. For the most recent ITDK datasets please see the updated Macroscopic Internet Topology Data Kit page.

Acceptable Use Agreement

Please read the terms of the CAIDA Acceptable Use Agreement (AUA) for Publicy Accessible Datasets below:

When referencing this data (as required by the AUA), please use:

  • CAIDA Macroscopic Internet Topology Data Kit (ITDK) #0204,
    provided to the Network Modeling and Simulation (NMS) community under DARPA grant N66001-01-1-8909.
    Cooperative Association for Internet Data Analysis, San Diego Supercomputer Center (SDSC), University of California, San Diego (UCSD), 2002.
  • CAIDA Macroscopic Internet Topology Data Kit (ITDK) #0304.
    Cooperative Association for Internet Data Analysis, San Diego Supercomputer Center (SDSC), University of California, San Diego (UCSD), 2003.
You are required to report your publications using this dataset to CAIDA.

ITDK data access

Access the publicly available 2002 and 2003 ITDK data.

skitter Monitors

The set of skitter monitorsused in the ITDKs varies depending on the kit.

  • Kit #0204: contains data from 21 monitors.
  • Kit #0304: contains data from 23 monitors.

Probe Lists

Over several years, CAIDA has developed and managed lists of hosts to probe using our lightweight traceroute tool skitter. CAIDA seeks to maximize address space coverage (1 probe per /24 in the IPv4 space) while remaining sensitive to do-not-probe requests from particular hosts. Two of CAIDA's major probe lists are 1) the IPv4 list, containing destinations spread over BGP-routed IP space and /24 blocks; and 2) the DNS list, containing destinations collected from queries to the DNS root server. There are several different sized versions of the IPv4 list to accomodate the varying capacity of the skitter monitors.

  • Kit #0204:
    • IPv4: 133k, 301k, or 825k addresses
    • DNS: 143k addresses
  • Kit #0304:
    • IPv4: 147k, 366k, or 865k addresses
    • DNS: 147k addresses (different than the 147k IPv4 addresses)

ITDK #0204 Directory Structure

There are several directories containing different categories of data. README files in these directories describe files and file formats in more detail. The master processing sequence resides in SKI/procSki.

  1. CAIDA's skitter-related Data
  2. DirectoryContents
    SKICAIDA's parsing scripts, C programs and command sequences for skitter data analysis,
    SKB0204measurement data from 21 skitter monitors aggregated on a daily basis.
    SKI0204merged datasets, including IP and AS graphs, their cores, distance and reachability statistics, and the scripts and commands used to generate them.
    IPC0204IP addresses with their counts, valid/private tags and all observed hop distances from each monitor. April 01 data is given as an example of parsing done using a script from SKI (above).
    LNK0204traceroute links, including placeholder nodes (non-replies and private addresses in the middle of a path).
    RTT0204per-destination and aggregated statistics about replies and RTTs.

  3. CAIDA's iffinder-related Data
  4. DirectoryContents
    IFF0204original iffinder file, links, connected components (sets of IP interfaces) and parsing scripts.

  5. U Oregon's RouteViews BGP tables
  6. DirectoryContents
    BGPparsing scripts, C programs and command sequences, (including a script for analyzing RouteViews tables, a program for converting IP addresses to prefixes, and scripts for abstracting data into BGP atoms..
    BGDOregon Route Views raw ("show ip bgp") tables for April 1, 15, and 16, 2002.
    BGD020401P35processed BGP tables, prefix tables with origin ASes and statistics, and BGP atoms [3] for April 1, 12:00 Route Views data (containing 36 full-size tables).
    BGD020415P36processed BGP tables, prefix tables with origin ASes and statistics and BGP atoms for April 15, 12:00 Route Views data (containing 36 full-size tables).

  7. Domain Names
  8. DirectoryContents
    SKI0204all DNS names found for IP addresses and for well-connected nodes (having an outdegree of 100 or more).

ITDK #0304 Directory Structure

There are several directories containing different categories of data. README files in these directories describe files and file formats in more detail.

  1. CAIDA's skitter-related Data
    DirectoryContents
    SKICAIDA's parsing scripts, C programs and command sequences for skitter data analysis.
    SKB0304traceroute data from 23 skitter monitors aggregated into daily files.
    SKI0304merged datasets, including IP and AS graphs, their cores, distance and reachability statistics, and the scripts and commands used to generate them.
    IPC0304statistics on IP addresses, including their frequency, validity, and observed hop distances from the monitor.
    LNK0304IP links, including links involving placeholder nodes (which represent non-replies and private addresses in the middle of a path).

  2. CAIDA's iffinder-related Data
    DirectoryContents
    IFF0304original iffinder output file

  3. U Oregon's RouteViews BGP tables
    DirectoryContents
    BGPBGP table analysis scripts
    BGD030508P39processed BGP tables, prefix tables with origin ASes and statistics, and BGP atoms for May 8, 12:00 PDT, Route Views data (containing 39 full-size tables).

  4. Domain Names
    DirectoryContents
    SKI0304all DNS names that could be found for all IP addresses included in this ITDK.

Bibliography

  1. A. Broido, kc claffy. Internet topology: Connectivity of IP graphs. SPIE Conference on Scalability and Traffic Control in IP Networks, Denver, August, 2001. https://catalog.caida.org/paper/2001_osd.

  2. B. Huffaker, A. Broido, kc claffy, M. Fomenkov, K. Keys, Y. Hyun, D. Moore. Skitter AS Internet Graph. CAIDA, April 2002. https://www.caida.org/projects/as-core/.

  3. A. Broido, kc claffy. Complexity of global routing policies. https://catalog.caida.org/paper/2001_cgr/.

Related Objects

See https://catalog.caida.org/dataset/skitter_itdk/ to explore related objects to this document in the CAIDA Resource Catalog.
Published
Last Modified