An Extensible Toolkit for
Resource Prediction in
Distributed Systems

Peter A. Dinda
Prescience Lab
Department of Computer Science
Northwestern University

RPS is an extensible toolkit for designing, building, and evaluating systems that predict the dynamic behavior of resources in distributed systems. The core of RPS is written in C++ and is portable to any platform with a relatively recent C++ compiler.

Components

RPS includes the following components.
  • Libraries and sensors for measuring host load, network flow bandwidth, Linux /proc data, and Windows performance counters.
  • A library and tools for analyzing and predicting time series data. Supported models include MEAN, LAST, BESTMEAN, BESTMEDIAN, NEWTON, AR, MA, ARMA, ARIMA, fractional ARIMA, Wavelet-based models, and others. Any model can wrapped with a management template, yielding a non-linear model (e.g., AR to TAR).
  • A library and tools for wavelet-based analysis and reconstruction of time series data. A huge variety of configurations is possible.
  • A template library for communication that makes it easy to build prediction components.
  • A set of prediction components that can be connected and configured at run-time to form online prediction and wavelet analysis systems.
  • Scripts for connecting prediction components together in common ways.
  • Scripts for turning any program that produces a text-based numeric output into an RPS measurement, prediction, or wavelet component.
  • Graphical display tools
  • A simple Java-based tool for displaying measurements and predictions..
  • A library and tools for predicting task running times using RPS-based host load predictions.
  • A library and tools for scheduling soft real-time tasks using RPS-based host load predictions.
  • Libraries and tools for accessing trace data.
  • A web interface.
  • Documentation.
    • You may be interested in some screen shots or a movie (AVI, 98 MB) of a web front-end built on top of RPS.

    Documentation

    The design of RPS is described in the following journal paper:
  • P. Dinda, Design, Implementation, and Performance of an Extensible Toolkit for Resource Prediction In Distributed Systems, IEEE Transactions on Parallel and Distributed Systems, Volume 17, Number 2, February, 2006. pdf
    • with additional detail provided in this older technical report:
  • P. Dinda, D. O'Hallaron, An Extensible Toolkit for Resource Prediction In Distributed Systems, Technical Report CMU-CS-99-138, School of Computer Science, Carnegie Mellon University, July, 1999. abstract, postscript, pdf.
    • The design of Tsunami, RPS's wavelet toolkit is described in the following technical report.
  • J. Skicewicz, P. Dinda, Tsunami: A Wavelet Toolkit for Distributed Systems, Technical Report NWU-CS-03-16, Department of Computer Science, Northwestern University, November, 2003. pdf.
    • The documentation tree that ships with RPS can be viewed here.

    A large number of research results have grown out of using RPS. These are described in papers available from the Prescience Lab and here.

    Download

    The current release is 2.0. It is available in binary form for Linux, Cygwin, and Windows environments, and in source code form that has been tested on Red Hat, Cygwin, and Windows environments on Intel processors, specifically, Red Hat 9, Cygwin 1.5.7, and Windows XP. The build system also includes provisions for build on Solaris/Sparc and on Digital Unix/Alpha. The system should compile straightforwardly on any environment that provides GCC.

    RPS is subject to a LICENSE.

  • Source code: RPS-2.0.tgz (3.5 MB)
    tar xvfz RPS-2.0.tgz
    Look at RPS-2.0/README and RPS-2.0/doc/BUILD

  • Linux installation: RPS-2.0-installed-I386-LINUX.tgz (27 MB)
    To use:
    tar xvfz RPS-2.0-installed-I386-LINUX.tgz
    edit RPS-2.0/ENV to reflect your installation directory
    source RPS-2.0/ENV
    read RPS-2.0/README and RPS-2.0/doc/RUN

  • Cygwin installation: RPS-2.0-installed-I386-CYGWIN.tgz (32 MB)
    To use:
    tar xvfz RPS-2.0-installed-I386-CYGWIN.tgz
    edit RPS-2.0/ENV to reflect your installation directory
    source RPS-2.0/ENV
    read RPS-2.0/README and RPS-2.0/doc/RUN

  • Windows installation: RPS-2.0-installed-I386-WIN32.zip (21 MB)
    To use:
    unzip RPS-2.0-installed-I386-WIN32.zip
    edit RPS\ENV.bat to reflect your installation directory
    RPS\ENV.bat
    read RPS\README and RPS\doc\RUN

    •  The previous release is RPS-1.0-rc2 (gzipped tar file, 2 MB). It is only distributed in source form. It has been verified to compile on Red Hat Linux 6.2 and 7.0, Digital Unix 4.0D, Solaris, and Cygwin using gnu make and gcc 2.95.2 or better, and on Windows 2000 using Visual Studio 6.0.

    An early version of RPS was as a part of the CMU Remos system, which can be downloaded from here. The directory tree is "PPS".

    Related Software

    You may also be interested in the following software that can be used with RPS.
  • Diffusion: Zero cost dissemination of resource information
  • Playload: Host load trace playback (and a trace library)

    • KNOWN BUGS

    Look here.

    Getting Help

    RPS comes with no warranty or support of any kind. Also keep in mind that it is a research tool. Nonetheless, the following informal support options exist.
  • Documentation and papers as described above.

  • Mailing list rps-list@cs.northwestern.edu. To subscribe, send mail to majordomo@cs.northwestern.edu with the text "subscribe rps-list [your email address]" in the body.

  • rps-help@cs.northwestern.edu: A help address of last resort that is answered only as time permits.
    • Please do not send help mail directly to Peter Dinda or his students. It will probably go unanswered.

    Contributors

  • Peter Dinda
  • Jason Skicewicz
  • Alex Shoykhet
  • Luka Spoljaric
  • Mike Knop
  • Brandon O'Bryant
    • Peter A Dinda
    Last modified: Mon Oct 31 13:37:09 CDT 2011