Printable version of this document

Emulab Tutorial - XMLRPC Interface to Emulab

This page describes the XMLRPC interface to Emulab. Currently, the interface mainly supports experiment creation, modification, swapping, and termination. We also provide interfaces to several other common operations on nodes end experiments such as rebooting, reloading, link delay configuration, etc. This interface is a work in progress; it will improve and grow over time. If there is something missing you need, please send us email.

The Emulab XMLRPC server can be accessed via an SSL based server. You need to request a certificate from the Emulab website in order to use the SSL based server. Go to the "My Emulab" portion of the Emulab website, and click on the "Create SSL Certificate" link. You will be prompted for a passphrase to use to encrypt the private key. Once the key has been created, you will be given a link to download a text version (PEM format). Simply provide this certificate as an input to your SSL client.

The API is described in detail below. A demonstration client written in Python is also available that you can use on your desktop to invoke commands from the shell. For example:

    sslxmlrpc_client.py startexp batch=false wait=true proj="myproj" exp="myexp" nsfilestr="`cat ~/nsfile.ns`"

which says to create an experiment called "myexp" in the "myproj" project, swap it in immediately, wait for the exit status (instead of running asynchronously), passing inline the contents of nsfile.ns in your home directory on your desktop. By default, the client will contact the RPC server at boss.emulab.net, but you can override that by using the -s hostname option. For example:

    sslxmlrpc_client.py -s boss.emulab.net startexp ...

which would invoke the RPC server on boss.emulab.net. You will be prompted for your SSL passphrase.

The sslxmlrpc_client python program is a simple demonstration of how to use Emulab's RPC server. If you do not provide a method and arguments on the command line, it will enter a command loop where you can type in commands (method and arguments) and wait for responses from the server. It converts your command lines into RPCs to the server, and prints out the results that the server sends back (exiting with whatever status code the server returned).f You can use this client program as is, or you can write your own client program in whatever language you like. Remember to use the --cert= option to tell the demonstration client where to find your certificate. You will be prompted for the private key passphrase when it attempts to contact the server.

A more sophisticated client called script_wrapper.py provides a traditional command line interface to Emulab, using the SSL transport described above. The client should run on any machine that has Python installed. For example, to swap the myexp experiment in the testbed project in:

    script_wrapper.py --server=boss.emulab.net swapexp -e testbed,myexp in

Each command has its own --help option:

    script_wrapper.py swapexp --help
    
    swapexp -e pid,eid in|out
    swapexp pid eid in|out
    where:
         -w   - Wait for experiment to finish swapping
         -e   - Project and Experiment ID
         in   - Swap experiment in  (must currently be swapped out)
        out   - Swap experiment out (must currently be swapped in)

A complete list of Emulab commands available via the script wrapper is available with the --help command:

    script_wrapper.py --help
    Usage: wrapper [wrapper options] command [command args and opts]
    
    Commands:
        readycount   Get readycounts for nodes in experiment (deprecated).
        startexp     Start an Emulab experiment.
        savelogs     Save console tip logs to experiment directory.
        endexp       Terminate an experiment.
        eventsys_control Start/Stop/Restart the event system.
        batchexp     Synonym for startexp.
        node_list    Print physical mapping of nodes in an experiment.
        expinfo      Get information about an experiment.
        node_admin   Boot selected nodes into FreeBSD MFS.
        create_image Create a disk image from a node.
        delay_config Change the link shaping characteristics for a link or lan.
        modexp       Modify experiment.
        nscheck      Check and NS file for parser errors.
        swapexp      Swap experiment in or out.
        os_load      Reload disks on selected nodes or all nodes.
        portstats    Get portstats from the switches.
        link_config  Change interface parameters for a wireless link.
        node_reboot  Reboot selected nodes or all nodes in an experiment.

As a convenience, you can symlink script_wrapper.py to each of the commands listed above, and avoid having to type the wrapper name:

    ln -s script_wrapper.py swapexp

    swapexp --server=boss.emulab.net -e testbed,myexp in

This has already been done in users.emulab.net:/usr/testbed/bin, so each of the above commands is already on your path.

Server API

The API for the server is broken into several different modules that export a number of methods, each of which is described below. Each method is of the form (in Python speak):

    def startexp(version, arguments):
        return EmulabResponse(RESPONSE_SUCCESS, value=0, output="Congratulations")

The arguments to each method:

• version: a numeric argument that the server uses to determine if the client is really capable of speaking to the server.
• arguments: a hash table of argument/value pairs, which in Python is a Dictionary. In Perl or PHP this would be a hashed array. Any client that supports such a datatype will be able to use this interface directly. For example, to swap out an experiment a client might:

      args = {};
      args["proj"] = "myproj"
      args["exp"] = "myexp"
      args["direction"]  = "out"
      response = server.swapexp(CURRENTVERSION, args)

The client specifies the proj and exp of the experiment he/she wants to swap, as well as the actual swap operation, in this case out. The response from the server is another hashed array (Python Dictionary) of the form:

• code: An integer code as defined in emulabclient.py.
• value: A return value. May be any valid data type that can be transfered in XML.
• output: A string (with embedded newlines) to print out. This is useful for debugging and for guiding users through the perils of XMLRPC programming.

Unless specifically stated, the return value of most commands is a simple integer reflecting an exit code from the server, and some output to help you determine what went wrong. Otherwise, the return value is documented in each method description.

Finally, a quick note about the types accepted and returned by methods. Most methods will accept a real XML-RPC type and try to coerce a string into that type. For example, in python, passing True is equivalent to passing the string, "true". When returning data, the methods will prefer to return typed values, rather than formatted strings.

• /XMLRPC/emulab

  The emulab module provides general information about this Emulab installation.

  • news: Get news item headers. The optional arguments are:
    
    

    Name	Type	Default	Description
    starting	date	-Inf.	The date to start searching for news items (e.g. "2003-07-23 10:13:00").
    ending	date	+Inf.	The date to stop searching for news items.

    
    The return value is a list of hash tables with the following elements:
    
    

    Name	Type	Description
    subject	string	The item's subject.
    author	string	The item's author.
    date	date	The date the item was posted.
    msgid	integer	The item's unique identifier. This value should be used as the anchor when redirecting to the web site (e.g. http://www.emulab.net/news.php3#32).

• /XMLRPC/user

  The user module provides access to user-specific information.

  • nodecount: Get the number of nodes you have allocated. There are no arguments and the method returns an integer.
    
    
  • membership: Get the list of projects and subgroups of which you are a member. The return value is a hash table where each entry is the name of a project and a list of subgroups. The optional arguments are:
    
    

    Name	Type	Default	Description
    permission	string	readinfo	The name of an action that the user would like to take. The result will then be narrowed to those groups where this action is possible. Supported values: readinfo - Read information about the project/group. createexpt - Create an experiment. makegroup - Create a sub-group. makeosid - Create an OS identifier. makeimageid - Create an image identifier.

• /XMLRPC/fs

  The fs module lets you examine the parts of the Emulab file system that can be exported to your experimental nodes.

  • access: Check the accessibility of a path. The path must lie on an exported file system or the call will fail. The return value is true or false. The required arguments are:
    
    

    Name	Type	Description
    path	string	The path to check
    permission	string	The access permission to check. Value should be one of: "read", "write", "execute", "exists".

    
    
  • listdir: Get a directory list for a given path. The path must lie on an exported file system of the call will fail. The return value is a list of tuples for each entry, described in detail below. The required arguments are:
    
    

    Name	Type	Description
    path	string	The path to check

    
    The return value is a list of tuples where each entry is organized as follows:
    
    

    Index	Type	Description
    0	string	The name of the entry.
    1	character	The entry type, where: 'd' - directory 'c' - character device 'b' - block device 'f' - regular file 'l' - link 's' - socket 'u' - unknown
    2	integer	Unix permission mask.
    3	string	Name of the user owner.
    4	string	Name of the group owner.
    5	integer	The size of the file.
    6	integer	The last access time as the number of seconds since the epoch.
    7	integer	The last modified time as the number of seconds since the epoch.
    8	integer	The creation time as the number of seconds since the epoch.

    
    
  • exports: Get the root list of file system exports available to you. This method takes no arguments and returns a list of strings that represent the set of paths that are potentially accessible by your experimental nodes.
    
    Note 1: The list does not include other user directories even though they may be accessible.
    
    Note 2: The list does include all project and group directories that you are a member of, even though the nodes will only be able to access the project/group directories the experiment was created under.

• /XMLRPC/experiment

  The experiment module lets you start, control, and terminate experiments.

  • constraints: Get the physical/policy constraints for experiment parameters. There are no arguments and the return value is a hash table with the following elements:
    
    

    Name	Type	Description
    idle/threshold	integer	The maximum number of hours allowed for the idleswap parameter.

    
    
  • getlist: Get the list of experiments created by a user. There are no required arguments and the optional arguments are:
    
    

    Name	Type	Default	Description
    format	string	"brief"	The format of the data to return. The "brief" format contains a nested mapping of project names to group names and group names to a list of experiment names. The "full" format expands the experiment listing to include structures containing extra information about each experiment, instead of just the name.

    
    
  • startexp: Create an experiment. By default, the experiment is started as a batch experiment, but you can use the batchmode option described below to alter that behavior. You can pass an NS file inline, or you can give the path of a file already on the Emulab fileserver.
    
    The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which to create the experiment
    exp	string	The unique ID to call the experiment
    nsfilestr	string	A string representing the NS file to use, with embedded newlines, or,
    nsfilepath	string	The pathname of a NS file on the Emulab file server, within the project directory (example: /proj/myproj/foo.ns)

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    group	string	proj	The Emulab subgroup ID in which to create the experiment (defaults to project id)
    batch	boolean	true	Create a batch experiment. Value is either "true" or "false"
    description	string		A pithy sentence describing your experiment
    swappable	boolean	true	Experiment may be swapped at any time. If false, you must provide a reason in noswap_reason
    noswap_reason	string		A sentence describing why your experiment cannot be swapped
    idleswap	integer	variable	How long (in minutes) before your idle experiment can be idle swapped. Defaults to a value between two and four hours. A value of zero means never idleswap (you must provide a reason in noidleswap_reason)
    noidleswap_reason	string		A sentence describing why your experiment cannot be idle swapped
    max_duration	integer	0	How long (in minutes) before your experiment should be unconditionally swapped. A value of zero means never unconditionally swap this experiment.
    noswapin	boolean	false	If true, do not swap the experiment in immediately; just "preload" the NS file. The experiment can be swapped in later with swapexp.
    wait	boolean	false	If true, wait synchronously for the experiment to finish swapping. By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded

    
    
  • metadata: Get any metadata associated with the given experiment. The return value is a hashtable containing fields such as the experiment's description and creation date.
    
    The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID of the experiment
    exp	string	The Emulab experiment ID

    
    
  • nsfile: Get the NS file associated with the given experiment.
    
    The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID of the experiment
    exp	string	The Emulab experiment ID

    
    
  • swapexp: Swap an experiment in or out. The experiment must, of course, be in the proper state for requested operation.
    
    The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID of the experiment
    exp	string	The Emulab experiment ID
    direction	string	The direction in which to swap; one of "in" or "out"

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for the experiment to finish swapping in (or preloading if noswapin is true). By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded

    
    
  • modify: Modify an experiment, either while it is swapped in or out. You must provide an NS file to direct the modification.
    
    The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID of the experiment
    exp	string	The Emulab experiment ID
    nsfilestr	string	A string representing the NS file to use, with embedded newlines, or,
    nsfilepath	string	The pathname of a NS file on the Emulab file server, within the project directory (example: /proj/myproj/foo.ns)

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for the experiment to finish swapping in (or preloading if noswapin is true). By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded
    reboot	boolean	false	If true and the experiment is swapped in, reboot all nodes in the experiment
    restart_eventsys	boolean	false	If true and the experiment is swapped in, restart the event system (all events are rerun from time zero)

    
    
  • endexp: Terminate an experiment. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment to terminate

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for the experiment to finish terminating. By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded

    
    
  • state: Get the current state of the experiment. The return value is a string; one of active, swapped, activating, etc. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment

    
    
  • waitforactive: Wait for an experiment to reach the active state. Typically, you would use this if you launched a swapin asynchronously, and then wanted to wait later for the swapin to complete.

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    timeout	integer	forever	Timeout after this many seconds. The return code is is RESPONSE_SUCCESS if experiment reaches the active state or RESPONSE_TIMEDOUT if the timer expires.

    
    
  • statewait: Wait for an experiment to reach a particular state. State is one of swapped, active, swapping, activating, etc. If the experiment is already in desired state, returns immediately, otherwise blocks indefinitely until the experiment reaches the state. Use the timeout option below to terminate the wait early. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    state	string	The experiment state(s) to wait for. If a single string is specified the call will block until the experiment has reached that state. If a list of states is given, the call will block until the experiment has reached any of the states in the list. Finally, if an empty list is given, the call will block until the experiment undergoes a state change.

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    timeout	integer	forever	Timeout after this many seconds. The return code is is RESPONSE_SUCCESS if the state is reached or RESPONSE_TIMEDOUT if the timer expires.

    
    The return value is the current state of the experiment.
    
    
  • info: Get information about an experiment. The return value is a hash table (Dictionary) of hash tables. For example, the mapping request will return a hash indexed by node name, where each entry is another hash table of name=value pairs, such as type=pc850 The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    aspect	string	Request information about specific aspect of the experiment.

    
    The aspect is one of:

    Name	Description
    mapping	Request the mapping of nodes in your NS file, to physical testbed nodes. This request is ignored if the experiment is not swapped in
    links	Request information about all of the links in your experiment, including delay characteristics, IP address and mask

    
    
  • nscheck: Check an NS file for obvious parser errors. The return code is RESPONSE_SUCCESS or RESPONSE_ERROR. The required arguments are:
    
    

    Name	Type	Description
    nsfilestr	string	A string representing the NS file to use, with embedded newlines, or,
    nsfilepath	string	The pathname of a NS file on the Emulab file server, within the project directory (example: /proj/myproj/foo.ns)

    
    
  • delay_config: Change the link characteristics for a delayed link or lan. Note that the link/lan must already be delayed; you cannot convert a non-delayed link into a delayed link. When operating on a delayed lan, all nodes (links to the nodes) in the lan will be changed. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    link	string	The name of the link or lan to change; see your NS file
    params	Dictionary	A hashed array (Dictionary) of parameters to change; see below

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    persist	boolean	false	If true, the base experiment is changed in the Emulab Database; changes will persist across swapin and swapout. By default, just the physical experiment is changed, and changes are lost at swapout
    src	string		If specified, change a duplex link asymmetrically; just the link from the node specified will be changed. This option is ignored on lans; the entire lan must be changed

    
    In addition to the required arguments, you must also supply at least one parameter to change in the params argument. The reader is encouraged to read the ipfw and dummynet man pages on users.emulab.net. It is important to note that Emulab supports a smaller set of tunable parameters then NS does; please read the aforementioned manual pages:
    
    

    Name	Type	Range	Description
    bandwidth	integer	10-100000	Bandwidth in Kbits/second
    plr	number	0 <= plr < 1	Packet Loss Rate as a number between 0 and 1
    delay	integer	> 0	Delay in milliseconds
    limit	integer		Queue size in bytes or packets. Default is 50 ethernet sized packets
    queue-in-bytes	integer	0,1	Limit is expressed in bytes or packets (slots); default is packets
    These are valid for RED/GRED links only
    maxthresh	integer		Maximum threshold for the average queue size
    thresh	integer		Minimum threshold for the average queue size
    linterm	integer	> 0	Packet dropping probability expressed as an integer (1/linterm)
    q_weight	number	0 <= plr < 1	For calculating average queue size

    
    
  • eventsys_control: Control the experiment event scheduler. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    op	string	One of start, stop, or replay (stop and then start))

    
    
  • link_config: Change the link characteristics for a wireless lan. Note that the lan must already be a wireless link; you cannot convert wired link to a wireless link! The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    link	string	The name of the lan to change; see your NS file
    params	Dictionary	A hashed array (Dictionary) of parameters to change; see below

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    persist	boolean	false	If true, the base experiment is changed in the Emulab Database; changes will persist across swapin and swapout. By default, just the physical experiment is changed, and changes are lost at swapout
    src	string		If specified, change a duplex link asymmetrically; just the link from the node specified will be changed. This option is ignored on lans; the entire lan must be changed

    
    In addition to the required arguments, you must also supply at least one parameter to change in the params argument. The reader is encouraged to read the wireless tutorial to see what parameters can be changed.
    
    
  • reboot: Reboot all nodes in an experiment. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for all nodes to complete their reboot
    reconfig	boolean	false	If true, do a soft reconfiguration instead of rebooting
    power	boolean	false	If true, power cycle the node; do not try to reboot cleanly. Be very careful with this option!

    
    
  • reload: Reload the disks on all nodes in an experiment. You may specify an imageid to use for all nodes, or you can allow the system to load the default imageid for each node. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	true	If true, wait synchronously for all nodes to complete their reload. The default is to wait; you must turn this off if you want the reload to proceed in the background (not a good idea)
    imagename	string		Specify the imagename to load on all of the nodes. See next option for more info
    imageproj	string		Specify the Emulab project ID of the imageid. By default the system will look in the project of the experiment, and then in the system project for globally shared images.
    imageid	string		Specify image to load using Emulab internal ID for the image
    reboot	boolean	true	If false, the nodes will not be rebooted; you will need to do that yourseld. The default is true (reboot nodes)

    
    
  • savelogs: Save off the console tip logs for all of the physical nodes in your experiment. The logs are place in a subdir of your experiment directory. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment
    action	string	One of start, stop, or replay (stop and then start))

    
    
  • thumbnail: Get the thumbnail image of the experiment topology. The required arguments are:
    
    

    Name	Type	Description
    proj	string	The Emulab project ID in which the experiment was created
    exp	string	The Emulab ID of the experiment

    
    The return value is a byte array containing a PNG image.
    
  
  
• /XMLRPC/node

  The node module lets you control nodes in your experiments.

  • reboot: Reboot nodes. The caller must have permission to reboot all of the nodes in the list, or the entire request fails. The required arguments are:
    
    

    Name	Type	Description
    nodes	string	A comma separated list of nodes to reboot

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for all nodes to complete their reboot
    reconfig	boolean	false	If true, do a soft reconfiguration instead of rebooting
    power	boolean	false	If true, power cycle the node; do not try to reboot cleanly. Be very careful with this option!

    
    
  • create_image: Create an image from a node using a previously created imageid. The The required arguments are:
    
    

    Name	Type	Description
    node	string	The node to create the image from
    imagename	string	The image name (descriptor)

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	false	If true, wait synchronously for all nodes to complete their reboot
    imageproj	string	emulab-ops	The project ID in which the imageid was created; defaults to the system project

    
    
  • reload: Reload the disks on all nodes specified. You may specify an imageid to use for all nodes, or you can allow the system to load the default imageid for each node. The required arguments are:
    
    

    Name	Type	Description
    nodes	string	A comma separated list of nodes to reload

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    wait	boolean	true	If true, wait synchronously for all nodes to complete their reload. The default is to wait; you must turn this off if you want the reload to proceed in the background (not a good idea)
    imagename	string		Specify the imageid to load on all of the nodes
    imageproj	string		Specify the Emulab project ID of the imageid. By default the system will look in the system project for globally shared images.
    imageid	string		Specify image to load using Emulab internal ID for the image
    reboot	boolean	true	If false, the nodes will not be rebooted; you will need to do that yourseld. The default is true (reboot nodes)

    
    
  • adminmode: Boot node into administration mode. The node is rebooted, and its OSID is set to boot FreeBSD from a memory resident filesystem. This allows you to operate on the disk while the disk is offline, say to install a new OS, take a snapshot,etc. The required arguments are:
    
    

    Name	Type	Description
    mode	string	One of "on" or "off". Off returns the node to its previous OSID
    node	string	Node name (pcXXX)

    
    The optional arguments are:
    
    

    Name	Type	Default	Description
    reboot	boolean	true	If true, reboot the node. If false, reset the OSID, but do not reboot the node; you must do that yourself.
    wait	boolean	false	If true, wait synchronously for node to complete its reboot

• /XMLRPC/osid

  The osid module lets you operate on OS descriptors.

  • getlist: Get the list of OS identifiers you can access. There are no arguments and the return value is a hash table containing the OS IDs and their descriptions.

• /XMLRPC/imageid

  The imageid module lets you operate on Image descriptors.

  • getlist: Get the list of Image identifiers you can access. There are no arguments and the return value is a hash table containing the Image IDs and their descriptions.