News

Emulab will be completely down and unavailable this coming Saturday, July 26th from 6-10AM MDT. They are cutting all power to our building during that time frame.

The Emulab documentation has officially moved into a Trac Wiki at https://users.emulab.net/trac/emulab/wiki so that all of our users and collaborators can help us improve our documentation.

Please tell us what you think of the new wiki.

We've released the full Emulab software as open source for your perusal or installation or hacking pleasure. This comes with mostly-online installation documentation, as well. This version 4.9.0 should be fine for installing new Emulabs, but we strongly recommend that people upgrading their existing Emulabs wait for version 5.0, which we intend to release in July.

Get your source and your docs from our software page.

Kudos to the skilled and hard-working Utah Emulab developers and operators for this magnum opus. Special acks to Grant Ayers for testing and the docs, and to Mike Hibler and Leigh Stoller for many things.

A new form is available to record any publications, theses, reports, talks, software, or services for which you used Emulab. If your publication or significant software is not already listed in our publication list and you have used Emulab to validate your research, please use the form to add it (must be logged in). And thank you-- doing so will help keep Emulab funded!

We'd like to make our users aware of a call for papers that might be if interest to many of you. CSET is a workshop seeking short papers on testbed use and tools. It is primarily aimed at, but not limited to, security-related research.

The following is an excerpt from the CFP. The full CFP can be found here.

Security challenges constantly grow in complexity and scale. To meet these challenges, security professionals need safe experiment environments, tools, and methodologies to:

• capture new threats,
• study threats through interactive experimentation,
• dissect and reassemble malware,
• pit new attacks against proposed defenses, and
• test defensive technologies in a large-scale, realistic setting.

This workshop aims to gather both researchers who use testbeds for security experimentation and testbed developers to share their ideas and results and to discuss open problems in this area. While we particularly invite papers that deal with security experimentation, we are also interested in papers that address general testbed/experiment issues that have implications on security experimentation such as traffic and topology generation, large-scale experiment support, experiment automation, etc.

Tomorrow (Tuesday) at 7 AM MDT, we'll be switching our routes back through our bypass of the campus firewall. This is to "back out" the temporary workaround put in place to fix Friday's outage, getting us back to our usual configuration, which exempts us from campus firewall rules.

The outage is expected to be short, and we'll simply revert if it causes problems.

Longer-term, we have discussed a solution with our campus network operators that should cause more graceful failover when things go wrong, and should give us more control over our routes, so that we can fix things up on our own. The plan is to implement this in a few weeks.

Update: This outage is complete, with < 1 minute of downtime

There is now a Fedora 8, Emulab-supported disk image available for use under the name FEDORA8-STD. In addition, a 64 bit (x86_64) version of Fedora is available under the name FEDORA8-64-STD.

The 32 bit and 64 bit versions have an identical set of packages installed, apart from some 32 bit compat library versions on the 64-bit image. For additional information on the 64 bit image, see this KB entry.

Note that end node traffic shaping is unavailable; we need to port some modifications made to 2.4 kernels to the 2.6 line.

If you find something missing or broken, please let us (testbed-ops) know.

We've added Emulab support for NetFPGA boards (programmable network devices designed at Stanford University). NetFPGAs appear largely as a normal Emulab node, so you can add them to your experiment topology, reprogram the FPGA, and experiment with your own hardware-assisted router designs! Currently we've installed one NetFPGA each in two pc3000s, but this is only the first step in a larger deployment.

For more information on how to use these devices, , look at the tutorial at

   http://www.emulab.net/tutorial/docwrapper.php3?docname=netfpga.html

For general NetFPGA information, look at http://netfpga.org/.

If you have questions or problems with the NetFPGAs, send mail to testbed-ops.

Experimenters who have used our building-scale wireless testbed, or plan to, should read this note.

This week, we'll be replacing our existing 18 building-scale wireless nodes, starting on Tuesday (Dec 18). The deployment of these new nodes is the beginning of a larger deployment, but one that will take some time. The new nodes have faster, dual-core processors, and different wireless cards (based on the Atheros AR5006XS with the AR5414 chip).

Based on our usage observations, we don't believe this change this will jeopardize any in-progress paper efforts; however, please let us know if you need specifically the current set of nodes and the existing hardware for your experiments, and we'll try to adjust our deployment plan to accommodate your needs.

We apologize for the short notice! If you have questions, please send mail to testbed-ops.

To get the per-node-type free counts like used to be there, click on the little "Free PCs" box in the banner. It's a toggle.

Thanks to Leigh for adding this. It took some hacking due to restrictions of iframes.

Just a heads up that sometime in the next day or two, Emulab will get a slightly different look. The main menus, which have traditionally been on the left side of the screen, are moving to the top of the screen as drop down menus. The banner across the top will also be smaller and more space efficient.

The motivation for these changes is to leave more room for actual content in the main window. Our ongoing "experimenters' workbench" project needs that real estate, but the additional space is generally useful.

We have tried to test this out on as many browsers as we can, but there are bound to be glitches. We do know that it works in Firefox (1.5 and 2.0), Safari (2.0 and 3.0) and on Internet Explorer Version 7. IE6 has some visual issues that might make it a little annoying, but view it as incentive to upgrade your version of IE (IE7 has been out a long time!).

With the looming deadline on next Tuesday, there has been a surge in demand especially for 'pc3000' nodes. We have one experimenter who needs to get a 51 node pc3000 experiment swapped in so I have started reserved nodes for his use. What everyone can do:

* If you do not need Gb ethernet or fast machines, please use the pc600 and pc850s. Use tb-set-hardware to force use of those.

* If you need Gb ethernet and fast machines and your experiment is small (< 10 nodes) and it is short duration (< 12 hours), consider using the 'pc2400c2' nodes which are the fastest machines out there, but there are only 20 of them. Again, ask for them with tb-set-hardware.

Update: I forgot two other important criteria for using the pc2400c2s. One, is that you must use either FBSD62-STD or FC6-STD as the OS. Two, is that you cannot need more than one experimental interface per node (they have only one interface hooked up).

The Flux Research Group at the University of Utah is pleased to announce the availability of a new 802.11 wireless testbed within the Utah Emulab facility. We've added 802.11a/b/g cards to 36 rack-mounted PCs and deployed the antennas in a compact grid (3m x 2.25m) on the back of the racks in our machine room. These machines are already part of the Emulab infrastructure, with five wired interfaces per node, and the many existing Emulab wired and wireless features are available on them.

The goal of this mini-testbed is to give users an opportunity to conduct experiments with wireless nodes in a congested, compact, and hostile environment. By deploying these nodes in a machine room, we expose them numerous opportunities for interference, multipath effects, and congestion. We believe that experimenters can use these nodes to effectively test applications and protocols in congested, close-quarters scenarios, such as a meeting with many laptops in a crowded conference room. Experimenters can easily specify wireless topologies, modify wireless and link parameters during experiment runtime, and include in their experiment nodes that are deployed as part of our existing building-scale 802.11 wireless testbed.

You can look at the hardware configuration and some environment information for these PCs at

  http://www.emulab.net/tutorial/docwrapper.php3?docname=wireless.html

Sign up for an Emulab account here:

  http://www.emulab.net

A number of people have expressed interest in Intel machines with VT technology for doing hardware-assited VM experimentation. To accomodate these people we have setup 20 Dell Optiplex 745 dual-core machines, each with a single Gb control interface (see here for details). These are known as pc380-pc399 and you should ask for nodes of type pc2400c2 to get them. Current, only a small subset of OSes run on these machines: FreeBSD 6, FreeBSD 7, and Fedora Core 6. We have no plans for further OS support (read: Windows) at this time.

This should enable some small scale experimentation with Xen and other VMs.

We've improved wireless network configuration, both statically via NS files and dynamically during experiment runtime, for both 802.11 and GNU Radio devices, and have updated the default wireless disk images.

Details:

• (802.11) More control via NS files and better wireless mode selection. For instance, you can now set up adhoc mode in an NS file, and you can specify more hardware control parameters to iwconfig.
• (802.11) More options for dynamic network configuration (changing wireless device properties via link_config or XMLRPC) at runtime. In addition, dynamic config can now help you work around possible issues in the madwifi drivers and Linux wifi support.
• (GNU Radio) Support for dynamic reconfig. Also added important params that can help you get your GNU Radios talking to each other more easily.
• (802.11) Updated the madwifi drivers on both wireless images to the latest version (0.9.3). For RHL90-WIRELESS, we also updated the kernel to version 2.4.34 and patched the wireless extensions in the kernel to version 18.

The FC4-WIRELESS and RHL90-WIRELESS standard wireless images now contain these and other changes. If you encounter any problems, you can use the FC4-WIRELESS-OLD and RHL90-WIRELESS-OLD images.

Check out the wireless and GNU Radio tutorials for more information:

http://www.emulab.net/tutorial/docwrapper.php3?docname=wireless.html
http://www.emulab.net/tutorial/docwrapper.php3?docname=gnuradio.html

As always, please let us know if you have any problems with these images.

The new images described in the previous message are now installed at -STD. Again, if you encounter any serious problems, you can use the -OLD images.

UPDATE: Make that Thursday May 10th! I have been side-tracked by other issues and don't want to install the new images this late in the day.

UPDATE: Make that Wednesday May 9th!

On Wednesday May 8th we are going to install a batch of new images. Once again, there are no changes to the OSes or standard utilities, but this time there will be a significant change to the Emulab client-side infrastructure.

We are replacing the underlying mechanism of the Emulab event system. We have been using Elvin, but due to restrictive licensing, we have decided to replace it with a much simpler, home-grown publish-subscribe system.

What does this mean to you? Well, hopefully nothing! In theory, everything will continue to work as it did. The potential implication is that your events might behave differently. So if you encounter problems with your scripted actions (e.g., program agents don't start) let us know.

If you don't want to deal with potential problems right now, you can continue to use the old standard images by specifying -OLD images (e.g., RHL90-OLD instead of RHL90-STD).

If you want to preview the new images now, use the -UPDATE versions (e.g., RHL90-UPDATE).

The affected images are: FBSD410, FBSD54, FBSD62, RHL90, FC4, FC6.

There is now a Fedora Core 6, Emulab-supported disk image available for use under the name FC6-STD. We've tried to include a similar set of software packages to those installed on the FC4-STD image (of course, the versions are updated), so the environment should feel similar.

Note that end node traffic shaping is unavailable on FC6-STD; we need to port some modifications made to 2.4 kernels to the 2.6 line.

If you find something missing or broken, please let us (testbed-ops) know.

We have a new experiment specification GUI that has been in testing for quite some time now. We're happy to announce that it is now available to all Emulab users. There is a link to the new GUI at the top of the Begin Experiment page, or you can access it directly. The old NetBuild GUI is still available.

The new applet supports a much broader range of Emulab features, such as software installation, program control, traffic generators, and setting of environment variables. It uses our XML-RPC interface to automatically download lists of images you are allowed to use, tarballs/RPMs in your project directory, and more. Let us know if you have any problems with it or feature requests.

Emulab now has an Opensearch plugin. This means that you can use that little search box in the toolbar of Firefox 2.0 or IE 7 to search the Emulab documentation and knowledge base. In Firefox, the dropdown arrow next to your current search engine should glow, and you can add the Emulab search engine by clicking on it. Or, in any browser that supports Opensearch, click on the new link at the bottom of the search results page.

Note that it is now DST in the United States, three weeks earlier than usual. The biggest impact of this is that older OS images will display the incorrect time. I have fixed the FBSD410, RHL90, FBSD54 and FC4 -UPDATE images, and will get to the other newer images. I will move these to the standard images once the SOSP deadline has passed.

Of course, if you use UNIX timestamps or UTC, you will be unaffected.

UPDATE: and those with a sense of irony will note that the time stamp for this message, as well as the time displayed in the upper right side of the frame, are wrong! The web server needs to be restarted to pick up the corrected timezone info...

We've posted our final NSDI papers on Flexlab and the Experimentation Workbench. These are running in alpha test in Emulab right now. If you'd really like to try one of them for your research, especially Flexlab, let us know.

As part of the Emulab Federation and Workbench projects, we are continuing to rollout substantial changes to the Emulab software, particularly the web interface code. These changes are primarily to allow for users and other core abstractions to be globally named. These changes should be mostly invisible, but some bugs are bound to creep into the system. Please report anything that seems wrong or questionable.

We've updated our banner, usage table (that thing in the upper right corner), and a few other minor things. We've tested it with all the browsers we could lay our hands on, but if you notice any formatting problems, let us know.

We have added more disk space on users.emulab.net. At the moment, this is manifested by a larger (40GB -> 200GB) /users filesystem and a much larger (430GB -> 2TB) /proj and /groups filesystem.

Is it enough? Well, our limitation here is really how much we can afford to backup to tape, and not how much disk space we can buy. As a compromise, we will be making available another 2TB of space, with the caveat that it will not be saved to tape. However, this is still part of the larger RAID5 array, so hardware failure will not cause loss of data.

All files from the old filesystems should be copied over to the new filesystems but in the event that something is missing, the old filesystems are temporarily available in /old (e.g., /old/proj, /old/users).

More changes were installed today; the newuser and newproject pages have been fully reimplemented in the Emulab backend so that they can be accessed from the XMLRPC interface in addition to the web interface.

These changes should be mostly invisible, but some bugs are bound to creep into the system. Please report anything that seems wrong or questionable.

As part of the Emulab Federation project, we are rolling out substantial changes to the Emulab software, particularly the web interface code. These changes are primarily to allow for users and other core abstractions to be globally named. These changes should be mostly invisible, but some bugs are bound to creep into the system. Please report anything that seems wrong or questionable.

For previews of three major new features that we'll put into beta production in weeks to months, see the new papers and slides we've recently posted, or better, go to the three new project project pages linked from the Flux Group web site:

• Flexlab:
  1) The best of PlanetLab and Emulab, wrapped up into one system.
  2) Separated network model to drive Emulab experiments.
  In alpha test in the live Utah Emulab, with 3 models.

• Experimentation Workbench: Automate, record, and export all your work, experiments, data, and analyses. In alpha test in the live Utah Emulab.

• Fully stateful swapout and time-travel of experiments. Interrupt your running experiment at any point and transparently continue it tomorrow. Can't find that bug? Then go back and forward in time. The pieces for stateful swapout are done.

We've been working hard on each of these projects for at least 12-18 months. More news on several other developments coming soon.

I have updated the Windows images with the current Emulab test node-side (linktest and event system improvements) and increased the C: disk space from 4 to 8 gig.

WINXP-UPDATE* include Windows Updates through the 11/14/2006 Patch Tuesday. The WINXP-SP{0,1,2}* images, as always, are unpatched. Use WINXP-UPDATE for normal security.

I have once again updated the standard images. The set includes: RHL90, FC4, FBSD410, FBSD54, FBSD61.

Once again, the changes are confined to the Emulab client scripts (and once again, fixes to linktest!) and no kernels or standard applications are affected.

If you do encounter changed behavior, let us know ASAP!

We added instructions on how to configure your Mac OS X browser to do SSH to Node. Please let us know if you have any problems or questions.

I have updated the -STD versions of: RHL73, RHL90, FC4, FBSD410 and FBSD54. The new versions have updated Emulab client-side scripts. The most important change is to "linktest," the link tester that runs at experiment swapin time (whether you want it to or not!) The new version should run much faster for LAN topologies and generate fewer false positives.

The only other difference between the new and old images that has been reported is that /usr/local/bin/emulab-iperf has moved to /usr/local/etc/emulab/emulab-iperf where all the other emulab specific binaries are.

There should be no changes to the kernel or standard utilities, but let us know if you encounter problems.

Many people have noticed that Emulab "disappeared" for hours at a time, twice this week. This was due both times to routing problems in our upstream providers. We are going to get together with them to see what we can do to prevent this in the future.

Just FYI, to let you know that these were not Emulab-related downtimes that we forgot to inform people about. But we apologize just the same, especially to those of you who are frantic due to imminent deadlines!

File backups have resumed on Emulab. Please recall that we only backup the filesystems on the Emulab servers and _NOT_ those located on the experiment nodes. This covers the /proj, /group, and /users trees on users.emulab.net (ops), which are NFS mounted on the experiment nodes.

The Emulab Windows XP images are no longer specific to a particular hardware type, and will work on pc600, pc850, pc3000, and pc3000w nodes. See the Change Log for more information.

Need to build binaries to run on PlanetLab? We now have a disk image that acts as a PlanetLab DevBox. This means it has software as similar as possible to the PlanetLab nodes. Unlike the PlanetLab nodes, however, it has a compiler. It also has the usual Emulab software, so you'll get your account, home directory, etc.

You can use it by asking for the PLAB-DEVBOX image. If you don't want to have to mess around with an NS file, you can use our tool to automatically generate an NS file using this image.

Find out what OS is favored by studly kernel hackers from Truth or Consequences, NM, in this interview at http://blogs.ittoolbox.com/unix/bsd/archives/007589.asp.

I finally put out the pictures from our cluster expansion last summer. See the Photos section.

We are now running NS files through the real NS parser as a check that our own parser is functioning correctly. This change should not cause a problem for most files, however, there is some syntax that works in our parser but not the real one. For example, the following NS would not parse correctly anymore because the "node1" and "node2" strings are not variable references:

  set node1 [$ns node]
  set node2 [$ns node]

  $ns make-lan "node1 node2" 10Mbps 0ms

  $ns make-lan "$node1 $node2" 10Mbps 0ms

  for {set i 0} {$i < 10} {incr i} {
    set name node$i
    set $name [$ns node]
    append lanstr "$name "
  }
  $ns make-lan $lanstr 10Mbps 0ms

    append lanstr "[set $name] "

1. Due to high demand for resources, we've changed the idle timeout from 4 hours to 2 hours. After that your experiment will be swapped automatically.

2. You are expected to pro-actively swap out your experiment when you're done, not wait for Emulab to detect it idle. Please try to remember; it helps everyone.

3. Finally, if you're consuming lots of testbed resources, and you could be using our virtual nodes (FreeBSD jails) but aren't, you can expect us to start leaning on you, possibly hard. So you might start doing that application port to FreeBSD now, from Linux. Spend a day, save 50 PC3000s day after day.

We experienced a lockup of our ops/users machine earlier tonight. After forcibly rebooting it, it hung again as it was coming up. It appears that the second time at least, was caused by massive NFS activity.

A reminder: please avoid "excessive" logging of experiments to the /users and /proj NFS mounted filesystems. Excessive here means not only high volume from a single node, but also moderate volume from a large number of nodes!

Look into the loghole facility, which allows you to log to the local nodes and return the logfile automatically later.

I have once again installed new versions of the so-called "standard" images: FBSD410-STD, RHL90-STD, FBSD54-STD and FC4-STD.

The changes were only to update the Emulab client-side (test node-side) linktest scripts; no other kernel or application changes have been made. So you should see no functional or performance differences.

We've added an RSS feed for this news page. To subscribe, simply paste the URL from the "RSS Feed" at the top of this page into your favorite RSS news reader.

Firefox, Safari, and other modern browsers will auto-detect the presence of the feed and present you with an icon (usually in the URL bar or the status bar) to allow you to subscribe.

After a very long interruption, Emulab's PlanetLab portal is up and running once more. Users with PlanetLab access should definitely give it a try (and provide feedback).

Note that you can quickly and easily setup a PlanetLab slice including custom software installation, automatic script startup, and powerful but simple node selection via the "Create a PlanetLab Slice" menu link. The traditional NS script and XMLRPC paths are, of course, also available.

Read the PlanetLab interface documentation for more info.

I have installed new versions of the so-called "standard" images: FBSD410-STD, RHL90-STD, FBSD54-STD and FC4-STD.

The changes were only to update the Emulab client-side (test node-side) scripts; no other kernel or application changes have been made. So you should see no functional or performance differences.

The new images will allow automatic link testing (connectivity, latency, loss, bandwidth) at swapin time when enabled. This feature ("linktest") has been present for awhile but was optional and not very reliable. The latest version is significantly better and will be enabled by default, for all experiments, when we've worked out some issues.

We have updated the legacy Emulab RHL73-STD image to run on the new pc3000 nodes. In the process, we updated the packages and kernel to get security and bug fixes (all updates are from the RHL73 channel of fedoralegacy.org). Please also note that the kernel has been compiled to support SMP, and has a modern Intel Pro/1000 driver.

Emulab has just made available a new set of collaboration and development tools, which we hope will assist you in your research, development, and experimentation.

On your left hand side is a new menu called the Collaboration menu with four entries (more entries to be added later).

• My Wikis: Each user has their own Wiki page, and each project has its own WikiWeb. Users are of course free to modify their Wiki pages and any pages belonging to the projects they are members of. Project leaders may change the permissions on project pages, making them readable (and writable) by other Emulab projects and/or members, or they can make them world readable (and writable). External users (without Emulab accounts) may also apply for wiki accounts so they can access your Wikis.
• My Mailing Lists: Emulab has switched its mailing lists over to the popular, open source, Mailman package. All of the per-project and group lists are now maintained as a Mailman list on users.emulab.net. In addition, any registered Emulab user can create (and administer) new lists. External users (without Emulab accounts) are free to join your lists.
• My Bug Databases: Emulab has installed the popular FlySpray bug tracking package, making it available to all projects. Each project has its own bug tracking database, initially accessible by just project members. Project leaders may also allow external users read or read-write access to project bug database.
• My CVS Repositories: Each project now has its own CVS repository, hosted on users.emulab.net. Repositories can be accessed by project members via the web, or directly on users.emulab.net, or with CVS over SSH. Project leaders can make their repositories publically readable via a toggle on the Emulab project page.

As always, please feel free to comment on these new services, report bugs, request new features, etc. We have plenty more coming and your comments can affect that.

For more details on the pc3000 machines, see this document.

Just a very quick note to help you avoid trouble; more details later.

The 160 new 'pc3000' nodes are available to all. 3.0Ghz, 2GB memory, 2 big fast SCSI disks, 4 experimental interfaces, with between 1-4 of those connected to a fast Gbit-capable switch; all are 100-mbit capable.

The potential "trouble" is that these nodes are hugely different in performance from the existing pc600/pc850 nodes that you use most of the time. So if your experiment specificies type "pc" you could now get a mix of nodes of very different performance characteristics. Previously, that was much less likely (we only had a small number of pc3000w's and pc2000's).

More later, but feel free to use them now. Try to wait to ask questions, as our upcoming docs should answer most.

We have changed the node type name for WiFi nodes. What was once 'pc3000' is now 'pc3000w'. I have attempted to fix all existing experiments, so you just need to make sure that you specify 'pc3000w' in all future experiments.

Note that this is the type name only, and does not affect the names of individual nodes. So if you use tb-fix-node for a specific pcwf node, you do not need to change that.

Why the change? Because we want to use 'pc3000' for our 160 new cluster machines. Given that we have pc600, pc850 and pc2000, pc3000 is the logical follow on.

An official Red Hat Fedora Core 4 image, using the 2.6 kernel, is now available for use under the name FC4-STD. This is a supported, functional Emulab image. Many thanks to Kevin Atkinson for putting this together.

Please note the following two caveats:

• linktest (topology verifier) is unavailable as it needs to be ported to FC4
• End-Node Traffic Shaping ("linkdelays") is unavailable as the required kernel modifications need to be ported to Linux 2.6.

Well, it took most of the day, but users.emulab.net has been upgraded. It went from dual-850Mhz to dual-3000Mhz CPUs so you will notice a speed boost. More importantly, it has double the amount of disk space, so we should not be constantly running out!

If you had an experiment swapped in during the transition from old machine to new machine, you may have to reboot all the nodes in the experiment to clear "stale NFS file handles". Let us know of any problems.

I have updated the common standard images (FBSD410-STD, RHL90-STD, FBSD53-STD). This is largely a maintenance update to get the Emulab client scripts up to date. Both FreeBSD and Linux also have new kernels, required to include support for the SCSI controller on our upcoming cluster nodes. This kernel change should not affect the performance or function of the kernel.

Let us (testbed-ops) know if you encounter any problems.

I'm pleased to announce that we will be bringing online a lot of new hardware in the next couple of months. Fast nodes, gigabit, a really fast switch, and tons of disk space! I/O capacity was the primary technical criterion for the nodes.

• 160 new nodes, Dell 2850s:
    Processor: 3.0 GHz 64-bit Xeon, 800Mhz FSB
    Memory: 2GB DDR2 400Mhz
    I/O: Multiple PCI-X 64/133 and 64/100 busses
    Network: 6 10/100/1000 Intel NICs spread across the busses (one NIC is the control net)
    Disk: 2 x 146GB 10,000 RPM SCSI
• New servers: users/ops, boss, and a new fileserver:
  Like the test nodes, but dual processor, 4G RAM, more disk, RAIDed.
• Multi-terabyte storage array
• Switches:
  Cisco 6509/Sup720 - 720Gbps switching fabric; hi-speed Gbit line cards
  Cisco 6513 - mostly 100Mbps
  Cisco 6506 - control net

Many machines will not have all 5 experimental NICs attached to switch ports, but we will have almost 400 Gbit switch ports immediately. We may directly connect some nodes. The Magic of Assign will transparently handle all this physical network heterogeneity.

We are retaining our current 200-odd nodes, so will have about 360 total. If we solve the enormous scaling problems on swapin, our back-of-the-envelope suggests one could run a 10,000-20,000 virtual node experiment. There are about 18,000 AS's in today's Internet...

We have fixed the problem that was causing Windows experiments created with Netbuild to fail. Netbuild creates all experiments with "Static" routing enabled, and that form of route computation was broken. There was also a problem with Windows finding all the interfaces when it booted. Hopefully that is fixed as well.

To summarize, use the WINXP-UPDATE image. This is an image of WindowsXP SP2 with security fixes up through yesterday. The firewall is disabled (as it will inform you repeatedly!) by default.

We've opened for public use four major new features in Utah's Emulab network testbed. They are documented and ready for anyone's reasonable research or educational use, or just take them for a test drive.

1. 900MHz Motes and 802.11 Stargates on robots: a truly mobile wireless testbed, with experimenter-specified movements, simple path planning by Emulab, a vision-based tracking system, live maps, and webcams. Control and monitoring are completely integrated into the existing Emulab system, so a mobile experiment is very similar to a classic Emulab experiment. This first installation is small, but it all works.
Mobile robotic wireless tutorial.

2,3. Secure environments for running malicious or risky code:
      A per-experiment switch-enforced firewall.
      A per-experiment Emulab, by providing Emulab-in-Emulab.
Plus, at no extra charge, color-coded security levels that automatically set up different defense levels.
Secure environment docs.

4. Windows XP is now a supported test node OS, joining Linux and FreeBSD. Most Emulab features work, including ssh keys and file service. It's got Remote Desktop if you need Windows look-and-feel; it's got Cygwin if you prefer Unix.
Windows XP docs.

Finally, let me remind you that we also have a non-mobile 802.11a/b/g testbed available: 27 PCs scattered around our building, most with two Atheros cards. You can change the madwifi driver, the OS, everything.

Click here to request an account.

Kudos to the whole Emulab team, but especially to Tim Stack, David Johnson, Dan Flickinger, Leigh Stoller (robots); Mike Hibler, Leigh Stoller (secure env); and Russ Fish, Kirk Webb (Windows).

Jay Lepreau
University of Utah

In order to reduce the number of wasted nodes, we are now defaulting Max Duration in the Begin Experiment page, to on. Idle or not, your experiment will be forcibly swapped out after the allotted duration, currently 16 hours, unless you uncheck the box or change the duration.

Please don't uncheck it without good reason. Should you later find you need more time, you can alter this param using the "Edit Experiment Metadata" link on the experiment info page.

I have installed a new default image. The only changes should be to the boot-time Emulab configuration scripts which should not affect your experiments, past, present or future. Let us know if you have any problems.

Due to popular demand (ok, one person asked for it :-) we have put out a prototype Fedora Core 2 image. This image should be considered unsupported, but of course we would like feedback about anything misconfigured or missing.

To use the image, put:

    tb-set-node-os $node FEDORA2-NONSTD

Mike added support to Emulab that provides a flexible control-network firewall between any experiment and the outside world. For example, key syntax is:

        # Create a firewall node
	set fw [new Firewall $ns]
        # Configure it
	$fw set-style <closed|open|basic>

On Monday, September 13th at 9am MDT we will be switching our "default" OS images from FreeBSD 4.7 and RedHat 7.3 to FreeBSD 4.10 and RedHat 9.0.

How will this affect you? Well, if you explicitly specify an OSID in your new experiments (e.g., tb-set-node-os FBSD49-STD), then you won't notice at all. However, if you specify one of the "standard" images (FBSD-STD or RHL-STD) you will get the new OS version. Likewise, if you do not specify any OS, which defaults to RHL-STD, you will get the new version.

When you swapin an existing experiment which uses (directly or indirectly) either FBSD-STD or RHL-STD, the OS you get depends on when the experiment was created. Experiments created prior to 9am Monday will continute to be mapped to FreeBSD 4.7 and RedHat 7.3. Experiments created after that time will get 4.10 and 9.0. If you prefer that your older, existing experiment get the newer OS, then you will have to modify the experiment and set the node-os explicitly.

UPDATE: I have moved the changeover time to 10am MDT to make sure I am on-site when it happens.

We have reenabled the web interface and brought up 40 (Sep 7th: all) of our nodes.

If the new air conditioner is still running smoothly tomorrow (Sunday), we will bring up the rest of the testbed. We took advantage of the down time to upgrade a few things:

• ops.emulab.net got a minor face lift. It is now a dual 850Mhz machine with 1GB of RAM (up from 1 500Mhz CPU and 512MB). It is also now back on a gigabit interface rather than 100Mb.
• Both ops and boss were upgraded to FreeBSD 4.10 (from 4.7 and 4.9) and their various ports upgraded. There should not be compatibility issues, but let us know if there are!
• Emulab has changed its IP space. As part of a campus wide "re-IPing", we have moved from 155.101.128/20 to 155.98.32/20. Addresses of note:

          boss    from 155.101.128.70 to 155.98.32.70
          ops     from 155.101.129.74 to 155.98.33.74
          pcNN    from 155.101.132/22 to 155.98.36/22
  

Still to come, later next week:

• We will be switching the default OS images for the nodes from FreeBSD 4.7 and Redhat 7.3 to FreeBSD 4.10 and Redhat 9.0. If you do not specify any OSID, or use the generic "RHL-STD", for a new experiment you will now get RHL90-STD. If you specify the generic "FBSD-STD" you will get FBSD410-STD. Existing swapped experiments with those generic OSIDs will continue to get mapped to the old RHL73-STD and FBSD47-STD when swapped in.

To make sending events to multiple agents easier, we have added Event Groups. Quick example, say if you have four links, named link0, link1, link2, link3:

	set mylinks [new EventGroup $ns]
	$mylinks add $link0 $link1 $link2 $link3

	$ns at 60.0 "$mylinks down"

I'm pleased to announce public availability of these features and hardware on the Utah Emulab facility, for anyone with a valid research, educational, or development use. See www.emulab.net for details or to sign up.

1. These five new types of nodes and networks can be mixed and matched, are fully integrated into Emulab, and support (almost) all of its features.
   • Wireless 802.11a/b/g. 9 nodes now, 30 any day, more later.
   • 8 IXP1200 network processors.
   • VM-based nodes and networks; scales to 2000 high-degree nodes.
   • All PlanetLab nodes; many features: www.emulab.net/doc/plabexamples.pdf and plab.html
   • ns-based simulated nodes and networks.
     And existing:
   • 180 PCs with 5 ethernet interfaces
   • 45 RON/netbed widearea nodes

   We've had some of these node types for ~8 months (VMs, PlanetLab) but they were not widely advertised; others are newly featureful (ns).

2. XML-RPC interface to Emulab. Control Emulab from your desktop command line or roll your own client in your favorite scripting language.
3. GUI client for creating and interacting with experiments, as an adjunct or alternative to the Web interface. Build/edit topologies graphically, click on nodes to login, click on links to change properties, etc. Alpha software, but useful. See www.emulab.net/netlab/client.php3
   • Build and interact with the topology from a single interface.
   • Interact with existing topologies.
   • Support for Emulab-specific extensions to the NS syntax.
   • Uses the Emulab XML-RPC interface.
   • Integrated access to node consoles.
4. Many other features added long ago, including:
   • Flexible barrier synchronization
   • SFS support
   • Integrated event system:
     -Schedule events statically as in 'ns', or dynamically
     -Control arbitrary programs, links, nodes

Jay Lepreau
University of Utah

We have extended the Emulab software to support wireless LANs, and have started deploying a medium-sized public testbed running 802.11a/b/g. These can be mixed and matched with other types of nodes and links. 9 wireless nodes are currently available; 20 more in boxes will be installed soon. We expect to add more in the coming year, as well as many mote-level devices.

Read all about it and give it a spin, but pay attention to the acceptable use policy.

I have updated the documentation for Hybrid Experiments with Simulation and Emulation with all the new capabilities. There is also a picture illustrating a detailed example on using this.

I've added a couple new pages of documentation:

• the control net
• virtual nodes
• IXP network processsors

An XML-RPC interface to Emulab is now available! This interface allows you to perform common operations from the command line, or you can roll your own client in your favorite scripting language (Python, Perl, etc).

Full documentation is here.

We have made two new FreeBSD images available, one for FreeBSD 4.9 and one for the recently released FreeBSD 5.2.1.

FBSD49-STD is intended to become our default image, replacing FBSD47-STD for use in delay nodes and virtual (jail) nodes as well as for everyday use. 4.9 is supposed to be the end of the road for the 4.x series, so I figured it was time to upgrade. Sources are accessible via /share/freebsd/4.9/src.

FBSD52-STD is a first attempt at the latest FreeBSD line. This image also has updated ports. Sources are at /share/freebsd/5.2.1/src.

Emulab user Craig Rodrigues of BBN has been kind enough to share his hard work in upgrading the standard Emulab RHL73 image to RHL9. We are providing it as an interim solution for those seeking RedHat 9 while we work on producing an official testbed release.

You can reference this image as "RHL9-NONSTD" in your ns files or from Netbuild.

Note that there are a few things to be aware of if you choose to use it:
* The image is _NOT_ supported.
If you run into problems, you're on your own. You are welcome to email Testbed Operations (NOT Craig!) with problems, but don't count on any responses.

* The 2.4.20 kernel in the image contains modifications.
It has been modified to include support for the SCTP protocol (an alternate to TCP and UDP). This shouldn't present a problem, but please be aware.

We have made a slight change in the way /etc/hosts files are generated on the nodes. The previous behavior was that we would only create "aliases" (short names, referring to the name of a node, and not a spicific interface) for nodes that are not directly-connected if there is a route between them. The new behavior is to do this for all nodes, regardless of whether or not a route exists.

For an explanation of the various ways you can name nodes within an experiment, see this FAQ entry.

Emulab's Planetlab support, for creating and managing dynamic slices/experiments, has been broken by Plab's abrupt move to new software, triggered by their security breakin. We'll re-target as soon as we get details on the new APIs, and APIs that work.

The program object no longer allows you to specify the command to run when using the command line interface. The old syntax:

tevc -e foo/bar now prog0 start COMMAND='ls /'

You no longer specify COMMAND. Instead, the command is that which was specified when the progam agent was created in your NS file. In other words, we no longer provide a remote execution facility for arbitrary commands.
Instead you can start/stop existing program agents as they are specified in your NS file.

I have put out a "snapshot" of the current Frisbee source code in the Software section. We have made a couple of bug fixes and improvements and wanted to get the code out there. See the README for more information. There is also an updated ISO image.

We've made PlanetLab resources available through the familiar Emulab interface, with many of the standard Emulab features and some new ones. New ones include load-driven allocation with admission control. Optionally combined with Emulab's queuing facilities, that is useful for sharing PlanetLab resources when they are scarce.

You might use the Emulab interface to provide a richer and more friendly interface to the PlanetLab nodes and to your experiment. Or, you might use it to get the N least-loaded nodes that satisfy your node type criteria, perhaps queuing your request until enough become available.

Note that we don't provide tunnels or link or topology emulation! Our PlanetLab support gets you a bunch of rich (v)nodes fully interconnected through the normal Internet: not any specific topology.

More details are available in the announcement message and in the prototype ns file, planetlab.ns.

Sorry to to have to do this, but our cluster is overloaded and people with imminent important deadlines are not able to swapin, despite active coordination with us and trying at all hours by them. Often they cannot map due to just a couple of missing nodes.

Therefore, for the next week, until Tuesday 9/23 just after the NSDI deadline, or until earlier notice, use of the Emulab cluster is limited to people working on deadlines that are

• imminent, and
• important.

If you think you might qualify, send mail to testbed-ops@flux.utah.edu. Otherwise, please don't create or swap in *any* experiments until 9/23 or further notice from us, and swap out any active experiment right away; thanks.

With luck the main deadline-driven users will get their work done soon and we'll be able to relax this policy early.

Additional alternatives are the University of Kentucky's Emulab cluster, which is available for external research, and perhaps Georgia Tech's is also. They are linked off the "Other Emulabs" link on our home page.

p.s. If you wish, talk to your funders, asking them to support getting us more nodes.

With the impending NSDI deadline, many people are having trouble getting experiments swapped in. A heretofore undocumented feature of batch experiments can help. If you submit an experiment as batch mode, without including tb-set-node-startcmd commands in the NS file, the experiment will be queued until sufficient nodes are available. See the FAQ entry and batch mode introducation for details.

In order to support more than 255 links in an experiment, we have switched the IP subnet base from 192.168.x.x to 10.x.x.x. If you have any scripts that assume 192.168, you will need to change them.

This only affects new experiments and experiments that you modify with the Modify option on the experiment info page. Existing experiments that are either swapped in or swapped out will continue to use their existing 192.168 address until they are terminated (obviously) or "modified"

You may notice changes in the "Begin an Experiment" form. We modified it to incorporate some new features we've recently developed. Along with it, the documentation in the Node Usage Policies/Swapping page has been significantly updated.

The previous "Swappable" field is replaced by the "Idle-Swap" field. "Idle-swap" lets you choose whether or not your experiment can be automatically swapped after it has been idle the designated time. If you uncheck it, you have to give a good reason. Note that even if you uncheck this, but your experiment is idle a long time, the system will send you mail, and eventually we will manually swap it for you if you don't respond. You may lower but not raise the idle-swap time.

To be extra clear, if you don't check idle-swap it doesn't mean you're totally unswappable, it just means that we'll attempt to interact with you before we manually swap you out. Talk to Testbed Ops if you must have a totally unswappable experiment or need the idle-time raised.

The new "Max Duration" field lets you set a timer to cap the length of time your experiment runs after swapin. At that time it will be swapped-out, whether or not it's idle. This helps if you forget to swap out, or know when your experiment will end. We send warning email an hour before your max duration expires, so you can avert disaster if necessary.

You may also notice that on the Experiment Information page the new fields are shown. On that page, you can edit those settings by using the "Edit Experiment Metadata" option. So you can turn the settings on or off, and adjust the times, for instance to change your max duration.

As always, we're here to help, so contact Testbed Ops with any questions, problems, or feedback about these changes, or other changes you'd like to see.

See how your fellow Emulab experimenters are using the testbed on this page of topology pictures.

In the Software section, we have added an ISO image that you can download and burn on a CD to give you a hard-diskless environment from which to run the frisbee client or the disk image creation tool. The README has more info.

In conjunction with our USENIX paper, we have released the source to the Frisbee disk loader and associated disk imaging tools. See the Software section for details.

Due to recent issues with disk space on the disk that holds all the /users directories, we've had to make a change in our quota policy. Quotas in home directories (/users) are now 50MB. Most users are already well under that in their current usage, but some people will need to clean up a bit.

Due to the disk space limitations we're under, our general policy is that /users should be used for your personal configuration information, and a small amount of space for experiment-related storage. Everything else should go in the larger /proj storage volume, where quotas are higher.

If you are over your quota, you'll get an email letting you know about it. The grace period is 7 days to get under the quota before you won't be able to use any more space. Besides deleting files, you may want to compress some of your files, do "make clean" where applicable, or move some of your files to storage at your institution or to one of the /proj directories.

To check your usage, run the "quota" command on ops.emulab.net or on an Emulab node.

There is now a list of papers incorporating research conducted on Emulab/Netbed, via the "Emulab Users" link on the left. There is also a list of classes that have used the Testbed. If you have any additions or corrections, please let us know-- we'd like to keep both lists current.

Our Emulab Tutorial has recently been updated and expanded. Some changes that may be worth checking out include:

• A new example that includes the use of LANs.
• A new example of using loops in ns files, especially for creating LANs.
• Improved text (and some links) regarding the difference between the "control net" and the "experimental net", and why not to use the control net for your experiments.

We're starting an effort to improve the Emulab documentation, partly based on preliminary results from the Survey. If you have other requests or ideas regarding documentation, please contact Testbed Ops and let us know!

Added the long desired halt/swap event directives. You can now put this in your NS file:

The first causes the experiment to terminate, the latter causes it to swap out. The units are seconds, as are all "at" statements.

NOTE: You can use halt (terminate) to cancel a batch experiment, but you cannot schedule a swapout.

I've installed the latest, greatest Frisbee (our multicast disk image distribution software). Gives us about a 5% improvement in performance, which means a speedup for swapping-in experiments which use custom images. See the ChangeLog for a summary of changes.

The Linux image has finally been upgraded from Redhat 7.1 to Redhat 7.3. Many users had requested a more recent version of Linux, and we finally found someone crazy enough (thanks Kirk!) to take it on.

Emulab has a new news system; news items less than a week old will show the icon next to their titles, as well as on the navigation bar to the left. As new features are added to the system or important events approach, we'll update the news on this page, so be sure to read new updates!

The first external site using our software to run their own Emulab is now up. Other sites are in the process of, or have plans to, bring up their own Emulab. If you're intrested in running your own, contact us!

uky.emulab.net, at the University of Kentucky, will be used primarily for classes, starting this fall.

Emulab now has much improved routing support which should make setting up routes in your experiments easier than ever! Please see the updated "Setting up IP routing between nodes" section of the Emulab Tutorial.