D0 UPS/UPD installation

This document describes how to do the initial installation of UPS, UPD and Perl and the initialization of the D0 specific ups databases that are needed to allow for remote distributions of the D0 code to your site.

We are using Fermilab's ups/upd (Unix Product Support and Unix Product Distribution) to maintain and distribute the multiple versions of our code. ups/upd allows a large number of variations on how it is installed, and on how the code it is managing is installed. This document describes only one such, essentially the one we are using here at D0.

The steps you'll need to take in order to setup your site to be able to receive remote distributions of the D0 software are:

Once this is done, you use ups and upd to pull over the products you need, including the D0 code .

  1. Organization

    At D0 we maintain three ups databases:
    1. <main-ups-dir> contains those products maintained and distributed by Fermilab from fnkits.fnal.gov . These include products such as ups, upd, perl, cern, python etc.
    2. <d0prod-dir> are those products that are similar in nature to the <main-ups-dir> products but are maintained and distributed by D0. These include products like ace that haven't yet gotten into fnkits and special versions of the main products that we maintain.
    3. <d0dist-dir> is the real D0 code base.
    The <main-ups-dir> and <d0prod-dir> areas could be combined. But we believe that it makes for simpler maintenance to keep them separate. Their source and therefore the command needed to update them are slightly different. <d0dist-dir> must be separate since it has a different directory layout. Therefore we recommend that you organize your db's into three as well. These three areas can all be on one physical disk, or be separated.

  2. UPS/UPD/Perl installation

    If you've installed a Fermilab version of your OS, this part is almost certainly already done. If you can "setup ups" then go to the next section. But if you must first install ups/upd and perl then follow the instructions here.

    The full documentation for ups and upd is at http://www.fnal.gov/docs/products/ups/ . But a very easy to follow bootstrap procedure can be found at ftp://ftp.fnal.gov/products/bootstrap/current/index.html . I used the "Automated install on a UNIX system with a user defined configuration" prescription to create an installation in my home area that is independent of any that are already on the machine. I just had to use the "-F" option when running the "stage1.sh" script.

    Once done with that, you'll have a directory with the following in it:
          catman/ db/ etc/ man/ prd/

    1. db/ is the actual database,
    2. prd/ is the code area,
    3. etc/ is the "startup" area.
      It contains setups.csh and setups.sh scripts that users must source in order to bootstrap the ups setup. The appropriate one can be sourced in their login scripts or whenever they need ups.
    4. catman/ and man/ are man page areas used by some products.
    This directory will be referred to as <main-ups-dir> below

    At this point you have ups/upd and perl installed and you have an operation ups system. We'll tell you how to access it later.

  3. D0 specific ups databases

    We now add the other two, D0 specific, databases and modify the main one to make those two usable.
    1. mkdir <d0dist-dir>
    2. cd <d0dist-directory>
    3. Using a web browser, download and save:
    4. tar -xvzf UPSd0dist.tar.gz
    5. ./fix-d0dist.sh
      answer "y" is it's OK to change the files.
    6. Repeat the above for <d0prod-dir> area using

  4. Modify main database

    Now you need to modify the original (main) database to make the two new ones available.
    1. edit <main-ups-dir>/etc/upsdb_list
      add the two new db's, usually after the main one. You can add as many databases as you wish here, even ones that don't exist. ups will look through them at startup and will only use ones that exist. The new file might look something like:
      Example upsdb_list
      NOTE: only the 3 uncommented lines are really necessary.
      NOTE: the two D0 ones should end in "/upsdb"
    2. cd <main-ups-dir>/db/.updfiles
    3. download and save:
      http://www-d0.fnal.gov/software/cmgt/example_updconfig
    4. edit the saved example_updconfig. Look for the FOUR occurrences of "UPS_THIS_DB". You'll need to change the values of all four of those.
      • The first two should point to <d0dist-dir>.
      • The next one should point to <d0prod-dir>.
      • The last one should point to your main database, <main-ups-dir>/db normally. This one should be the same as in the existing updconfig. This one is the default if upd's search doesn't match one of the previous sections.
    5. grep UPS_THIS_DB example_updconfig
    6. ls -a each of the four directories found. This is to check that you haven't made a mistake.
    7. save the existing updconfig for safety
    8. mv example_updconfig updconfig
    You now have a functional, though empty D0 ups/upd installation.

  5. Accessing the ups databases

    Before you can use ups to "setup" products, you have to setup ups. In order to do this, you need to know where it is. In particular, you need to know where <main-ups-dir>/etc/setups.sh or .csh is. That script bootstraps setup. All you need do is:
        source <main-ups-dir>setups.csh
    or
        . <main-ups-dir>setups.sh
    depending on your shell family. This can be done in your login scripts. Once this is done once the variable $SETUPS_DIR is defined to point to the etc/ directory and can be used in any executed scripts that need the "setup" command or other ups aliases.

NOTE: To support D0 code you must modify <main-ups-dir>/db/.updfiles/updconfig . If you have installed any of the databases in a non-standard location (didn't use the predefined configurations) you must modify <your-ups-db>/.upsfiles/dbconfig

  • from ftp://fnkits.fnal.gov/products/ ups/ , upd/ and perl/
    look for the latest version, then the "flavor" either the "NULL" flavor (for upd) or for your machine type for the others.

    You can save the tar files using your web browser, but in order to use upd or anonymous ftp you'll need to be registered. These programs are freely available for anyone in the high energy physics community. But you do need to fill out and email the UPD Registration Request form to me at d0-release-mgr@fnal.gov . I'll get you registered at the FNAL kits site (fnkits) as well as at D0's distribution site.

    If you have or will have only a single "flavor" of operating system reading from this disk, unwind them into: /usr/products/ups/v4_x or the directory appropriate for the version you pulled. If you'll have multiple "flavors" using that disk then add a further level of subdirectories named for the flavor you are pulling (ie IRIX+6/). The latter is assumed in several of the examples below.

  • Do this for each of the three db's, /usr/product/upsdb/... Only the main db needs all of the subdirectories. All of them need the .upsfiles/ and the .upsfiles/statistics/ (if you want to collect statistics, we don't) directories.

  • I hope you won't be doing this!

  • or this!

  • The version specified should be the one you pulled over.

    NOTE: you shouldn't need to edit any fermi files. In fact we are not recommending that you install them. You can, but don't need to. For our purposes all you need do is to tell your users who want/need to use the D0 software to put one of the following lines into their .login or .profile files: 

        > source <your-setups_dir>/setups.csh
or
    > . <your-setups_dir>/setups.sh
  • won't need this

  • probably won't need this unless you want ups to startup things like web servers. We do it and it's rather convenient, but not necessary.

  • this is just to keep people from having parts of both ups v3.x and ups v4.x in use at the same time. A new installation doesn't need it.

  • shouldn't be need.
    Alan Jonckheere
    Last modified: Wed May 11 15:20:59 CDT 2005