The SoftRelTools suite is mostly a set of short scripts, running under the Bourne Shell plus a set of makefiles, examples, documentation and, perhaps, most importantly a prescribed directory structure. The requirements on the users are to decide the details of the lower level parts of the directory structure and to provide relatively simple makefiles, for which examples and skeletons are provided. It uses GNU make (gmake), cpp for preprocessing and g++ for C++ compilation etc.
This document attempts to give the novice user enough information with simple instructions and examples to get started using the full system to manage their code development without a lot of detail. The detail is supplied by links to various sources of "official" documentation, most of which are listed above.
If, after being notified that your principal or key has been installed, you are still having problems,
Read and write access are controlled separately. Any bonified member of D0 is allowed read access. Write access requires read access plus authorization to write to a specific package. The request to add you to the access list for a particular package should be sent to d0-release-mgr@fnal.gov by one of the existing authors. Make sure that the message contains both your principal and your email address.
NOTE: you do not need read access to link against any of the currently supported releases of the D0 code (see setup and simple use below). You need this access only if you need to modify a fair amount of the released code. In the latter case, it'd be easier to use the developer tools described below. These require read access, and maybe write access to the repository.
setup D0RunII <version>
<version> specifies which particular release version you will be using. Unlike in the Run I system, there is no specific limit (other than disk space to store them) on the number of versions that can be available at any given time. If <version> is omitted, the release version currently declared as "current" will be setup.
To see what release versions are available, do:
setup D0RunII does a lot of things, including setting your $PATH etc to point to the appropriate release. To undo all of that, just unsetup D0RunII.
In addition to setting up a particular release version of the D0 Software (D0RunII) it is also possible, though not recommended, to setup a particular version of a package. This might be done to pickup an older version of a data file, for example. However, such a setup will not change the archive libraries that you link against. Those are a part of the release not the package. Before you setup a package, be sure you know what you're doing. To override the particular package version present in a release you'll have to use the developer tools described below.
In the C++ world, you can't "mix and match" versions of packages like we did with LIBTEST in Run I, without having to recompile (possibly large fractions of) the code. The developer tools and the SRT directory structure are designed to make this, if not easy, then at least doable within the lifespan of a graduate student. This recompilation isn't as bad as it once was, given today's disks and machines.
The <version> that you specify to setup is merely a name. We have developed a convention for these names that is aimed more at their eventual use in production releases than at their current use in a rapid development system. The <version> consists of four fields: a prefix and three two digit numbers, an example might be p11.03.01.
We currently have only a few prefixes:
In addition to specifying a particular release version directly. It is possible to use a "version alias" to specify the version. Currently only "current", meaning the latest release that is known to work, ie) has been in place for a while and "test" meaning latest completed release are available.
Individual code "packages" also have versions. These all have a prefix of "v" and the usual three two digit numeric fields. These fields have the same meaning as for a release. The only major difference between release versions and package versions is that the numeric fields are delimited by "." (dot) in releases and by "-" (dash) in packages. The latter is because cvs does not allow "." (dot) in tags, that is version names. This also helps to keep the two seperate.
Packages are just logical groupings of code that a single developer or a small group would work on. A package is the basic unit of development. There are as many ways to organise a package as there are developers. Some prefer very small packages, but a lot of them. Others prefer larger packages organised into subdirectories, or subpackages. Almost anything is possible.
Likewise, there are several ways to organize releases. The two main possibilies are:
The simplest way to use already released code is to directly use the main $SRT... environment variables from the command line, in your own scripts or makefiles. After having done the appropriate setup described above, you can use the following directory specs to find:
NOTE: Many of the "directories" are actually links to other directories. For example $SRT_DIST/releases/current is a link to a specific numbered version.
Compiling and linking from the command line can then be done using the
-I and -L compiler switches to specify where the compiler
and loader should search for include files and object code respectively. In
addition to anything that your code needs, you'd include:
-I$SRT_PUBLIC_CONTEXT/include/
to tell the compiler where to find all includes from this release and
-L$SRT_PUBLIC_CONTEXT/lib/$SRT_SUBDIR/libxxx.a
one per object library to tell the loader where to search for unsatisfied
externals.
The same lines should/could be put in any scripts or makefiles that you might use. We do not intend to provide ever provide such scripts. You are encouraged to use the more elaborate, but easier to use system discribed in the next sections.
setup the release version you wish to base your work upon and the
d0 cvs repository:
> setup D0RunII <version>
> setup d0cvs or setup d0sshcvs
At this point you can view the man pages via:
> man <command>
for each of the SRT commands mentioned.
Create a working directory, say "~/workdir" and change your working
directory into it:
> cd
> mkdir workdir
> cd workdir
You must now create an empty version of a release structure. This is known
as a local or private release:
> newrel -t <base-release> <new-release>
ie) > newrel -t t02.11.00 mytest
This will produce a directory <new-release> = mytest in your
working directory and an empty release tree below that. There will be no
actual packages included, even via links. Those are produced in the next
steps.
The string you specify as <base-release> is extremely important. This string is saved in a file called .base_release and is used during all gmake builds to find the release you are building against. This is independent of which version you actually have setup at the time you issue the gmake command. It is used as a "sanity" check. In C++ you generally can't "mix and match" different builds and parts of builds. In addition, gmake does not allow one to go backward in time easily. You need to be careful about this "stickyness" of the tools.
Change directory into your newly created <new-release>
directory:
> cd mytest
The first time you go through this, it's a good idea to check what files
have been created at each stage. So:
> ls -RFa
to see what's been created. Note, in particular, the makefile,
GNUmakefile that's been copied into your top level directory. This
file will steer all subsequent test builds of your work. Also, note the two
files .base_release which stores the <base-release>
information and .experiment which tells the procedures that you're
working on D0 code. You'll normally not need to modify any of
these files. However, you need to be aware that the contents of these files
do effect the actions of gmake during the builds. The string
in .experiment causes some D0 specific flags etc to be set.
The version listed in .base_release is the release version used in all
links. It overrides the version of D0RunII you have setup. This is
needed because switching versions of the base release can have unpredictable
consequences. If you must switch versions of the base release, you
should edit this file, putting in the new version number, then do gmake
clean to give you a clean slate so you can start over.
At this point:
> d0setwa
This initiallizes an empty rcp database and tells the system where
the top of your private build area is located. Whenever you change to a
different build area, you must cd to the top area and repeat the
d0setwa. Obviously, if you login a new window, you'll need to do
this in that window as well.
Now you use addpkg to fetch the code you'll be working on from the
CVS repository.
> addpkg [-h] <pkgname> [<version_string>]
for example:
> addpkg -h calorimeter
> addpkg -h xxx
the -h will fetch the latest or head version of the
code and is highly recommended for most use. Omitting the -h
will fetch the version that the base release was build on. The result
is "sticky". CVS "remembers" that you checked out a particular
version. If you make changes you'll have to put them back onto
a cvs branch, a topic beyond the scope of this document. See:
CVS Branch Usage and
Patching a production version of D0 code
for a general and a more specific discussion of this sort of usage.
If you're developing new code, you'll want to use the -h. But if you're trying to fix bugs or just need to link against a package version different from the one in the base release, either omitting the -h or specifying an explicit version is what you'd want. Check the files that have been created to make sure that you've gotten what you want. In particular, notice the file include/calorimeter. This is a symbolic (soft) link to the include directory of the calorimeter package. There will be one such link for each package that you've added.
Now, assuming that no other packages depend on yours, that
is, no other packages that you'll link into an executable #include
any of the header files in your package, and, if your code is fortran, you
haven't changed any calling sequences or include files that anyone else uses,
you can:
> gmake
> gmake test
to do some directory manipulations and then build your package object
libraries and/or executables. The gmake test runs the required
package tests.
Now you can add new routines, edit old ones etc to your hearts content. Run gmake at any time to update the object libraries and executables etc.
There are several gmake "targets" that you need to know about because you might want to use them:
We will discuss in the section on CVS fundamentals what to do with the code once you are happy with the results of your tests.
The question immediately arises, "How do I know that another package
depends on mine?" That's easily answered using the SoftRelTools
tool depend. Do man depend for details. In this case just:
>depend <mypkg> <version>
will list all packages in <version> that depend on headers
in <mypkg>
In this example we will discuss briefly how to solve this problem.
Follow the previous example: Simple code development situation through where you add the packages that you'll be modifying. At this point, you'll have actual copies of all the code you'll be modifying. You could go on and addpkg all of the dependent packages. However, this uses a lot of disk space and is unnecessary. Instead, you should now add links to the particular version of the dependent packages. These need to be added in two places:
gmake will now build the object libraries and any executables for the packages you've addpkg'd as before, but will also recompile and build the object libraries and any executables for the packages you've used lnkpkg on.
The rest is the same as in the Simple development example. The only difference now is that the first time you run gmake you will see your lnkpkg'd archives being recompiled and built. After the first time, any changes in your addpkg'd package may or may not provoke recompiling routines in the lnkpkg'd packages. You will also notice that mytest/lib/$SRT_SUBDIR and the corresponding /bin directories contain the object libraries and executables created from code in your lnkpkg packages.
NOTE: this is a case where it would be very advisable to
explicitly invoke the separate gmake targets:
> gmake lib
> gmake calorimeter.bin
The first gmake recreates all the required object libraries. The
second links only the executable for calorimeter instead of for
both calorimeter and cppreco, saving you some time since
you don't need the executables for that cppreco.
Some more helpful hints: setup d0cvs set's up CVS, so you don't
need to do that too.
> cvs -H gives you general usage info on switches
> cvs --help-command gives you a list of commands
>cvs <command> -H gives you usage info on a command
Once you've "checked out" a module, for example by running addpkg You just use any editor to make your mods. To make your changes permanent, if you don't have write access to the CVS module=package you'll have to send the code to the library czar. If you have write access you can use the following commands:
When you are completely done: > cvs release -d module_name from the directory *above* module_name to release your checkout and to delete the working directory tree.
NOTE: cvs is "sticky". If you checkout a specific version, it'll put changes back into a "branch" off that version, not the "head" or primary branch. It'll also remember that you used this particular CVSROOT. You'll occasionally see some funny results due to this, but only if you use more than one repository as I do.
For much more info on cvs than you can possibly want, see:
http://www.loria.fr/~molli/cvs-index.html
and links therein. In particular:
http://bing.rz.uni-ulm.de/cvs/train/cvstrain.html
It may seem superfluous to have a directory called pkgname within the pkgname product. But it does have an advantage. When testing the code, you can do your compilations and building from within the top level directory of the package, and find the includes. The search path will then be: -I./ which will find pkgname/include-name.
Therefore, a suggested directory structure would contain a GNUmakefile in the top level of the tree and subdirectories pkgname/ containing your include files, headers etc, src/ containing your source code, doc/, man/, data/ containing any data files needed, etc. NOTE: src/ could be a group of directories, or could itself contain directories for subgroups of your code.
Each directory that requires action during a build, must contain a
GNUmakefile. The makefiles should be copied from
$SRT_DIST/releases/current/SoftRelTools/GNUmakefile.example
and modified according to the instructions in the section on
GNUmakefiles.
In addition, we have decide to use the Apache Web Server's automatic directory index feature to present much of our code documentation. Therefore each directory in your package (except doc/ should contain a HEADER.HTML which is displayed as the pages header or a README.HTML which is displayed as a footer, or both. Between the two, they must contain the following:
The doc/ directory may, if you wish use this same mechanism to display your documentation. But it may also contain an index.html files. This is the default name of a file displayed if no filename is specified in a url. This give you complete freedom to use all the html power you need to adequately present your documentation.
See examplePkg for an example of all of these features.
In the releases/<version>/ directory you'll find a GNUmakefile. This file is put there by the release procedures. You should not change it. If you find that a change is needed, you must feed the change back to an expert for incorporation into SoftRelTools so that the change will be made for other test releases and for official releases. This file contains most of the make targets. It steers the build by walking through the subdirectory tree, invoking GNUmakefiles it finds there. So basically, it should never be necessary to modify it.
In each subdirectory in which resides code that needs to be compiled or otherwise processed. There must be a GNUmakefile to tell make what to do. These are the files that you are responsible for providing. More on these later.
The other makefiles that you need to be aware of, because you will
use them are the makefile fragments in SoftRelTools. You
can see them:
> ls $SRT_DIST/releases/current/SoftRelTools/*.mk
All of these should be modified (except for testing) by an expert only. They
are used by the SRT procedures during a release, from the
SoftRelTools release area. So any changes to them have to have been
released before they can be used. This requires action by the
SoftRelTools czar
These are of three sorts:
In each subdirectory in your product where an action needs to be performed when building a release, there needs to be a GNUmakefile that specifies the actions to be taken. Actions include continuing down a directory tree to other GNUmakefiles, compiling code, linking executables... Since these files are a part of each package, they must be put into CVS and are under it's version control, just like the code in the same directories.
There is an example GNUmakefile that can be copied into your working
directory, modified and saved along with the code. To get started;
> cd workdir/<test-dir>[/<subdir...>]
that is, change into the directory in your working area where some action
needs to be performed.
NOTE: you need at least one GNUmakefile in your package, in the top
level directory.
Then:
> cp $SRT_DIST/releases/current/SoftRelTools/GNUmakefile.example
GNUmakefile
Edit the GNUmakefile, defining the variables described below,
according to your requirements. These variables are just space delimited lists
of files and directories. Generally there should be no path
specified, just the name(s).
When starting to create you own GNUmakefile your best bet is to start form SoftRelTools/GNUmakefile.example.
The header of standard.mk which needs to be included in all of the user GNUmakefile outlines, briefly, the standard targets and variables. Following is a short description of what each of the variables does, how it's used and an example of it's use. Caveat: many of these I've never used, so am imfering their function from the code.
In the following, the ones that most people will need are: SUBDIRS, LIB, LIBxxFILES and perhaps BINS, DOCS and MANPAGES.