D0 GROUP LEVEL3
ONLINE LESSONS

Episode VI
Using the D0 Framework and Building an Integrated Test

The framework documetation is contained on http://www-d0.fnal.gov/run2_offline_software/framework/framework.html

Component testing, testing the code in isolation, is a good idea. However, there comes a time when code must be tested with with D0 data. Using the D0framework for this task is the easiest way of accomplishing that. To this end, I have an example of making an integrated test within the CTEST enviroment.

The example that I have prepared has minimal content. In this test, I have made a package that gets CalData from an MC file and out prints the cells.

Recall that your components will be called by L3 tools, not by the offline framework--so make sure that your framework packages are just a shell that calls the things that will actually do the work. In point of fact, that is what should be happening with your offline code as well, although there the interface is perhaps better defined.  Give careful consideration to the interfaces; at level 3, we'll want to minimize access to the event.

I'm going to try to ease up on the step by step command line instructions, starting with this episode, because by now you should be familiar with the basics. And this episode is going to involve typing out lots of files, without much action.
 

The example exist in the CVS repository and can be checked out by
addpkg -h tutorial_integratedtest

see Episode 1 for setting up a release area

In order to keep the components separate from the framework so that the package does not depend on framework, the source code for the integrated test is in the tutorial_integratedtest/test.  In this example, there is no component to test, since what little work is being done is being done directly in the test.  If you are writing a package for offline,  the framework dependent code would be in the /src area.

When you look in
tutorial_integratedtest/src, you will see another CTEST directives file called OBJECT_COMPONENT. In order for framework packages to work, they must be registered and the registration file has be a bare object, not part of the library. In this case the registry files are called RegisterITestPkg.hpp and RegisterITestPkg.cpp.

The integrated test can be seen in tutorial_integratedtest/test. There are three CTEST directive files, ITESTS, LIBRARIES, and OBJECTS.
ITESTS contains the names of the integrated tests. In this case, it is something called ITestPkg. There is an ITestPkg.cpp file which contains the code. ITestPkg.sh is a shell script file that copies the data file and the framework rcp file to the itest area and runs the test. The results end up in tmp/IRIX6-KCC/package/itest/testname. This is analogous to what we have seen with component testing. Itest_framework.rcp is a special rcp file that gives directives to the framework, including the rcp files for the package being tested and the ReadEvent file.  The other rcp files can be found in the /rcp directory.

What MC seems to exist is listed on
http://www-d0.fnal.gov/newd0/d0atwork/computing/MonteCarlo/production_doc_062698.text

LIBRARIES should contain the libraries on which your test directly depends.   Remember to include the library you are testing.  The LIBRARIES file is distinct from the package LIBDEPS file in that the LIBDEPS should only contain the package's dependances, while LIBRARIES contains the test's dependances.  For example,  in L3, your tool packages should not depend on framework, but the integrated tests certinaly will.   OBJECTS contains the bare object files for the framework packages (as opposed to the SRT packages). For some reason, OBJECTS is not automatically put into the area by ctnewpkg.

To create the integrated test:

<d0mino> gmake tutorial_integratedtest.lib
<d0mino> gmake tutorial_integratedtest.test

The lib step is necessary to build the bare object file.

When your package is releases, the test will run as part of the build.  If the test fails, you'll get a error message from the release managers.

If you want to build a executable which is not transient (the itest exe is deleted on sucessful test completion), follow the procedure as above in the /bin area
 

Because the tests run as part of the release, they are silent with all output redirected.   You can see the output in /tmp/BFARCH/tutorial_integratedtest/itest/ITestPkg/ok.out
 

Another framework package that you might want to play with is analyze. This is Serban's package for looking at MC data. To use it, get it out of the release by going to your release test area  and do

<d02ka> addpkg -h analyze

This will get analyze out of the library and stick in your area. to build

<d02ka> setup cern
<d02ka> gmake analyze.all

That will make analyze.x (an executable) in the analyze/test area. Yes, I know that's not where I told you to put executables. This package does not use CTEST (obviously) so to figure out what's going on during the build you'll have to look at the GNU makefiles. But remember that for Level 3 filter developement, component testing is nonnegotiable, so don't pick up any bad habits.

Addendum: by Gustaaf Brooijmans

The basic Analyze code doesn't need to change, but I modified it so it complies with the recommended use of the HepTuple package - addressing histograms and ntuples through references rather than pointers. The only other change is that I moved out the framework registration line to a new file following Amber's model. According to Amber that change is mandatory. The component test tests the statics and constructor and the integrated test runs on a few MC events. See episode VI to set up an integrated test.

I kept Serban's way of running it too although there's a small Makefile clash doing that - the new Makefile supports both methods but that leads to a lot comments in the style of "overriding commands...". The prelinker comments are unrelated but I'm not sure what their meaning is. They don't contain the word warning or anything worse so I just ignored them.

One last comment is that when you set up an integrated test, do not forget to add the package name in the LIBRARIES file (in addition to the libraries also used in the component test).

All the stuff can be found on d02ka in directory /d02ka/home/room1/gusbroo/l3test/example/analyze/

I you have any questions - fell free. (References to d0chbreplaced with d02ka)