Current Code Release: February 24, 2004

Store: reco DSTs and mergerd thumbanils.

Do not store unmerged thumbnails....

mc_runjob should be installed as a ups product: version v06-02-01

NEWS: Upgrade to p14.05.01.
mc_runjob v06-02-01
Updated translate.py, macro.default, Defaults.py
Please do not store d0gstar events in p14.05.01

The tarballs to obtain are:

Minimum Bias Model (Unchanged)

This will be defined with a request see instructions below. The Overlap model has not changed in mcp14.

The Request ID is: 5257

Each overlap file generated MUST be declared to SAM.

Finally, Once you have generated all of your MB files you need to create a MB data set of them. This is done using a python script with SAM setup. An example script follows:

 import mcreq
defid = mcreq.createMinBiasDataset("filename1,filename2,filename3,....",
        "my_stations_mcp14_dataset",
        "dzero",
        desc="Lancaster Phase mcp14 plate Geometry Minbias Dataset",keywords=None)
print defid

You then just run the script with your filenames.

This then becomes the macro line 
   cfg d0sim define string MinBiDataset my_stations_mcp14_dataset
 
An exmple macro can be found here (note only have to run up until d0gstar stage)

SAM

Environment Variables

The environment variables need to change now that runjob is a UPS product. There are now less environment variables required.

. /d0/fnal/local/etc/setups.sh
export SRT_LOCAL=$MC-SOFTWARE-DISTRIBUTION # directory containing mcc-dist
cd ${SRT_LOCAL}
cd mcc-dist
source mcc_dist_setup.sh
cd ..
export BFCURRENT=p13.08.00
setup mc_runjob

Cardfiles

To add a new version of the cardfiles implement the following (this should always work):

$MC-SOFTWARE-DISTRIBUTION # directory containing mcc-dist
mkdir -p $MC-SOFTWARE-DISTRIBUTION/packages/cardfiles/vxx-xx-xx
cd $MC-SOFTWARE-DISTRIBUTION/packages/cardfiles/vxx-xx-xx
setup d0cvs
cvs checkout -r vxx-xx-xx cradfiles
mv cardfiles/* .
rm -fr cardfiles

How to use the Request System:

1. You will need an up to date version of SAM to extract the requests
2. Requests will be delivered via a request number.
3. To get the request detaiuls for request ID you need to run the command:
   sam get request details --request-id=1158 --dictfmt > Request_1158.py
4. Run: python translate.py Request_1158.py (needs macro.default and Defaults.py)
5. When you have started running the job carry out the command
   sam modify request --request-id=1155 --status=running
6. When the job is complete run the command
   sam modify request status --request-id=1155 --status=finished
   
   

Storing Metadata

Some changes have been made to the location and content of metadata. The metadata is now stored in the same directory as the file is written to. This will allow us to store the file in SAM as part of rujob in the near future.

Each stage of processing only contains the metadata related to that stage of processing. I.e. d0gstar files will only have global, and simulated metadata.

The thumbnail file can now be stored. We want you to store three files at fermi for now:

• d0reco (DST)
• Thumbnail (Merged see next section)

An example directory structure looks like:

d0gstar:
d0g-p13.04.00_CAEP-off_Geo-plate_Iain-Bertram_algo_recocert_lancs_2404_02347172401
import_kw_d0gstar_d0g-p13.04.00_CAEP-off_Geo-plate_Iain-Bertram_algo_recocert_lancs_2404_02347172401.py
 
d0reco:
import_kw_d0reco_reco-p13.04.00_Iain-Bertram_algo_recocert_lancs_2404_02347172401.py
import_kw_d0reco_tmb-p13.04.00_Iain-Bertram_algo_recocert_lancs_2404_02347172401.py
reco-p13.04.00_Iain-Bertram_algo_recocert_lancs_2404_02347172401
tmb-p13.04.00_Iain-Bertram_algo_recocert_lancs_2404_02347172401
 
d0sim:
import_kw_d0sim_sim-p13.04.00_Noise-on_NMB-0.0_MB-Fixed_Iain-Bertram_algo_recocert_lancs_2404_02347172401.py
sim-p13.04.00_Noise-on_NMB-0.0_MB-Fixed_Iain-Bertram_algo_recocert_lancs_2404_02347172401
 
generator:
gen-pythia-p13.04.00_Dec-incl_NumEv-1_Prod-qcd_Iain-Bertram_algo_recocert_lancs_2404_02347172401
import_kw_pythia_gen-pythia-p13.04.00_Dec-incl_NumEv-1_Prod-qcd_Iain-Bertram_algo_recocert_lancs_2404_02347172401.py                        

If you have any problems with this please let me know.

Merging Thumbnails

Thumbnails are very small files and cause problems if stored in the tape robot. For this reason we need to start merging thumbnails. The latest version of runjob has the capability of doing this (it may have a few bugs). I suggest merging all requests into 20k subsections in which all the thumbnails will be merged.

To have automatic merging you need the following in your macros after the d0reco stage:

attach d0reco
cfg d0reco define string D0Release p13.08.00
cfg d0reco define int UseMaxopt 1
cfg d0reco define string LinkToRundata on

attach merger                                                       # Attach a sectioon which tracks all simlar jobs
cfg merger define string MergeLocation /info/Transfer/Request-5257  # NFS mounted disk seen by all worker nodes
                                                                    # Should be different directory for all requests
cfg merger define string RegistryName gen-merge                     # this will caintain list of all files to be merged
cfg merger define string MergeFilenameTag ThumbName                 # A states to merge thumbnail rather than DST output

attach copyd0om                                                     # This carries out the merging when the last job in 
                                                                    # the gen-merge.registry file is finished
cfg copyd0om define string D0Release p13.08.00 

The output of the merging will be stored in the ./dest/copyd0om directory of the last job to complete. You will have to declare all of the thumbanil files produced by ach seperate job. Then store the merged thumbnail file.

Iain Bertram
Tuesday, February 24, 2004