Tutorial: Basic DØ Data Analysis

Page updated June 14, 2005

Content and Purpose

Global references:
D0 At Work > Computing & Core Software > Software and Documentation
D0 At Work > Physics Groups > Common samples

Other more complete tutorials:
Software and Documentation > DØ "How To" Documents

Marco's software tutorial, Reiner's software tutorial, Reiner's d0tools tutorial,
Heidi's data access tutorial
In the following, what is written in green has to be typed on clued0.
You can find here the list of slides used below in PDF format.


  • log on a clued0 machine and go to your working disk (need several 100Mb)
    create there a tutorial directory

    (clued0 is the DØ linux cluster)
DØ computer accounts

DØ reconstructed data

  • What is a DØ reconstructed data ?
EDM tutorial
fixing, skimming, d0correct
CAF page
  • Where are the DØ data located ? In SAM !
Heidi's tutorial
For MC,
look at: D0 Official MC Production Catalog
> setup sam
> sam translate constraints --dim="run_number 205213"
> sam translate constraints --dim="run_number 205213 and data_tier thumbnail"
> sam translate constraints --dim="run_number 205213 and data_tier thumbnail \
   and file_name recoT_all_0000205213_mrg_%.raw_p17.03.03"
> sam get metadata --file=recoT_all_0000205213_mrg_097-099.raw_p17.03.03
> sam create dataset definition --defname=[your username]-205213-reco --group=dzero --dim="run_number 205213 \
    and data_tier thumbnail and file_name like recoT_all_0000205213_mrg_%.raw_p17.03.03" 
> sam translate constraints --dim="__set__ [your username]-205213-reco"

DØ releases

How To Use the DØ Software Releases,
CVS pages

DØ packages

How To Use the DØ Software Releases,
D0 CVS Repository Browser,
D0 Code Browser,
Marco's software tutorial,
Reiner's software tutorial

Running an executable

This tutorial uses the p18.01.00 release which is the recommended release for d0correct at the time of writing. Please replace p18.01.00 by the release version currently recommended for running d0correct.
  • d0tools

  • The is a tool that allows you to run every framework program (with the generic command rund0exe).
    Several specialized scripts exist for standard D0 programs:
      runreco for d0reco, runTMBAnalyze for tmb_analyze.

    Here we will use d0tools to run TMBAnalyze_x and create CAF trees.
Reiner's d0tools tutorial
  • Running the default version of TMBAnalyze_x to create CAF trees
  • > setup D0RunII p18.01.00 -O SRT_QUAL=maxopt
    > setup d0tools
    > runTMBAnalyze -h
    > cp /work/broglie-clued0/deliot/data/p17/CSskim-MUinclusive-20050508-001120-825095.raw_p17.03.03-first .
    (or any other thumbnail input file)
    create a file input.list with the path and name of the input file you just copied.
    > which TMBAnalyze_x
    > runTMBAnalyze -filelist=input.list -name=outputDir -fwkparams -num_events 10
    > runTMBAnalyze -filelist=input.list -name=outputDir -num=10
    look at the content of the CAF tree you produced (see Root section in this tutorial)
    located in the ouputDir directory.
  • Running a customized version of TMBAnalyze_x

  • We will customize the running of TMBAnalyze_x by changing values in RCPs.
    We need a local copy of the tmb_analyze package and a release working area for that.
    > setup d0cvs
    > newrel -t p18.01.00 work
    > cd work
    > ls -altr
    > d0setwa
    (specify your working area to the framework)
    > ls -altr
    > addpkg tmb_analyze
    > ls -altr
    > more tmb_analyze/CVS/Tag
    > cd tmb_analyze/rcp
    > cp /d0dist/dist/releases/p18.01.00/tmb_tree_maker/rcp/TMBCorePkg.rcp .
    (customize the content of the CAF tree by changing the TMBCorePkg.rcp file)
    > chmod u+w TMBCorePkg.rcp
    edit the file TMBCorePkg.rcp and modify it or take the one I already modify at:
    modify the main framework RCP to use the TMBCorePkg.rcp located into tmb_analyze/rcp:
    > cd ../..
    > ln -s ../input.list .
    > which TMBAnalyze_x
    > runTMBAnalyze -filelist=input.list -name=outputDir2 -localfwkrcp -localrcp \
        -fwkparams -num_events 10
    look at the new content of the CAF tree you produced (see Root section in this tutorial)
  • Running your local version of the TMBAnalyze_x executable

  • If you need to modify the code itself (or if your program is not built in a release),
    you will need to build a local version of TMBAnalyze_x.
    > cd tmb_analyze/src
    edit the file TMBTRefsPkg.cpp and add a printout in the constructor of the TMBTRefsPkg class.
    (method: TMBTRefsPkg::TMBTRefsPkg (Context * cntxt))
    > cd ../..
    > gmake all
    > which TMBAnalyze_x
    > runTMBAnalyze -filelist=input.list -name=outputDir3 -localbuild -localfwkrcp \
        -localrcp -fwkparams -num_events 10
    edit outputDir3/TMBAnalyze_x.out and search for your printout !
  • Running on a SAM dataset on CAB

  • CAB stands for D0 Central Analysis Backend. It's a fast cluster of machines to run analysis.
    You can only run batch job on CAB.
    > runTMBAnalyze -defname=[your username]-205213-reco -cabsrv1 \
       -scratch=[your scratch area] -name=outputDir4 \ 
       -localbuild -localfwkrcp -localrcp -fwkparams -num_events 10
    > qstat -u [username] @d0cabsrv1.fnal.gov
    To run on batch on clude0: cf. d0tools help (runTMBAnalyze -h)
CAB page
Marco's software tutorial,
Reiner's software tutorial
    List of main chunks in a thumbnail after unpacking:
    (the uncertified chunks are created by d0reco, the certified ones by d0correct)

    Object Chunk name Number of chunks per event
    tracks ChargedParticleChunk 1 chunk by event
    vertex VertexCollChunk several chunks by event
    electrons/photons EMparticleChunk 1 chunk by event
    EM chunk documentation
    muons MuonParticleChunk
    1 uncertified and
    1 certified chunks
    D0Note 4735
    taus TauChunk 1 chunk by event
    jets JetChunk uncertified chunk
    one chunk per jet algorithm
    D0Note 4457
    CorrJetChunk certified chunk
    one chunk per jet algorithm
    missing Et MissingETChunk 1 chunk by event
    D0Note 4474
    L3 trigger infos L3Chunk 1 chunk by event
    L1L2 trigger infos L1L2Chunk 1 chunk by event
    L1L2Chunk documentation

    + other chunks containing hits (D0Note 4362).
    See also list of chunks.


  • some very basic stuff to open a root file and draw some variables

  • (need a setup D0RunII first before you can use root)
    > cd outputDir
    > root -l
    > root[] TFile f("CSskim-MUinclusive-20050508-001120-825095.raw_p17.03.03-first.root");
    > root[] TBrowser t;
    > root[] gDirectory->ls();
    > root[] TTree* tr = (TTree*) f->Get("TMBTree")
    > root[] tr->Draw("Muon._deteta");
    > root[] tr->Draw("Muon._deteta","Muon._isMedium==1");
    > root[] .q
Root page
root/paw translator

Luminosity calculation

Accessing Luminosity Information,
    As example, we will compute the luminosity for our example dataset above.
    Here are the different steps.

  • Generation of the luminosity parentage infos
  • > cd work
    > setup sam -o -q prd
    > addpkg lm_access 
    > python lm_access/scripts/makeDatasetParentageList.py [your username]-205213-reco
    > ls -altr
    If you use an official skims, you can save time by using the centrally produced parentage files: see Marco's D0 news

  • Create your specific lists of bad LBN and bad Runs if needed

  • Look at the data quality infos of the objects you need: D0 Data Quality Coordination
    and at the data quality database: Offline Run Quality Database Queries
    create a file badRunsList and badLBNList
  • Create a list of input file
  • > sam translate constraints --dim="__set__ [your username]-205213-reco"
    create a file list.file
  • Create a run range file

  • with a list of triggers and run range
    create a file runrange.list with:
    MUH5_LM15 0 999999
  • Compute the luminosity
  • > runrange_luminosity file.list . runrange.list
    > ls -altr
    You will have to skip the bad LBN listed in the output file reconstructed_file.list_MUH5_LM15_0-999999.badlbns
    during your events selection (cf. lm_access).

This page maintained by Frédéric Déliot