Minutes for Analysis Tools Group 19 November 2002

• Minutes for Analysis Tools Group 19 November 2002
  • Streaming (Jon)
  • Luminosity Logic (Heidi)
  • Implementation (Jeremy)

Streaming (Jon)

Discovered that an init is not sent to the L3 nodes. Script runner assumes that an init is sent. For test from last week, check Michaels page. Physical stream name is now in the L3 header. If a physics run and a commissioning run are going simultaneously, then some nodes will ignore level 2 (when a node is reallocated). Have patched p12 of level 3 to work around this problem. The real fix is to get the supervisor to send an init when it starts a new run.

Jon wants non-recorded runs in the run database. But Scott does not write out a BRUNS file for non-recorded runs.

Once streaming is ready to go - then we need to do monitor and error. What if an event has an unpacking error in, say CFT, preventing certain triggers but not others. Should these go to a monitor stream -- should list the different types of errors.

Try the monitor stream out!

Luminosity Logic (Heidi)

The specification may be found at http://www.nuhep.northwestern.edu/~schellma/luminosity/lm_access_spec.htm.
Below are my quickly written notes on the discussions.

Must not and triggers for an analysis. Only OR triggers. (I guess the idea is that if you have to AND triggers to get what you want, you should have designed a trigger that had what you wanted).

For a stream -- must make sure that other streams were reconstructed with an "approved" verison of reco. The idea is to check that files in a set are recoed with the same version of reco.

Data quality per run (and per block) -- meant for killing last part of a run when there's a Muon HV trip [this is in the luminosity database which is not present yet].

Luminosity quality preference - Keep or chuck suspect blocks.

As an option, user can provide a list of runs or a run range (Greg Landsberg case where run range is the whole experiment and there's one event in the file).

Also add a user exclude list of lum blocks.

Database server calls do not exist yet.

Cannot OR a prescaled trigger with a nonprescaled trigger -- can't figure out lum.

Runs DB server handles the streaming tables.

What if two simultaneous runs with same trigger and same stream?-- only one will be called physics and that's the one that has luminosity (non-physics runs won't get lum determination -- does that cause problems? ).

Lum assumes that people will merge runs and merge streams (though we don't want people to merge streams) -- need administrative constraints on SAM metadata.

Implementation (Jeremy)

Jeremy's talk may be found here.

Where does luminosity logic sit? (Trigger and SAM servers exist, Runs server does not exist, lum is in prototype). User loads a thin client library and talks to...

Options:

1. Central python analysis server (ITC front end and Corba back end). The CORBA is a client to the DBServers. ITC is in DØ release and Michael knows it well, but not many others. (ITC is wrapped socket TCP/IP). Only one ORB necessary (since all python).
   
   
2. Same as above but with a CORBA front end. But we've never had a program that was a CORBA client and server (and a client to mutiple servers). Note that we don't have that much CORBA experience. Would need two ORBs - one for the "backend" with python-python and a different one on the frontend for C++-python (Orbacus -- don't have much experience with that).
   
   
3. Have a centralized server, but talk directly to the database, bypassing other DB servers. Problem is maintenance issues -- if a DB changes its schema then we have to change the server. Duplication of effort (since other DB servers need to be maintained as well). This is the easiest up front, but is hard to maintain.
   
   
4. Worst option is to have no central server. So each job talks to all of the DB servers.
   
   

Jeremy likes the second option (Corba front and back end). Can do caching easily. Gives abstraction and ease of maintenance. Also easier for people offsite to use. But this is more difficult to write initially due to our CORBA experience. Jeremy sees other uses for this "Analysis DB Server". For example, the SAM dimension editor could check files against good luminosity and not return runs that have all bad lum blocks.