Evaluate Streaming

Adam Lyon and Jeremy Simmons

Current version is v20020806

The evaluateStreaming tool allows you to try different stream schemes and see the effects on events.

Send questions/comments/problems to Adam Lyon at lyon@fnal.gov

See here for XML files.

Setting Up

The evaluateStreaming tool has been installed on d0mino and clued0 nodes. Execute “setup evaluateStreaming” to make the tool available to you.

If you are on a windows machine, may want to install the code there (I find the tool runs best on my WindowsXP laptop). See here for instructions.

XML Files

The evaluateStreaming tool works from XML files that hold data the tool needs. If you look at one, you’ll see that their just text files with lots of “tags”, or specific code words within angle brackets. There are three types of XML files that the tool uses:

Stream Scheme XML: Contains the definition of a particular stream scheme tree. You are using the evaluateStreaming tool to write and evaluate such an object. When you are done, save your scheme to an XML file – it can be sent directly to the online system for use in level 3.
Events Info XML: For the evaluateStreaming tool to work, it needs information about triggers in a run (or set of runs). Specifically, it needs to know what primal streams are assigned to what triggers and the trigger pattern frequencies (how many events have a particular pattern of passed triggers). You can create such an XML file by running the eventsToXML tool described elsewhere, or get already created files at the xml web page.
Override Primal Stream XML: The evaluateStreaming tool allows you to change the primal stream assignments (you can add your own and alter the existing assignments). You can save these overrides to an XML file so you don’t have to type them in later.

The XML files are complicated to create and examine – that’s why the evaluateStreaming tool was written to do this work for you!

The Analysis Tools Group will make some Events Info XML files available on the web. See here. There will also be some Stream Scheme XML files there for you to play with, including the template.xml file for making a new stream scheme.

Launching the tool

To launch the tool, you must provide an initial stream scheme XML file and an events info XML file (see above for an explanation of the XML files and where to find some). You may optionally provide an XML file with primal stream overrides as well. For example:

evaluateStreaming xml/WZScheme.xml xml/run159875.xml xml/WZPrimals.xml

(The above example assumes that all files live in your xml sub-directory). The last file may be omitted if you don’t want to use saved overridden primal streams (you can always override later with the tool).

Note that you can also use web-urls in place of any file name (if course you will have to save your work to a file on disk). For example:

evaluateStreaming xml/WZScheme.xml \
http://www‑d0.fnal.gov/computing/analysis_tools/tools/evaluateStreaming/xml/run155554.xml

You can only input web URL’s from the command line.

Be aware that it may take awhile for the tool to load all of the event information. On d0mino, it takes many seconds to just start python and then many more seconds to load the event information. On clued0, python starts nearly immediately, but it can take several seconds to load in the events information. On a reasonably fast Windows machine, the tool starts in about 2 seconds. Once you see the Stream Scheme displayed, the tool should respond fairly quickly.

If you type in evaluateStreaming with no arguments, you will get a usage message. If you see strange looking python errors, one of the files you entered probably does not exist.

Reading and Editing the Stream Scheme

Once the main window appears, you’ll see the stream scheme displayed. The primal streams are shown in all lower case and end with a question mark (since each is a “decision node”). Note that up green lines are "Yes" (follow if primal stream is satisfied) and down red lines are "NO" (follow if primal stream is unsatisfied). The upper case words are the physical streams (where events will end up following the tree). Note that the naming scheme for primal and physical streams is in no way set yet (and in fact putting the names of the passed or failed primal streams in the name of the physical stream is probably impractical for online use). What you see here is just what I find convenient.

Right clicking on the name of a decision node (primal stream) or a physical stream brings up a little menu of options. If you right click on a decision node, you can change the primal stream to another one, or turn the decision node into a physical stream, dropping any children nodes. If you right click on a physical node, you can rename it (give it any name you like – it’ll be converted to all uppercase though) or change the node to a decision node.

You can build a stream scheme by changing physical streams to decision nodes, working your way down the tree.

Note that as of yet, there is no undo, so save your work often.

Override Primal Streams

If you find that the primal streams given in the trigger list are not sufficient (e.g. you need to change electron to elehi and elelo), you can override the primal stream assignments. Go to the View menu and select "Override Primal Streams"). A list of the triggers and their primal streams is displayed. Any altered triggers are shown in red. Left click on any block of text to alter a trigger's primal streams. Then input all of the primal streams for that trigger as a space delimited list. You can also save primal stream overrides to a file and load previously saved overrides back in. The "Drop All" button drops all overrides, restoring the trigger list to its original state. Note that when you click on “Ok” to apply the overrides, you will be asked if you want to save the overrides to a file if you haven’t done so already. The overrides will be applied whether or not you save to a file.

Viewing Results

To see the overall effects of the stream scheme on the events, go to the View menu and select "Results". A text box with al relevant information will be displayed. You can save the information to a file if you wish.

Information about Trigger Patterns

If you want to see where events for your analysis triggers go, go to the View menu and select "Trigger Information". A selection box opens up for you to choose the desired trigger logic. Build the trigger logic by selecting triggers from the list on the left side. Triggers chosen together with CTRL-Click are AND'ed. Triggerss chosen separately are OR'ed. Here are some examples to illustrate what you do:

Your analysis deals with the 3EM_MN Trigger: Click on "3EM_MN" and click Add.

Then click Ok to see the results.

Your analysis requires that events pass both 3EM_MN and JT_95TT: Click on 3EM_MN

and then CTRL-Click on JT_95TT (this selects both of them). Click Add and Ok.

Your analysis requires that events pass 3EM_MN *or* JT_95TT: Click on

3EM_MN and click Add. Then click on JT_95TT and click Add. Then click Ok to see

the results.

Your analysis requires that events pass (3EM_MN *and* JT_95TT) or pass

EM_MX: Click on 3EM_MN and then CTRL-Click on

JT_95TT. Click Add. Then click on EM_MX and click Add. Click Ok to see the results.

Batch Mode

You may also run the tool in "batch" mode to get results without using the GUI. Use the "-b" option switch. If you wish, you may provide a trigger pattern set to evaluate. The trigger pattern is placed at the end of the call and is written as follows:

The entire pattern must be surrounded by '[' and ']'
Trigger logic is then placed within subsequent brackets
Trigger names must be surrounded by double quotes and spelled exactly correctly
Triggers within the same [] are AND'ed
Different [] sets are OR'ed

Some examples:

Your analysis requires that events pass the EM_HI_SH trigger:

trigger logic pattern = '[ ["EM_HI_SH"] ]'

Require EM_HI_SH *AND* EM_LO_SH triggers

trigger logic pattern = '[ ["EM_HI_SH", "EM_LO_SH"] ]'

Require events pass EM_HI_SH *OR* EM_LO_SH triggers

trigger logic pattern = '[ ["EM_HI_SH"], ["EM_LO_SH"] ]'

Require events pass ( EM_HI_SH *AND* EM_LO_SH ) *OR* pass (MU_LO) triggers

trigger logic pattern = '[ ["EM_HI_SH", "EM_LO_SH"], ["MU_LO"] ]'

Make sense?

An example call would be

evaluateStreaming -b xml/topScheme.xml xml/run158975.xml '[ ["EM_HI_SH"] ]'

If you are only interested in the desired trigger information, use –b –t.

Non-tree Stream Schemes

Level 3 is set up to handle only tree stream schemes (like the schemes discussed above). We may want to look into non-tree stream schemes though for evaluation. The logic of such schemes can be hard-coded in python and provided to the tool as a plugin module. Let me know if you want to explore this option.

2002 July 30 Adam Lyon (lyon@fnal.gov)