lm_tools documentation: getLuminosity

lm_tools is an ups product that packs together lm_access and lm_access_interface, providing some useful (and ready to use) commands to access the luminosity information. The measurement of the luminosity is obtained with a two step procedure:

first the list of luminosity blocks associated to a given dataset is obtained using the genLBNtables script (this list is the so called "parentage files" list). This first step has to be executed at least once (but you don't need to repeat it if you are calculating the luminosity for two different triggers on the same dataset).
then the actual luminosity calculation is done using the getLuminosity executable

NOTE FROM THE PHYSICS COORDINATORS

If you are using a single trigger in your analysis the luminosity system will give you the correct luminosity to use for the data normalization, taking into account automatically all possible prescales.

If you are using the OR of multiple triggers in the analysis you should be careful that the effects of possible prescales are taken into account either through the luminosity normalization or through the trigger efficiency.

You should discuss with the Trigger Studies Group conveners the best approach for doing this.

Important information about flags in getLuminosity

There are two new flags in getLuminosity that are important to understand. It is the recommendation of the Luminosity Group that you use BOTH of these flags, with the caveats mentioned below.

One of these flags is the --ignoremissingstreams flag. As of this writing (May 23, 2008), this flag is NOT the default. The default would be to label as BAD any LBN which is missing one or more of the data streams. If you set the --ignoremissingstreams flag, these LBNs will not be labeled BAD, and the luminosity will be scaled assuming all four streams were populated uniformly.

The other important new flag is --ignoreduplicates . Some datasets have duplicate LBNs, and the default in getLuminosity is to protect the user by declaring BAD all LBNs that appear more than once in the parentage files. We have found that for some datasets, this resulted in a large (15%) loss of luminosity. If the --ignoreduplicates flag is used, duplicate LBNs will not be marked as BAD. If you are 100% sure that you are removing duplicate events (by using the appropriate Cafe processor for example) then it is safe to use this flag. To see if you have a significant number of duplicate LBNs, you can run through your luminosity calculation with and without this flag to see the difference.

Also, as of this writing, the luminosity database and access to it are still under development. Therefore, you must use the --stage3 option noted below. When the --lmdb option is available, this documentation will be updated and the physics conveners will be informed.

In all cases, remember to redo your analysis after merging the bad LBN list produced by getLuminosity with your own bad LBN list.

Using getLuminosity

Before issuing the commands, make sure you have done:

setup D0RunII (must be t7.04.00 or later version)
setup lm_tools -t (use the -t option to get the correct version.)

getLuminosity usage:

	getLuminosity source [flags] [options] [input files]

The "source" argument is mandatory. It specifies the source of the luminosity data. It must be either
	--lmdb	use Luminosity Database (run genLBNtables with the -type DB option)
	--stage3	use stage3 files (run genLBNtables with the -type LM option)

options:	the options below are exclusive, that means you can use only one at a time:
	--runrange	Calculate the luminosity for a range of runs. It will output a short summary in the standard output, a more detailed summary in files with extension .sum, a list of bad LBN for each requested type of luminosity in the .badlbns file and luminosity information for caf producers in the .caf files. This option requires the input files below given in this exact order:
		filelist parentpath runrangelist
		filelist	should be a file in the current directory which contains a list of the files you want to get lum for
		parentpath	should be a directory which contains the parentage files. See below to know how to obtain them.
		runrangelist	should be a file with 'trigger first_run last_run' on each line
	You can pass your own list of good and bad runs as well as bad LBNs by mean of the files './goodRunsList', './badRunsList' and './badLBNList'. The default is to return all options: delivered, recordedall (recorded luminosity without applying your good/bad runs and LBNs lists), recorded (recorded luminosity, applying your good/bad runs and LBNs lists), and reconstructed (successfully passed through D0reco, usually the same as recorded). You can run faster by choosing which you want. Add one or more of --delivered --recordedall --recorded --reconstructed right before the --runrange argument. You can also get info for all ticks rather than the sum by adding --allticks before --runrange. By default, the interface to the DB will not return information for BAD LBNs. You can modify this behavior adding --ignorecondition id1,id2,id3... before --runrange. The condition_id can be obtained finding out one LBN for which the condition occurred (see the Luminosity ID home page and go to the Luminosity DB) and clicking the EXPLAIN_LBN link in the STATUS_TO section. The most likely IDs to be ignored are 22 (All low bias events lost in L3) and 23 (No low bias triggers). You can get a progress report to the screen (clog c++ stream) by adding --progress before --runrange --ignoremissingstreams will calculate the luminosity if one or more input stream is missing by prorating the luminosity based on the number of streams present, assuming all streams are uniformly populated. The default without this flag is to mark any LBN with a missing stream as BAD. The --ignoremissingstreams flag should be added before --runrange. --ignoreduplicates will calculate the luminosity without labeling as BAD any LBNs with occur more than once. This flag should also appear before --runrange. If you are using this flag, it is your responsibility to be certain the duplicate events have been removed in your analysis, and you should inform your editorial board.

Additional options: (cannot be used at the same time as --runrange)

		--dump	Dump the data for the specified LBN(s) and trigger. The arguments are:
			lbn [[# lbn][last_lbn]] trigger
			lbn	The LBN for which the information will be dumped.
			#lbn OR last_lbn	if this argument is a number and the number is smaller than the given lbn above, it will be taken as the number of LBN to dump (counting from lbn). If this number is bigger than the first argument, a range of LBNs will be dumped.
			trigger	the trigger for which the luminosity will be printed (default is EM_HI_SH)

		--ticklum	Prints out the instantaneous luminosity, proton halo, anti proton halo and the average number of inelastic interaction per bunch crossing for the given LBN(s). The arguments are:
			lbn [[# lbn][last_lbn]] [tick[=7]]
			The lbn is given the same way as above for --dump option.
			tick	Specifies the tick number for which the data will be dumped. The default is 7. If 0 is given, all the 36 ticks are printed out.

		--LBNStatus	(--lmdb option only) Prints out the LBN status flag (severity as in the LmDb table description) as well as the brief description and condition ID when something special happened for the given LBN(s). You can specify a single LBN or a list of them in a file, one per line.
			Example:	getLuminosity [--debug] --LBNStatus lbn	-> get status for a single LBN.
				getLuminosity [--debug] --LBNStatus lbnlist	-> get status for the lbns listed file lbnlist.

Additional flags (optional):
	--debug		Print out debuging information.

Example 1: Calculate the luminosity for the trigger zero_bias_NCU, runs between 189800 and 189900 (skim ZBMB):

	% setup D0RunII t07.04.00 % setup lm_tools	(You need this or a later version of the release to provide a compatable root version. )
	% setup sam	(If you have your parentage files already, you don't need setup sam)
	% genLBNtables -skim ZBMB -format TMB -pass p17pass1 -type LM	(Generate parentage files for the stage3 files. This step MUST be done on a Fermilab computer because the input files are available only on site)
	% echo "zero_bias_NCU 189800 189900">runrange.lis	(specify which trigger and runs to use)
	% ls -1 ZBMB\|sed -e's/.parentage//' > ZBMB.filelist	(create a list of the parentage files, stripping the file extension. Note that the -1 argument is the number one)
	% getLuminosity --stage3 --runrange ZBMB.filelist ZBMB runrange.lis

The output will be:

(several lines with bad LBN status like the first line below were deleted)

0 lbns were identified and 0 were rejected as duplicates
delivered luminosity for runs 189800-189899 is : 1.95443 pb^-1
processed 1523 lbns, of which 1416 were good

recorded luminosity for runs 189800-189899 and trigger zero_bias_NCU is : 5.23301e-07 pb^-1 L3 accepts = 37320
fraction lost was 1
processed 1523 lbns, of which 1357 were good

recorded luminosity for runs 189800-189899 and trigger zero_bias_NCU with bad lbn list is : 5.23301e-07 pb^-1 L3 accepts = 37320
fraction lost was 0
processed 1523 lbns, of which 1357 were good

reconstructed luminosity for runs 189800-189899 and trigger zero_bias_NCU with bad lbn list is : 5.23301e-07 pb^-1 L3 accepts = 37320
fraction lost was 0
processed 1523 lbns, of which 1357 were good
WARNING - reconstructed information depends on offline checks and is often not available for recent data - use recorded instead for studies

Example 2: Calculate the luminosity per tick:

	% setup t07.04.00 % setup lm_tools -t	(you don't have to do the setup again if you have done it already)
	% getLuminosity --stage3 --ticklum 3039999

% setup t07.04.00

% setup lm_tools -t

(you don't have to do the setup again if you have done it already)

% getLuminosity --stage3 --ticklum 3039999

The output:

LBN Tick Luminosity PHalo AHalo Interaction/Xing(*)

3039999 7 0.2783 832.5 64.25 0.3430

(*) Average number of ppbar inelastic interaction per bunch crossing.

Last update May 3, 2008, by MDC.