mc_runjob

Section 0 : Introduction to mc_runjob

mc_runjob is a package containing Python scripts design to aid in running D0 Offline executables. This document covers the usage of mc_runjob version tagged v02-03-03 in CVS. If you don't like to read documentation, consult the Quick Start Guide. Section 0 of this document is also useful for getting started quickly.

While developed specifically for D0 offline executables, mc_runjob can be easily extended to handle many other tasks. Figure 1 shows the basic design ideas behind mc_runjob. There are two basic entities. The Configurator encapsulates all of the knowledge needed to run an executable or perform some simple task. The Linker is responsible for instantiating Configurators, maintaining a list of Configurators, and handling communication among Configurators and the user.


Figure 1 : Conceptual design of mc_runjob. The user interacts with the Linker, sending commands to attach and configure various Configurators. Each configurator knows how to run one D0 Offline executable or how to perform a simple task. The user then asks the Linker to make a job at which point each Configurator produces a /bin/sh script and release area with configured RCPs to run its corresponding executable. The shell scripts are collected by the Linker and compiled into a master shell script which is then presented to the user. The user can run the resulting shell script, or it can of course be run by another Configurator.

The user can communicate with the Linker using commands contained in a macro script or through a Graphical User Interface (GUI). The GUI architecture is conceptually the same as depicted in figure 1; the Linker GUI will instantiate a Configurator GUI and maintain it in a visual list. The GUI implementations are related to the non-GUI implementation through inheritance; in fact the GUI Configurators are built automatically from the contents of their non-GUI counterparts with only a few additional cues regarding presentation and item ordering.

The contents of this manual are as follows :

Mc_runjob, as of this writing, consists of nearly 7500 lines of Python code. Much use is made of inheritance among the Configurators to keep the code size down.

Section 0.1 : Versions, Compatibility, and All That

The supported D0 offline executables as of the time of this writing are

The versions compatible with mc_runjob, however, may be easily extended to any test release version. This topic will be handled in Section 4.

Section 0.2 : Getting mc_runjob and Setting it up

The procedures for getting mc_runjob and setting it up are much like those for getting any package code saved in CVS and setting up the D0 RunII envioronment.

mc_runjob can be setup using the UPS setup command. NOTE : This will be implemented week of 10/9/00.


mc_runjob can also be installed by hand as described in the following sections. Before the UPS setup is implemented week of 10/9/00, this is recommended.

Section 0.2.1 : First - Setting up the D0 Run II Environment and Local Release Area

The mc_runjob Linker uses the D0 special environment variables $SRT_DIST and $SRT_LOCAL. These are setup by the usual D0 environment setup for Run II.

If you have a local release area, or create one if you need to

And finally,

Section 0.2.2 : Second - Obtaining the Package Code

To get mc_runjob, add the package mc_runjob to your local release area.

Section 0.2.3 : Third - mc_runjob Specific Setup

The mc_runjob Linker can be driven with macro scripts. Furthermore, the GUI Linker can also run macro scripts for configuration purposes. The Linker will always look in the current directory first for macro scripts; however, a search path may be set up that the Linker will use to find macro scripts that are not found in the cuirrent directory. The search path is kept in an environment variable called MC_RUNJOB_PATH which is read by the Linker upon startup.

mc_runjob comes with an array of configuration and test scripts. These are kept in the subdirectory py_script/scripts in the mc_runjob package. To include these scripts in the search path,

A similar search path is used for pythia and isajet card files. Standard card files are provided with mc_runjob for many processes. To include the standard card files in the search path,

The Linker python code is in the mc_runjob/py_script directory. If you did not do the UPS setup, you will most likely run the Linker from here. However, you can run the Linker from anywhere if you also add the mc_runjob/py_script directory to your PYTHONPATH environment variable. To see if this is defined,

If you are in csh/tcsh and get a non-null answer that does not include the string "Undefined variable" or similar, then the variable is defined. If you are in csh/tcsh and get a non-null answer that does include the string "Undefined variable" or similar, then the variable is not defined. If you are in sh/bash and get a non-null answer, then the variable is defined. If you are in sh/bash and get a null answer, then the variable is not defined. If the variable is defined, then while in the mc_runjob/py_script directory, Note to D0 Users : This variable had better be defined if you did a setup D0RunII ! If the variable is not defined, then while in the mc_runjob/py_script directory,

Section 0.2.4 : Check Setup

The following environment variables should be setup if you followed the above steps. This can be checked with the command "echo" in any shell. NOTE : These are examples only. Do not use these for real. Your values will be similar but not exactly the same.

SRT_DIST is set by the "setup D0RunII" command, and SRT_LOCAL is set by "d0setwa." mc_runjob needs both of these variables to be set. 




echo $SRT_DIST
/d0dist/dist

echo $SRT_LOCAL
/home/ggraham/umd2/mc_runjob/t109

And optionally ... 




echo $PYTHONPATH
/tmp_root/710/umd_2/ggraham/mc_runjob/t109/mc_runjob/py_script:/d0dist/dist/releases/t01.09.00/lib/IRIX6-KCC
_3_4:/d0dist/dist/packages/daq_monitor/v00-04-15/py:/d0usr/products/pmw/NULL/v0_8_3b:/usr/products/db_server
_gen/v1_8_5/NULL/template:/usr/products/db_server_gen/v1_8_5/NULL/src:/usr/products/python_dcoracle/v1_1_0:/
usr/products/IRIX6n32/fnorb/v1_0_1/:/usr/products/python/v1_5_2a/lib/python1.5/plat:/usr/products/python/v1_
5_2a/lib/python1.5/lib-tk:/usr/products/python/v1_5_2a/lib/python1.5/lib-stdwin:/usr/products/python/v1_5_2a
/lib/python1.5/lib-dynload:/usr/products/python/v1_5_2a/lib/python1.5

echo $MC_RUNJOB_PATH
.:/tmp_root/710/umd_2/ggraham/mc_runjob/t109/mc_runjob/py_script/scripts

echo $MC_RUNJOB_CARD_PATH
.:/tmp_root/710/umd_2/ggraham/mc_runjob/t109/mc_runjob/templates/cards

Section 1 : Using the Linker Macro Commands

The Linker macro commands can easily be organized into scripts and run by the Linker script reader. The GUI Linker includes an interface that allows macro commands to be run one at a time, however.

There are also several command line options that can be passed to the Linker. These are described in Section 1.3.

Section 1.1 : Linker Directives

#
# is the comment character. It will comment out from where it is in the line to the end of the line.

attach cfgtype [named cfgname]

This directive creates an instance of a configurator of type cfgtype and optionally assigns a name of cfgname to the configurator. You cannot attach more than one instance of a given configurator type with the same name. Casual users will not need to set Configurator names. The usage of names with Configurator dependencies is considered in Section 4 : Advanced Topics. The cfgtype is one of : The Configurators have built-in dependencies that disallow adding them to the linker out of order. Only filestream can appear anywhere in the list. For more information, see Section 2 on the Configurators.

cfg cfgtype [named cfgname] directive

The Linker will pass Configurator directive directive to the first configurator which matches cfgtype. If cfgname is given, then the Linker will pass the directive to the configurator which matches cfgtype and cfgname. The syntax is not checked by the Linker; the directive is passed directly to the Configurator in question. Casual users will not need the cfgname as explained above.

make job

The Linker will create a job directory for you and then pass the make job directive to each activated Configurator; each of which may also put things into that directory. The Linker will collect /bin/sh scripts from applicable Configurators and create a master shell script. Configurators that are deactivated will not respond to the make job directive.

MiniDB [0|1]

When set, this option causes a line describing the output of each executable to be included in a file ".database/database" under the CurrentDir specified in the samglobal Configurator. Unless you are doing production, you won't need this.

os shell-command [args]

The Linker will execute the given UNIX shell command with args using the Python commands.py module. Usually this is used to execute a sleep command.

print [filename | script]

The Linker will print out to the screen the current configuration of all of the Configurators it has attached. If a filename is given (not named 'script'!) then the configuration is printed to the file filename instead of to the screen.

If the special keyword 'script' is used after the print statement, then the name of the /bin/sh script corresponding to the last job created with 'make job' is printed to the screen.

repeat [N] ... end

A block of commands between the repeat statement and the end statement will be repeatedly executed. The number of times that the Linker executes this block of commands is specified by an integer N after the repeat statement. If N is omitted, the loop repeats forever or until a break condition is signalled by one of the Configurators. Currently, only the FileStreamConfigurator sets a break condition when it runs out of files to process.

reset [opt]

The reset message is sent to all attached and active Configurators. The behavior is up to the Configurator, but generally it means that all of the Configurator parameters are reset to null values. The behavior can be modified by passing an arguement opt along with the reset. For example, for Configurators that work with D0 Offline executables, if opt is set to 'chain', then the action is to reset only the input/output filenames for the current job. The configurators will interpret this option to set input filenames to "<Use Global>" and to set output filenames and tuple names to "<Construct>".

return
Signals to the Linker that the script is finished. Control passes back to the Linker.

save

The Linker will print out to a special file the current configuration of all of the Configurators it has attached. The file is located in the job directory of the last job created with 'make job'. The name of the file begins with 'td' and ends in '.conf' and is referred to as the 'td-dot-conf' file. The td-dot-conf file is important because it is a record of the configuration saved along with the actual job itself and because it is currently used to help generate SAM Metadata for import.

SaveOnMake [0|1]

When set, a "save" command is automatically executed during "make job".

source filename

This directive causes the linker to search for the file filename and execute the commands in that file. The current directory is searched first and then in order as specified by the environment variable MC_RUNJOB_PATH.

StandardD0 [0|1]

When set, each executable is prepared using the "setup n32; setup D0RunII; d0setwa" troika. This is recommended for running on d0mino. When not set, each executable is prepared using a special shell script which mimics the actions of setup D0RunII and d0setwa. When using the mini-tarfile executables in a remote location without the standard D0 software installation, this is required. Specifically, if the executable is run from within a shell where setup D0RunII has already been executed, the option must be set so that the relevant unsetup D0RunII can be executed.

Section 1.2 : Example Script

A script that generates Pythia ttbar events and runs them through RecoAnalyze might look like the following. Do not worry about the Configurator directives for now; they will be explained later. 
MiniDB
StandardD0
SaveOnMake
#
#  Attach and initialize configurators
#
#
#  Global defines the output directory
#
attach samglobal
source samglobal.cfg
#
#  Pythia is a configurator for the MCPythia_x program
#
attach pythia
source pythia.cfg
#
#  d0gstar is a configurator for the d0gstar2.x program
#
attach d0gstar
source d0gstar.cfg
#
#  d0sim is a configurator for the D0Sim_x program
#
attach d0sim
source d0sim.cfg
#
#  mc_exam is a configurator for the MCexam_x program
#
attach mc_exam
source mc_exam.cfg
#
#  d0reco is a configurator for the D0Reco_x program
#
attach d0reco
source d0reco.cfg
#
#  RecoAnalyze is a configurator for the RecoAnalyze_x program
#
attach recoanalyze
source recoanalyze.cfg
#
#  RunJob_d0mino is a configurator for running jobs on d0mino
#  Submit scripts to mc99 queue.
#
attach runjob_d0mino
cfg runjob_d0mino define int RunInBackground 0
cfg runjob_d0mino define int RunInBatch 1
cfg runjob_d0mino define string BatchQueue mc99
#
#  Set up parameters for this job
#
cfg pythia define string Production ttbar
cfg pythia define string Decay incl
cfg pythia define int NumRecords 10  # 10 events
#
#  Make a job area and script to run job.  Do it twice.
#
repeat 2   

  #  Chain outputs of earlier jobs to inputs of later ones 
  reset chain

  #  Make random number seeds 
  cfg pythia make seeds
  cfg d0gstar make seeds

  #  Make the job
  make job

  #  Save config options in file in job area
  #  technically unnecessary if we enabled SaveOnMake.
  save

  #  Print the full pathname of the job script to stdout
  print script

  #  Run job on d0mino
  cfg runjob_d0mino run

end

#Done
return

Section 1.3 : Running the Linker from the Shell Prompt

The syntax for running the Linker from the Unix shell prompt is

python Linker.py option-list

Available Options :

In the case that no arguments are given, nothing will happen. In the case that the script argument is given, the Linker will try to find the file scriptname in the current directory or in one of the directories specified by MC_RUNJOB_PATH and will execute the commands in the file if found. In the case that the control=GUI argument is given, the GUILinker window will appear. The GUI Linker is related to its non-GUI partner by inheritance. All of the methods defined by the Linker are used verbatim by its GUI partner so that equivalent functionality is easily maintained.

Section 2 : The Configurators

The inheritance heirarchy of Configurators is shown in Figure 2. The class definitions have been written so that directives sent to a given configurator object are interpreted by the interpreter function of the class that the object belongs to. If an object does not understand a directive, it is passed to the interpreter function of its immediate superclass. Inheritance is used in this way to define the set of directives understood by sets of Configurators.

The attributes available in each configurator are compiled at object instantiation using Python lists. By explicitly chaining together the Python constructors, the lists can grow down the inheritance chain. Thus the Configurator attributes are also inherited. Figure 2 shows the current inheritance heirarchy of Configurators.

The GUI version (not shown) of each Configurator is derived from its non-GUI partner and from a special mixin class that contains the method used to build the GUI from information in its non-GUI partner. All of the methods defined by a Configurator are used verbatim by its GUI partner so that equivalent functionality is easily maintained as in the Linker.


Figure 2 : The inheritance heirarchy of Configurators. The Configurator base class is very general and contains basic tools used by all other configurators. Other functionality is added by subclassing. Only D0 configurators, for example, contain tools for dealing with RCP files. Only MCC configurators know how to deal with Monte Carlo Challenge standard filenames. GUI configurators are not shown.

Section 2.0.1 : General Configurator Directives

The Configurator base class uses a Typed Dictionary to keep track of parameter names, values, and types. Each Configurator has a list of valid parameters of which it keeps track. All Configurators therefore understand the following directive :

define type varname value

The Configurator will update the value of parameter varname with value vartype. The type must match the internally defined type of the parameter. ( NOTE : type here is now superfluous, but it has been retained for historical reasons. ) If varname does not correspond to a valid parameter for that Configurator, an error condition is generated. The valid types are 'string', 'int', and 'float'.

Configurator derived classes also have dependencies, called requirements, built in. For example, it is not possible to attach a Configurator if it depends upon another Configurator that is not present already in the Linker. (eg - You cannot add a "d0reco" Configurator unless a "d0sim" Configurator is already present. Of course, the "d0sim" Configurator could be a list of files previously processed by D0Sim_x maintained by a FileStreamConfigurator. ) The commands which affect the naming of requirements are

chreqtype class subclass type

Changes the type specifier of requirement class subclass.

chreqname class subclass name

Changes the name specifier of requirement class subclass.
More on the way in which dependencies are currently handled is covered in section 4 : Advanced Topics.

Configurators may be deactivated or activated in the Linker.py. When activated, the Configurator will respond to the "make job" directive passed by the Linker. When deactivated, it will not.

activate

Changes Configurator state to activated.

deactivate

Changes Configurator state to deactivated.

Finally, all Configurators must implement a hook method called "MakeJob()". This is called by the user or by the Linker with the

make job

directive.

Section 2.1 : Available Derived Configurators and Their Parameters and Methods

The following list describes various configurators that are available in the following format :

Class : ClassName Alias : aliasname
Description : Brief description of Configurator ClassName.
Dependencies dep1, dep2, etc.
Brief description of attributes..
type1 parametername1 Brief description of type1 parametername1
type2 parametername2 Brief description of type2 parametername2
... ...
Brief description of directives..
directive1Explanation of directive1.
.......
The aliasname is the name that must be passed to the "attach" method of the Linker, described in section 1.1. If aliasname is given as None, then the Configurator in question is not attachable (eg- it is an intermediate class in the heirarchy, not meant for instantiation.) If a Configurator has a certain parameter or method, then all derived Configurators also share that parameter or method unless otherwise noted.


Class : SamGlobalConfigurator Alias : samglobal
Description : This Configurator does not correspond to a D0 offline executable; rather it configures the environment in which the other configurators run. However, all D0 offline Configurators depend on samglobal, so it should always be attached first. The first two parameters listed are crucial for correct execution. The rest will be used to compile SAM metadata at the end of the job. If you are not importing the output into SAM, it is not necessary to fill these in (but it is a good idea for book keeping purposes or if you change your mind about SAM later.)
Dependencies : None
Brief list of attributes.
string CurrentDir The current directory is where the Configurators will set up for processing. For example, the master shell script and the "td.conf" file will reside here. Also, the local release areas where each executable runs will reside under this directory.
string DestinationDir The directory which will contain the output from the executables. This should point to some large disk partition.
string GroupName The SAM Group name.
string OriginName The SAM Origin name. This is the name of the institution where the data was generated or processed.
string FacilityName The SAM Facility name. This is the name of the machine or cluster that produced or processed the data.
string ProduceForName The name of the person for whom the data were produced or processed.
string ProduceByName The name of the person who actually initiated the data processing or production.
int WorkRequestID Part of the interface to the SAM automatic processing tables. If 0, this is converted to the calendar year.
int RequestID Part of the interface to the SAM automatic processing tables. If 0, this is converted to a timestamp based on day of year, 24-hour time.
string TimeStamp A time stamp variable.
string Phase A string describing the monte carlo phase.
string JobName An arbitrary string which will be inserted into the name of the job directory under which the job is run. For user tracking purposes.
Brief list of directives.
time This resets the timestamp to current clock time.


Class : D0Configurator Alias : None
Description : Adds the attribute D0Release to the inheritance heirarchy.
Dependencies : None
Brief list of attributes :
string D0Release The D0 Release version. For example, pmc03.00.01, preco04.00.03, etc.
Brief list of directives : None

Class : D0Configurator
withSpecialRCP 		Alias : None
Description : Adds the attribute SpecialRCPDir to the inheritance heirarchy.
Dependencies : None
Brief list of attributes :
string SpecialRCPDir The full pathname of a directory containing RCP scripts to be used in place of standard scripts to be used in place of the standard RCPs. The format is in this directory, to put scripts in subdirectories package_name_1/rcp. The Configurator will recursively copy these subdirectories to the local release area where the corresponding executable is actually run, NOTE : This is considered a stopgap solution. A more sophisticated D0Configurator_withRCPEdit will eventually replace this feature.
Brief list of directives : None

Class : ReadJobMixin Alias : None
Description : A mixin class which adds attributes to a Configurator to describe input to an exeuctable that reads input data.
Dependencies : None
Brief description of attributes :
string InputFileName The name of the input file containing records to be processed by the executable.
int NumRecords The number of records the executable will process from the input file.
int SkipRecords The number of records the executable will skip from the input file.
Brief description of directives : None

Class : WriteJobMixin Alias : None
Description : A mixin class which adds attributes to a Configurator to describe input to an exeuctable that writes output data.
Dependencies :None
Brief description of attributes :
string OutputFileName The name of the output file to contain records to be written by the executable.
Brief description of directives :None

Class : CreateJobMixin Alias : None
Description : A mixin class which adds attributes to a Configurator to describe input to an exeuctable that generates data.
Dependencies :None
Brief description of attributes :
int NumRecords The number of records the executable will create.
Brief description of directives :None

Class : RunNumberMixin Alias : None
Description : A mixin class which adds attributes to a D0Configurator to keep track of monte carlo run numbers.
Dependencies :None
Brief description of attributes :
int RunNumber The Monte Carlo run number. This number is ignored if RunNumberFile is specified.
string RunNumberFile A file containing the Monte Carlo run number. On job creation, this number is incremented and saved for the next job. If this parameter is null, then the run number is determined from the RunNumber parameter.
Brief description of directives :None

Class : MCC_Mixin Alias : None
Description : A mixin class which adds functionality to handle MCC standard filenames. If an input filename in a Configurator derived from MCC_Mixin is set to "<Use Global>", then the input filename will be generated from the output filename of the Configurator that it is dependent upon. If an output filename in a Configurator derived from MCC_Mixin is set to "<Construct>", then the output filename will be constructed in the official MCC format.
Dependencies : None
Brief description of attributes :None
Brief description of directives :None

Class : D0MCCCreator Alias : None
Description :
Dependencies :None
Brief description of attributes : None
Brief description of directives : None

Class : GenConfigurator Alias : pythia
Description : A configurator for Monte Carlo Generators.
Dependencies : None
Brief description of attributes :
string RanSeed1 The string representation of the first needed random number seed.
string RanSeed2 The string representation of the second needed random number seed.
float PtGt Lower Pt Limit.
float PtLt Upper Pt Limit.
float EtaGt Lower Eta Limit.
float EtaLt Upper Eta Limit.
Brief description of methods :
make seeds The configurator will generate RanSeed1 and RanSeed2 according to the system clock.

Class : SingleConfigurator Alias : single
Description : A configurator for the MCSingle_x executable.
Dependencies :samglobal
Brief description of attributes :
int PdgId The Particle Data Group ID number for the particle to be generated. ( See MC Particle Numbers Page for a list. )
int AutoAntiOn When set to 1, both particles and anti-particles will be generated if applicable.
int Multiplicity The number of single particles to include per event. (Usually 1.)
int EtaFlat When set to 1, eta values are generated flat. Otherwise, the distribution is flat in solid angle.
float PhiGt Lower Phi Limit.
float PhiLt Upper Phi Limit.
string DecayFile The particles are decayed using isajet. This is the name of the decay file to use.
Brief description of methods : None

Class : PythiaConfigurator Alias : pythia
Description : A configurator for the MCPythia_x executable.
Dependencies :samglobal
Brief description of attributes :
string Production The string tag for the production process.
string Decay The string tag for the decay process.
float CollisionEnergy The collision energy in GeV. (ie- 2000.0)
Brief description of methods : None

Class : IsajetConfigurator Alias : isajet
Description : A configurator for the MCIsajet_x executable.
Dependencies : samglobal
Brief description of attributes :
string Production The string tag for the production process.
string Decay The string tag for the decay process.
float CollisionEnergy The collision energy in GeV. (ie- 2000.0)
Brief description of methods : None

Class : CosmicConfigurator Alias : cosmic
Description : A configurator for the MCCosmic_x executable.
Dependencies :samglobal
Brief description of attributes :
float Ethresh The lower limit of cosmic ray energy to be generated.
float MagicSquareX The x width in cm of a square constraining the cosmic ray trajectories through the D0 detector.
float MagicSquareZ The z width in cm of a square constraining the cosmic ray trajectories through the D0 detector.
float CenterX The x coordinate of the center of the "Magic Square".
float CenterY The y coordinate of the center of the "Magic Square".
float CenterZ The z coordinate of the center of the "Magic Square".
Brief description of methods : None

Class : D0MCCProcessor Alias : None
Description :
Dependencies :None
Brief description of attributes : None
Brief description of directives :
inc skip N Increments the attribute SkipEvents by N.

Class : D0gstarConfigurator Alias : d0gstar
Description : A configurator for the d0gstar2.x executable.
Dependencies : samglobal, generator
Brief description of attributes :
int SimRanSeed1 The first of the random number seeds needed to pass to d0gstar2.x.
int SimRanSeed2 The second of the random number seeds needed to pass to d0gstar2.x.
string TzFilesDir An alternate directory to search for .tz geometry files. Any .tz files in the specified directory are copied to the local release area.
Brief description of methods : None

Class : D0SimConfigurator Alias : d0sim
Description : A configurator for the D0Sim_x executable.
Dependencies : samglobal,simulator
Brief description of attributes :
float NumMinBi The number of minimum bias events to add using the pileup package.
string MinBiOpt Set to poisson or .
float MinBiFileName A full pathname specification of a file or whitespace separated list of files containing min bias events.
Brief description of methods :

Class : D0RecoConfigurator Alias : None
Description : A configurator for the D0Reco_x executable.
Dependencies :samglobal,digitizer
Brief description of attributes : None
Brief description of methods : None

Class : MCexamConfigurator Alias : mc_exam
Description : A configurator for the MCexam_x executable.
Dependencies : samglobal,digitizer or simulator
Brief description of attributes :
int UseRoot When set to 1, a root tuple is generated. When set to 0, an HBOOK ntuple is generated.
string InPrefix When set to "sim", the Configurator will look for D0Sim_x output. When set to "d0g", the Configurator will look for d0gstar2.x output.
string TupleName The name of the output ntuple file. When set to <Construct>, the TupleName will be constructed on "make job" using a standard naming convention.
Brief description of methods : None

Class : RecoAnalyzeConfigurator Alias : reco_analyze
Description : A configurator for the RecoAnalyze_x executable.
Dependencies : samglobal, reconstruction
Brief description of attributes : None
Brief description of methods : None

Class : RunJob Alias : runjob
Description : A Configurator that runs the script returned by the Linker after a "make job" directive.
Dependencies :None
Brief list of attributes :
int RunInBackground When set to 1, the job will run in the background of the current process. When set to 0, the job will run in the foreground.
Brief description of methods :
run The master script maintained by the Linker is run.

Class : RunJob_Stack Alias : runjob_stack
Description : A Configurator that stacks the script returned by the Linker after a "make job" directive into a master script maintained by this Configurator.
Dependencies :None
Brief list of attributes
int StackInParallel When set to 1, the stacked jobs will execute in parallel processes. When set to 0, the stacked jobs will execute sequentially. NOTE : This functionality was requested for some remote sites of the Monte Carlo Challenge. I will not be responsible if this functionality is abused on d0mino. If you try, you will more likely than not be caught. Do not try. And I in no way encouraged you, and we never met.
Brief description of methods :
stack The current script contained in the Linker is stacked inside a master script maintained by the runjob_stack configurator.
run The master script maintained by the runjob_stack Configurator is run.

Class : RunJob_Batch Alias : None
Description : A Configurator that runs the script
Dependencies :None
Brief description of attributes :
int RunInBatch When set to 1, this overrides the "RunInBackground" option and submits
int BatchQueue The name of the batch queue to which to submit the job.
Brief description of methods : None

Class : RunJob_d0minoAlias : runjob_d0mino
Description :
Dependencies :None
Brief description of attributes : None
Brief description of methods :

:
Class : FileStreamConfigurator Alias : filestream
Description : The filestream Configurator manages a list of files to be used as input for processing by some other Configurator. The files in the list must be locally resident and full pathname is recommended.
Dependencies :None
Brief list of attributes :
string FilesToProcess A filename which refers to a file that contains a list of files to be processed as input files.
int MergeFiles When set to 1, this signals filestream to serve filenames in whitespace separated bunches (ie - for a ReadEvent package multiple file input.) When set to 0, filenames are served one at a time.
int MergeFrequency The number of files per bunch when MergeFiles=1.
Brief list of directives
setinfo classname None filestream For D0 offline, the class name should be one of {generator,simulator,digitizer,reconstruction}; whichever best describes the content of the files handled. For casual use, the second two arguments should be used as indicated above. This directive allows you to satisfy the dependencies of Configurators to be added later.

Section 2.2 : Automatic Initialization of Configurators

The mc_runjob.config file is a special file which contains Configurator specific sections in it. Each section begins with a "#% cfgname" header and is followed by a number of parameter configuration statements. When a Configurator is attached to the Linker, the Linker searches the directories in the MC_RUNJOB_PATH for the first encountered mc_runjob.config file and executes the statements in the corresponding section. Configurators can be given a set of user defined defaults in this way. A sample mc_runjob.config file looks like the following : 

#% samglobal
cfg samglobal define string JobName test
cfg samglobal define string DestinationDir ./dest
cfg samglobal define string CurrentDir ./curr
cfg samglobal define string Phase mcp03
cfg samglobal define string OriginName fnal
cfg samglobal define string FacilityName d0mino
cfg samglobal define string ProduceForName Who For
cfg samglobal define string MyName Who Am I
cfg samglobal define string GroupName Which Group

#% isajet
cfg isajet define string D0Release pmc03.00.01
cfg isajet define int NumRecords 10
cfg isajet define string RunNumberFile ~/run.number
cfg isajet define string DecayFile notau_decay
cfg isajet define float CollisionEnergy 2000.0
cfg isajet make seeds

#% pythia
cfg pythia define string D0Release pmc03.00.01
cfg pythia define int NumRecords 10
cfg pythia define string RunNumberFile ~/run.number
cfg pythia define float CollisionEnergy 2000.0
cfg pythia make seeds

(... etc.)

Section 3 : The mc_runjob GUI

The mc_runjob package comes with an automatically generated graphical user interface. The architecture of the GUI is shown in Figure 3. This architecture was designed in response to user advice during the Monte Cralo Challenge, Phase II. While the GUI in phase 2 was popular, it was also the only method for using mc_runjob. For script driven applications or for users with a slow or unreliable X windows connection, this was clumsy. The new mc_runjob provides a GUI that uses all of the functionality of the non-GUI components.


Figure 3 : The relationship between GUI classes and non-GUI classes in mc_runjob. The GUI Configurators inherit all core functionality from their non-GUI couterparts. They also inherit GUI-building methods from the GUIConfigurator base class. The GUI Configurator class (and any intermediate classes) essentially only add cues relating to how to layout the GUI elements. The GUI Linker has a similar architecture. The GUI Linker will attach GUI Configurators, while the non-GUI Linker will attach Configurators.

The GUI Configurators inherit all core functionality from their non-GUI couterparts. They also inherit GUI-building methods from the GUIConfigurator base class and take advantage of specially written widgets to organize the display of information. The GUI Configurator class (and any intermediate classes) essentially only add cues relating to how to layout the GUI elements. This includes a choice of widget for display, the order of widget display, and whether the widget appears in the default Configurator window or under "Advanced Options." The TypedDictionary used to store parameters is replaced by a TkTypedDictionary which in addition keeps track of the Tk instances of these parameters linked directly to their corresponding widgets (ie- they do not require explicit snychronization between what is seen and internally stored values.) The GUI Linker has a similar architecture. The GUI Linker will attach GUI Configurators, while the non-GUI Linker will attach Configurators.

Section 3.1 : Running the Linker GUI

Section 3.1.0 : Startup

The Linker GUI is started with the command

python Linker.py control=GUI
A reliable X connection is required as well as a python product which supports the Tk widget interface.


Figure 4 : The Linker GUI on startup.

On startup, a small window appears with three menus.

The File menu allows you to


The Job menu allows you to
The Options menu allows you to set various linker parameters.

Section 3.1.1 : Attaching Configurators

When the attach option is selected from the File menu, the "attach" dialog box appears. This is shown in Figure 5. The "attach" dialog will remain until explicitly dismissed. To attach a Configurator, select the name of a Configurator from the list with the mouse and press the "attach" button. You can optionally give your Configurator a name by filling in a value other than "None" in the name box. Names are covered in Section 4 : Advanced Topics.


Figure 5 : The "attach" dialog box. To attach a Configurator, select the name of a Configurator from the list with the mouse and press the "attach" button. The "attach" dialog will remain until explicitly dismissed.

Section 3.1.2 : Working with GUI Configurators

Each GUI Configurator has special controls which affect the display of parameters. On attachment, all of the Configurator window is hidden. This is because each Configurator window can be fairly large, and a single X terminal may be extremely cluttered if forced to contain many configurator windows at once. This is shown in Figure 6.


Figure 6 : The Linker has four different Configurators attached. Each Configurator GUI shown above is shown in "hidden mode."

The check button labeled "Show" (or "Hide" when the corresponding window is open) controls the show/hide behavior of each Configurator window. When Show is activated, the check button turns on and the default window is displayed for that Configurator. When deactivated, the window is hidden again.

When open, every configurator window has two features. First is the "Configurator Active" check button. When checked, the corresponding Configurator becomes active. When unchecked, the corresponding Configurator becomes inactive. Second, another checkbox allows the user to see the Advanced Options window. When opened, the Advanced Options window displays parameters that are usually hidden from the user. This is shown in Figure 7.


Figure 7 : The same Linker in Figue 6 is shown with the pythia Configurator in "show" mode. The advanced options button is also checked for this Configurator, so that the other, usually hidden, window is also present.

Section 4 : Advanced Topics

This section is under construction.

Subsection 0.0 : Title

Last updated by Greg Graham on September 29, 2000