From grivaz@lal.in2p3.fr Mon Apr 12 14:40:06 2004
Date: Fri, 09 Apr 2004 19:35:56 +0200
From: Jean-Francois Grivaz <grivaz@lal.in2p3.fr>
To: Herbert Greenlee <greenlee@fnal.gov>, d0dfwg@fnal.gov
Cc: d0-conveners@fnal.gov
Subject: Re: Common root-based data format.

Hi Herb and d0dfwg,

Here are the NP group responses to your questionnaire:

1.  What analysis data formats and analysis tools are members of
your group currently using?

There is no uniform data format in the NP group.
Some analyzers work directly on thumbnails and create their own
custom roottuples. A few rely on top_analyze, Some others use
TMBtrees (with d0correct used in the production).

2.  What analysis data formats or analysis tools does your group
recommend to its members?

There is no explicit recommendation from the NP group.

3.  Do you encourage or discourage people to use tmb_tree?  Why or
why not?

TMBTrees are centrally produced, which can be considered as some
kind of encouragement.

4.  How does your physics group support the efforts of analyzers?
That is, does your group provide centrally managed data sets,
tuples/trees, or analysis tools?

See above. There are no specific NP group analysis tools.

5.  Would your group benefit from the availability of common,
possibly centrally produced root trees?  What requirements would
a common root format have to fulfill for your group to benefit?

We'd certainly benefit.
See answers to #6 and #7 for the second question.

6.  If tmb_tree were chosen as the basis for a common format,
what changes would be required to make it attractive to your
group?

Speed is the main issue. Size is also important.

Skimming of TMBTrees (selecting events, branches, or even objects
in branches) should be made easy (and fast). This is essential as
we expect that people will want to reduce the size of their final
analysis sample.

The interface of TMBtree objects should be made as similar as
possible to the corresponding objects in chunks.

See also answer to #7.

7.  Does your group develop algorithms in root?  Should algorithm
development in root be encouraged?  What is the best way to allow
the entire collaboration to benefit from algorithms developed in
root?

If developments are made in root, they must be "analysis framework
neutral" (i.e.  not specific to tmbpro / d0root / top_analyze, etc)
and be made as close as possible to an algorithm which can be fit
into reco.

If speed, size and user-friendliness issues can be solved,
the "best way to allow the entire collaboration to benefit from
algorithms developed in root" might be to have a root data format
with the exact same interface as thumbnail or DST. This would allow
code developed on the tree to be run in d0reco without any change.
As you probably know, a solution of this kind (edmroot) has already
been worked out, and should be evaluated.

8.  Is there any other information that you would like to bring
to the attention of the Data Format Working Group?

Good documentation is essential as well as tutorials (simple
analysis, tricks to speed up the analysis, skimming), and they
should be maintained. Any new version of root should be tested
to work with sample analyses before beingadopted for releases.

Best regards,

J.-F. & Volker

--On mercredi 7 avril 2004 19:37 -0500 Herbert Greenlee <greenlee@fnal.gov> 
wrote:

> Hello Conveners,
>
> As you know, the D0 Data Format Working Group has been formed to
> review analysis data formats (tuples/trees) currently being used in D0,
> and to propose and develop a common root-based format.  We are now in
> the information gathering stage.  We are interested in your opinions.
> We are asking you, the physics group conveners, to assist our group in
> its efforts by answering the following questions.
>
> As we currently envision it, a common root format would consist of a
> centrally maintained set of tools (framework packages and executables) for
> producing root files from thumbnails.  We also envision having a centrally
> produced and managed set of root files produced from Common Sample Group
> thumbnails, stored in sam and possibly pinned on disk.  Where and by whom
> such common root files would be produced is net yet determined.  The
> common root files should incorporate thumbnail fixes and certified object
> corrections (d0correct).  Common root files would be offered to the
> physics groups as an alternative format to thumbnails for doing analysis.
>
> Thank you for giving this matter your thoughtful attention.  Please send
> replies to d0dfwg@fnal.gov.
>
> Here are the questions.
>
> 1.  What analysis data formats and analysis tools are members of your
> group currently using?
>
> 2.  What analysis data formats or analysis tools does your group
> recommend to its members?
>
> 3.  Do you encourage or discourage people to use tmb_tree?  Why or why
> not?
>
> 4.  How does your physics group support the efforts of analyzers?
> That is, does your group provide centrally managed data sets,
> tuples/trees, or analysis tools?
>
> 5.  Would your group benefit from the availability of common, possibly
> centrally produced root trees?  What requirements would a common root
> format have to fulfill for your group to benefit?
>
> 6.  If tmb_tree were chosen as the basis for a common format, what
> changes would be required to make it attractive to your group?
>
> 7.  Does your group develop algorithms in root?  Should algorithm
> development in root be encouraged?  What is the best way to allow the
> entire collaboration to benefit from algorithms developed in root?
>
> 8.  Is there any other information that you would like to bring to the
> attention of the Data Format Working Group?
>
> Regards,
>
> The D0 Data Format Working Group