Welcome to the DØ GRID Group!!!

Dzero VO

• You've received a notice that your dzero VO membership is about to expire.
• You don't have a clue as to what that is, what it means or how to do it.
• Or you tried to follow the instructions and ran into an error at some point.
• Got to a page that has no obvious relation to re-signing anything.
  Especially if you got to a page that says "Registration (Phase 1)" DO NOT fill it out.
  You do not want to re-register.

Link to user VO membership registration instructions

D0 Grid

D0 Grid is a virtual project whose core is the D0-PPDG group at Fermilab and which includes off-site D0 collaborators under the aegis of various Grid projects. It's mission is to enable fully distributed computing for the experiment, by enhancing SAM as the distributed data handling system of D0, incorporating standard Grid tools and protocols, and developing new solutions for Grid computing together with Computer Scientists. Under this mission, the project strives to unite the D0 efforts from the multifarious Grid activities (PPDG, EU DataGrid, GridPP and more), off-site analysis work and other aspirations distributed throughout the D0 collaboration. The two main areas of work are Job Handling (including specification, brokering, scheduling etc.) and Monitoring and Information Services.

If you are a D0 Collaborator seeking to develop, or adopt, a working solution for D0 distributed computing that can be used by the whole experiment, and perhaps by other experiments, and/or if you are interested in Grid solutions for future HEP collaborations and wish to prototype or test a solution in the real life settings of the D0 experiment, you are encouraged to join and contribute.
 

SAMGrid - the Software:

• To become a SAM-Grid system developer, look at Getting Started
  
• The Monitoring System
• The SAMGrid software administrator and user manual
• SAM-Grid Release Cut
  
• Future Production Paths(Doc)
  

Individual Product Documentation

• Execution site documentation:
  
  • The Grid/Fabric interface: the jim_job_managers (html | doc)
  • The local scheduler grid adaptation (html | doc)
  • The intra-cluster job-related file management: the jim_sandbox (html | doc)
    
  • Application-sensitive Resource configuration (Aug 01, 2005 release cut) (pdf | doc)
    
• Client site documentation:
  
  • the jim_client documentation and examples
• Configuration management documentation:
  
  • The jim_config documentation
  • The xmldb_client documentation
• Security Infrastructure documentation:
  
  • The draft of the SAM-Grid security model (doc| pdf) with an animated slide for job submission (ppt).
  • Web authentication with x509 certs and Kerberos. [Windows] [Linux]
    

D0 Grid Production Computing Initiative (D0GPCI)

• Operational Documents resulting from the initiatives (https://plone3.fnal.gov/SAMGrid/Wiki/).
• D0GPCI Project documentation

Work in Progress: General Documents

• Use cases
• Technology review document (MSWord)
• Architecture (information flow): MSWord, PS. Also available the architecture diagrams in ppt ( old / new style).
• The SAMGrid package dependencies at run time (Dec 17, 2002) (ps). OLD
  
• Remarks from the deployment of the SAM-Grid for SC2002 (Nov 2002).
  Our panel at SC2002. A collage of screen shots.
• Native XML Database - A Review (Feb 2003) (pdf).
• Some useful openssl commands: converting grid certificates from p12 format to pem and other miscellaneous commands.
• The JIM V1 packaging and installation strategies (MSWord| pdf). See also the JIM V1 package dependencies (Feb 05, 2002) (ppt). OLD
  
• A schematic view of the SAM-Grid suite (JIM and SAM) focussing on service cardinality and logistics (ppt| jpg).
• The SAMGrid software administrator and user manual.
• Lesson learned during the deployment of SAM-Grid for DZero Montecalo production
• The Work Breakdown Structure for Reconstruction with JIM (word | html)
• The SAM-Grid / LCG Integration project:
  Project work page : SAMGridLCGStatus
  "SAM-Grid / LCG Interoperability project: Architecture and Plan" (pdf).
  "Report of the trip to Lyon" (pdf).
  "Moving the test bed to production: WBS" (pdf| doc).
  Talk on the interoperability system at the "Joint OSG and EGEE Operations Workshop - 3" (ppt)
• SAM-Grid / Runjob Integration project (pdf)
• Miscellaneous Diagrams of the SAM-Grid system: Typical Disk Configuration at the Execution site (pdf | ppt); Execution Site Components (pdf | ppt); The SAM-Grid Job Management Services (pdf); Flow of Job Submission on the SAM-Grid (pdf)
• Integrating VOMS with SAM-Grid. Proposal Document - Revision 2 (word | pdf)
• Controlling the flow of grid jobs to the execution site. Proposal document (pdf) and implementation (pdf | doc)
• Release Cut of Aug 01, 2005: upgrade instructions and configuration documentation (pdf | doc)
  
• Supporting Phase Dataset for MC recovery jobs. Proposal Document (pdf|MSWord)
• Release Cut of Nov 01, 2005: upgrade instructions.
• Plans for the migration of JIM from SAM v5 to v7: current plan; initial plan.
• Plans for the migration to new samgrid server: plan.
• Samgrid release Notes for Sam V7 support: Draft-I
• Key SAM-Grid nodes Standard Configuration and Installation Instructions: Forwarding node and Grid Queuing node
• How to get the PBS Job Id of a job running on CAB and submitted to the SAM-Grid ( doc)
• Documentation on the SAM-Grid alarming service (Feb 08, Andrew Baranovski) ( doc)

SAMGRID FAQ

• FAQ's for Shifters
• Configuration related FAQ's(partially obsolete)

Related Papers and Thesis

• T. Kurca, "Grid Computing at the DØ Experiment", Proceedings of 2007 Nuclear Science Symposium and Medical Imaging Conference, Honolulu, Hawaii, Oct 27 - Nov 3, 2007 (pdf).
• B. Abbott, A. Baranovski, M. Diesburg, G. Garzoglio, T. Kurca, P. Mhashilkar, "DZero Data-Intensive Computing on the Open Science Grid", presented at Computing in High Energy Physics (CHEP07), Victoria, British Columbia, Canada, Sep 2007; published in the "Journal of Physics: Conference Series (JPCS)" (pdf). Slides also available in ppt format.
• A. Iamnitchi, S. Doraimani, G. Garzoglio, "Filecules in High-Energy Physics: Characteristics and Impact on Resource Management", Proceedings of the 15th IEEE International Symposium on High Performance Distributed Computing (HPDC-15), Paris, France, June 2006 (pdf)
• T.S. Reddy, "Bridging Two Grids: The SAM-Grid / LCG integration Project", Thesis of Master in Computing Science, The University of Texas, Arlington, May 2006 (pdf)
• T.S. Reddy, D. Levine, G. Garzoglio, A. Baranovski, P. Mhashilkar, "Trust Model and Credential Handling for Job Forwarding in the SAM-Grid/LCG Interoperability Project", submitted to the 7th IEEE International Conference on Grid Computing (GRID 06), Barcelona, Sep. 2006 (pdf)
• J. Snow, D. Wicke, M. Diesburg, G. Garzoglio, G. Davies, "DZero Data Reprocessing with SAM-Grid", in Proceedings of Computing in High Energy Physics (CHEP06), Mumbai, India, Feb 2006 (pdf)
• G. Garzoglio, A. Baranovski, P. Mhashilkar, T. Kurca, F. Villeneuve-S�uier, A. Rajendra, S. Reddy, T. Harenberg, "The SAM-Grid / LCG interoperability system: a bridge between two grids", in Proceedings of Computing in High Energy Physics (CHEP06), Mumbai, India, Feb 2006 (pdf)
• G. Garzoglio, A. Baranovski, P. Mhashilkar, L. Perkovic, A. Rajendra, "A Case for Application-Aware Grid Services" ,in Proceedings of Computing in High Energy Physics (CHEP06), Mumbai, India, Feb 2006 (pdf)
• S. Veseli, "SAMGrid Web Services", in Proceedings of Computing in High Energy Physics (CHEP06), Mumbai, India, Feb 2006 (pdf)
• G. Garzoglio, "A Globally Distributed System for Job, Data and Information Handling for High-Energy Physics"; Ph.D. Dissertation, DePaul University, Chicago; Dec 05 (html).
  Ph.D. Research Proposal, Sep 04 (pdf)
• A. Rajendra, "Integration of the SAM-Grid Infrastructure to the DZero Data Reprocessing Effort", Thesis of Master in Computing Science, The University of Texas, Arlington, Dec. 2005 (pdf)
• B. Balan, "Enhancements to the SAM-Grid Infrastructure", Thesis of Master in Computing Science, The University of Texas, Arlington, Dec. 2005 (pdf)
• A. Nishandar, D. Levine, S. Jain, G. Garzoglio, I. Terekhov, "Extending the Cluster-Grid Interface Using Batch System Abstraction and Idealization", in Proceedings of Cluster Computing and Grid 2005 (CCGrid05), Cardiff, UK, May 2005 (pdf)
• A. Nishandar, D. Levine, S. Jain, G. Garzoglio, I. Terekhov, "Black Hole Effect: Detection and Mitigation of Application Failures due to Incompatible Execution Environment in Computational Grids", in Proceedings of Cluster Computing and Grid 2005 (CCGrid05), Cardiff, UK, May 2005 (pdf)
• A. Nishandar, "Grid-Fabric Interface For Job Management In Sam-Grid, A Distributed Data Handling And Job Management System For High Energy Physics Experiments", Thesis of Master in Computing Science, The University of Texas, Arlington, Dec. 2004 (pdf)
• S. Jain, "Abstracting the hetereogeneities of computational resources in the SAM-Grid to enable execution of high energy physics applications", Thesis of Master in Computing Science, The University of Texas, Arlington, Dec. 2004 (MSWord| pdf)
• I. Terekhov, "SAMGrid Experiences with the Condor Technology in Run II Computing", in Proceedings of Computing in High Energy and Nuclear Physics (CHEP04), Interlaken, Switzerland, Sep 2004 (pdf).
• G. Garzoglio, I. Terekhov, J. Snow, S. Jain, A. Nishandar, "Experience producing simulated events for the DZero experiment on the SAM-Grid", in Proceedings of Computing in High Energy and Nuclear Physics (CHEP04), Interlaken, Switzerland, Sep 2004. (pdf). Also available the transparencies from the talk (ppt|pdf)
• G. Garzoglio, I. Terekhov, A. Baranovski, S. Veseli, L. Lueking, P. Mhashilkar, V. Murthi, "The SAM-Grid Fabric services", talk at the IX International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT-03), Tsukuba, Japan, Dec 2003; published in Nuclear Instruments and Methods in Physics Research, Section A, 534:33-37,2004(pdf|ps). Also available the transparencies from the talk (ppt)
• M. Burgon-Lyon, A.S. Thompson, I. Terekhov, R. St. Denis, G. Garzoglio, S. Stonjek, P. Mhashilkar, V. Murthi, "Experience using grid tools for CDF Physics"; talk at the IX International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT-03), Tsukuba, Japan, Dec 2003; published in Nuclear Instruments and Methods in Physics Research, Section A, 534:38-41,2004 (pdf|ps). Also available the transparencies from the talk (ppt)
• I. Terekhov, A. Baranovski, G. Garzoglio, A. Kreymer, L. Lueking, S. Stonjek, F. Wuerthwein, A. Roy, T. Tannenbaum, P. Mhashilkar, V. Murthi, R. Walker, F. Ratnikov, T. Rockwell, "Grid Job and Information Management for the FNAL Run II Experiments", in Proceedings of Computing in High Energy and Nuclear Physics (CHEP03), La Jolla, Ca, USA, March 2003. (pdf). Also available the transparencies from the talk (ppt)
• A.S. Rana, "A globally-distributed grid monitoring system to facilitate HPC at D0/SAM-Grid (Design, development, implementation and deployment of a prototype)", Thesis of Master in Computing Science, The University of Texas, Arlington, Nov. 2002 (MSWord| pdf)
• A. Baranovski, G. Garzoglio, I. Terekhov, A. Roy, T. Tannenbaum, "Management of Grid Jobs and Data within SAM-Grid", In Proceedings of Cluster 2004, Sept. 20-23 2004, San Diego, California
  Rewrite of "Planning on the grid: a status report - draft"; PPDG Document 20; EDG WP1, D0-Grid, Condor-Project; 10/02 (pdf)
• R. Walker, A. Baranovski, G. Garzoglio, L. Lueking, D. Skow, I. Terekhov, "SAM-GRID: A System Utilizing Grid Middleware and SAM to Enable Full Function Grid Computing", in Proceedings of the 8th International Conference on B-Physics at Hadron Machines (Beauty 02), Santiago de Compostela, Spain, Jun. 2002 (pdf|ps)
• G. Garzoglio, A.Baranovski, H. Koutaniemi, L. Lueking, S. Patil, R. Pordes, A. Rana, I. Terekhov, S. Veseli, J. Yu, R. Walker, V. White, "The SAM-GRID project: architecture and plan.", talk at the 8th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT-02), Moscow, Russia, Jun. 2002, Published in Nuclear Instruments and Methods in Physics Research, Section A, NIMA14225, vol. 502/2-3 pp 423 - 425 (pdf | ps)
• I. Terekhov et al., "Meta-Computing at D0"; talk at the VIII International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT-02), Jun. 2002, Nuclear Instruments and Methods in Physics Research, Section A, NIMA14225, vol. 502/2-3 pp 402 - 406 (pdf)

Developers Corner:

• TODO list for Monte Carlo Support

  Old links:

The Work Plans, Lists and DAGs:

• The strategic document in PDF or MSWord.
• The current, detailed plan, to meet the summer 2002 goals (formerly known as 6-month milestone) in PS or MSWord. This is based on the task DAG (PS , MSWord).
• The task list for the JIM v1 release HTML or MSWord.

People and Contacts:

• Lee Lueking (general management), Igor Terekhov (technical leadership), Iain Bertram (European effort coordination) strive to coordinate the project in a fully democratic fashion.
• The e-mail forum is d0-grid (view archive , members ).

Meetings

• Meeting Minutes are available from the e-mail archive
• Meeting Materials

Related projects links:

• SAM is the data handling system for D0
• Grid projects (they also reference each other) PPDG, GridPP, EDG, Dutch Grid Efforts
• D0Race (D0 Remote Analysis Coordination Effort)

To maintain this area: work in the CD CVS package called www-d0grid, update, then commit. The new docs should appear automatically within one hour. If you want to force a refresh of the official Web area, go to /www-d0/home/WWW/docs/computing/grid on d0mino, ensure that your group is www-d0grid  and mask is 002 (group-writable), then run cvs update -d; these steps are in ~garzogli/cron/cvs_www-d0grid.sh
 
 

$Id: index.html,v 1.118 2008/10/24 22:35:41 garzogli Exp $