Minutes for the Monday Database Infrastructure Pre-meeting

22 May, 2000
Present  Wyatt Merritt, Vicky White,Amber Boehnlein, Herb Greenlee, Harry Melanson, Stu Fuess, Lee Lueking (scribe), Heidi Schellman, Elizabeth Gallas, Jeremy Simmons, Matt Vranicar, Steve White, Marc Paterno (and others who I have forgotten)
 

Database Applications

The following databases applications are in some stage of  planning, design, and/or development. Some are being used at the production level.  (The levels of completion and responsible parties needs to be filled in more carefully.)
 
Database Name Requirements/plan Design Dev/Int/Prd Responsible Party
Online SMT Calibration Y Y D  
Offline SMT Calibration Y      
RCP Y      
L1 & L2 Trigger Y Y Not complete    
L3 Y      
Hardware (Control System) Y      
Run Control/Configuration Y Y D Jeremy
Sam File and Event Y Y D/I/P SAM team
Online CALCalibration Y      
Offline CAL calibration Y      
Online Muo Y N D  
Offline Muo        
Calibration and Speakers' Bureau Y Y D/I/P  
Releases request Y Y ?  
VLPC calibraiton Y N D  
CFT/CPS/FPS N      
Data processing Management N      
Monte Carlo Management Y Y  D  
Online Luminosity  Y N    
Offline luminosity and streams N Driven by online N    

Jeremy raised the issue that the design of the Run Control is being held up by the Trigger database not being ready since there is a relationship between the crate/card configuration and the trigger setup, which is related to run. There are various options for managing the run control information, but it is currently being distributed in a file which always has the same name and is very hard to be sure is correct, for example on the farm when reconstructing data.  This needs to be resolved soon.
 

Procedure for Rolling Out a Database

The procedure for making a database application, from plan to production includes the following:
 
  1. Rough draft. Plan with rough idea of what the functionality, data and relationships among the tables might be.
  2. Use Oracle Designer to create the schema design, and designer repository.
  3. Employ the DDL which is created from Designer to create the tables, keys, triggers, etc.
  4. All development is performed using a "development" instance of the database. DDL saved in cvs.
  5. An INT instance is created with controlled DDL and thoroughly tested.
  6. When the application is ready for production use, DDL has version control imposed and the application is transitioned to PRD database instance.

Current Database Hardware


On-line has d0ola for development and d0olb is planned for production. Offline has d0ora1 (currently development) and d0ora2 (currently integration and production).
 

D0OM-Oracle and CORBA Backends


There are currently two ways for D0 applications to connect to the database through the d0 database server mechanism. D0OM-CORBA  has support only for reading information from the database and can NOT write or update the database. A specialized manager would need to call a tailored DB server  to provide additional functionality.

The framework needs to support multiple ORB users. There was an early discussion about this in a March SAM meeting and Steve White is working on a  set of CORBA utilities which will be distributed through the normal SAM distribution channel, and through the D0 code tree. Jim K. is prepared to use these in the D0Framework.
 

Database Servers

Database servers can run on any platform as they communicate to Oracle via  SQL*NET.  Currently, all the db servers run on the database server machines. Eventually, there may be a need for a caching server to be included in the calibration server, where this would best be operated is still to be determined. It might make sense, for example on the farms, to run such a server near the worker nodes (d0bbin might be appropriate). There may also be a need to run an additional calibration server on the central-analysis machine for analysis users.

There is a standard db_server_gen tooled to generate most of what is needed to make a database server for  a given set of database tables. Documentation for this is available though could be improved.

Online has created its own "custom" db server which does not use the CORBA layer since it is too slow. It is being discussed that we would like to create a  special flag (like  --nocorba) so the  db_server_gen can be made to to create what the on-line  needs. Getting data from the online to the offline is being worked on and some will be done with advanced queuing. Some of the larger sets of information will employ faster techniques. The run control is done, and the SMT is being worked on.
 

Other Ways to Access DB info

Al Ito and Veronica Sorin have been using Excel spreadsheets to enter alignment constants intot he database.For the muon chamber constants, the data is entered into a spreadsheet, then massaged and both the raw information and the new info is added to the database.  Web serving on D0world is not set up like the ora machines with some kind of configuration control.  This needs someone to clean it up. There are some cases of special GUIs that have been developed as interfaces for examining and manipulating DB info. Examples of tools written in Java include the L1/L2 trigger worksheet which allows drag and drop features for working with trigger sets, and the sam project editor and admin console.
 

Security

The issue of security was raised . Currently, none of the Oracle passwords are encrypted and there is some risk with these open passwords. It is considered, however, to be sufficiently obscure that we do not expect risk large enough to justify  the effort required . There is an Oracle Kerberos solution which would cost several $10's of thousands for the number of client licenses we have.