Reprocessing: Meeting of 28-Mar-2005: 9:30-10:30 ESNet video conference

Agenda

1. News
2. Status of production
3. Status of remote site Certification
4. JIM Deployment and Remote Setup of remaining sites.
5. AOB.

Minutes

Participants

Topics

1. News
   

2. Status of production
   • WestGrid: 35 datasets submitted for production. Head node was down yesterday (to be investigated).

     Until Wednesday running ok provided merging and production is done separately. Merging poses very high load on head node due to the fact that the head nodes work as a NAT box. At that time it still wasn't using fcp. Head node ist dual 2.4GHz, 3GB RAM. The SamGrid team will help to optimise the configuration.

     From Wednesday a new version of qstat caused problems which could only be resolved on Saturday (inconsistent version of python). Grid job got held. Batch jobs that aren't in the system at that moment won't find boxNNNNNN.MMMMM/sandbox. SamGrid shouldn't require this for reprocessing. The SamGrid team will investigate.

     There seems to be a problem with the 2.4.(20?) kernel in case of heavy network load caused by kswapd.

     Yann and Dugan are hoping to get a dedicated NAT box.

   • Lyon: Biggest problems from merging. Problems with d0production group (not enough space). Merging jobs consume all resources of the head node. Suggestion is to cross-check the fcp server configuration. Size of SAM cache plus durable location is 250GB. This is only used for unmerged-production. The SamGrid team will follow up to understand the details.

3. Status of remote site Certification
   • Status of production certification.
     We added plots Wisconsin at certification page.
     SAR: Production certification done after some problems with the local cluster.

   • Status of Merge Certification of Sites
     
     • DØFarm: done.
     • WestGrid: done.
     • Lyon: done
     • SAR: UTA: done
     • GridKa: done.
     • Wisconsin: done.

4. JIM Deployment and Remote Setup of remaining sites.
   • GridKa: Maintainace at GridKa ongoing until Wednesday.
   • Prague: Bimal is testing MC. Should be ready for certification. Need to contact them.
   • UK:
     • IC: Struggeling with d0repro tool and proxies, now solved.
     • Manchester: Should be ready for certification.
     • RAL: Needs to be negotiated.
     • Lancs: Hardware about to be setup.
   • CMS Farm: Condor crashes on a recursive link in Forb. The condor configuration of Wisconsin and the CMS farm are being compared.
     

5. AOB:
   
   • Physics groups requested to start with 2004 data. We'll slowly shift over .
   • We're planning to included RecoCert into the JIM production mechanism, but it isn't foreseen for remote processing.
   • Yann observed error messages in copyd0om. Will send them to Qizhong.

Action Items:

Next Meeting