P14 Reprocessing: Meeting of 5-Nov-2003

Agenda

1. Storing DSTs to tape.
2. Finalising certification for GridKa, Lyon and UK.
3. Problems of file delivery.
4. Status of Sites.

Minutes

Participants

Topics

1. Storing DSTs to tape:
   We will try storing DSTs directly to FNAL. This needs areful observation, to avoid bandwidth problems.
2. Finalising certification for GridKa, Lyon and UK
   Harry and Suyon need to look at Recocert histos, should send a mail to the list when a certain site is certified. (Answer expected later today) Lyon produced unselected TMBs, which can be used to make a full comparison with FNAL. Thereafter selective TMBs from Lyon can serve as "master histograms" as event lists are already approved. (Phil checked all event lists.)
3. Problems of file delivery
   Two problems may be overlapping here:
   1st: The cache on the central-router is not very large (~280GB) and had a lot of files in it. It is possible that a single slow remote station could throttle transport for everyone. We will look into expanding size of cache.
   2nd: SAM production databases and the ADIC robot were both down part of the day on Tuesday. This was normal maintenance Regular maintenance days are the first Tuesday of every month. People should expect delays on these days. We will have notices of significant downtimes sent to the reprocessing mail list.
4. Site Reports:
   • GridKa: All projects assign last week finished on Monday. Transport from FNAL very slow. Route cache was filled. Transport to FNAL working but done only for certification file.
   • SAR: Jae Yu: All Machines up and running. Certificates missing. Just before submitting MC jobs (End of week).
   • UK (Rutherford): Rod: Investigating the storage. Two files won't go through because of queue vsize limit (825MB). Successfully stored DST.
   • Nikhef: Current certification not done in Grid mode, yet. EDG: Grid mode working only within Nikhef. Problems similar to Dugans (Westgrid) ones.
   • Lyon: Waiting for data. Transport problems. Changing bbftp seems to give a factor 3. Delays of getting files to d0mino. (Lyon is staging to d0mino and then pushing files from d0mino to Lyon).
   • NPACI: Trouble storing back certification results.

   • WestGrid: Dugan "wish I was root" O'Neil: At the cpu center the jobs are "almost" running. We think the problem is still related to the network configuration. I sent a couple of requests to the sys-admins last night and will slip away from TRIUMF to chat in person with them this afternoon (this centre is on the same campus as TRIUMF).

     The storage facility is having trouble due to the firewall at the SFU site. I am negotiating there as well. If nothing is resolved by tomorrow we will meet with the sys-admins to straighten that out tomorrow afternoon.

     So, we are still at the "almost" stage on both campuses. I hope we have something useful to say in about 24 hours.

Action Items:

Next Meeting