Farm production summary for Saturday Oct 21 to Saturday 27, 2001
 


Major events

Power outage causes downtime from 6PM Saturday to around 6PM Monday as we wait for all systems
to recover

Database reboot on Wednesday the 24th corrupts metadata for ~100 processed files, not nice

home area on farm goes bad at 5AM on Friday the 26th
, recovered by 8AM - jobs in progress on worker nodes actually continue unharmed
as batch system recovers state on restart

farm weekly query for producing this report found to crash dbserver.  Farms miss the grid test as a result.

Here is the weekly processing information in another form.  Good days show ~450,000 events/day
 
 
  "         per/day  sum
10/20/01 198857 198857
10/21/01 0 198857
10/22/01 188891 387748
10/23/01 427516 815264
10/24/01 455745 1271009
10/25/01 448343 1719352
10/26/01 260758 1980110