CORRELATOR OPERATIONS REPORT, EVN MkIV DATA PROCESSOR AT JIVE EVN TOG MEETING, April 2009, Torun 13 April 2009 (statistics cover 13 Sep 2008 - 13 Apr 2009) Bob Campbell SCIENCE OPERATIONS The table below summarizes projects correlated, distributed, and released from 13 September to 13 April. The table lists the number of experiments as well as the network hours and correlator hours for both user and test/NME experiments. Here, correlator hours are the network hours multiplied by any multiple correlation passes required. User Experiments Test & Network Monitoring N Ntwk_hr Corr_hr N Ntwk_hr Corr_hr Correlated 35 366 439 25 137 137 Distributed 38 402 491 20 106 106 Released 41 422 511 23 118 118 The following table summarizes by session the user experiments from sessions since the previous TOG, with an additional column for experiments not yet distributed (entries = remaining to do / total). N_to.corr Corr.hrs N_to.dist session 2/2008 0/21 0/314 0/21 Jun e-VLBI 0/3 0/7 0/3 Sep e-VLBI 0/1 0/2 0/1 session 3/2008 0/15 0/231 0/15 Nov e-VLBI 0/3 0/23 0/3 Dec e-VLBI 0/3 0/24 0/3 Jan e-VLBI 0/3 0/21 0/3 Feb e-VLBI 0/2 0/22 0/2 Mar e-VLBI 0/2 0/15 0/2 session 1/2009 17/20 221/263 18/20 Apr e-VLBI 3/3 17.5/17.5 3/3 iaw apr e-blk.sched There has been some personnel movement in the Science Operations and Support Group since the previous TOG. Jun YANG began as a support scientist on 1 November 2008, bringing us back to full strength. Shortly thereafter, both Giuseppe Cimo and Antonis Polatidis left the group (GC: 1jan, to space applications group; AP: 1 feb, to ASTRON). Yurii Pidopryhora will be a half-time support scientist until the end of EXPReS (~sep'09), becoming full-time thereafter. We expect Mehreen Mahmud to complete the support-scientist team in the summer (~1aug'09). Jun has taken over the ANTAB-related responsibilities from Giuseppe. Friso Olnon from the Software Group has taken over the EVN-related responsibilities in sched from Antonis, although it is my understanding that Antonis continues to maintain the EVN Status Table. Landmarks from recent sessions: Tactics: Since session 1/2008, we have begun to correlate Gbps experiments first, to recycle as many disks as possible back to the stations for the next session (as opposed to the conventional next+1 session). For the just-finished Feb/Mar'09 session, this early recycling amounts to 122 TB. Logistics: In session 1/2009, there were well-distributed Mark 5A pack problems: *) assigning already-used VSNs to new packs (Wb, Tr) *) wrong internal VSNs (from conditioning with packs in each bank?) Cm - existing VSN put into packs Ur - StreamStor1 in a few packs *) Ur seems to have sometimes changed packs with no bank_check or bank_set entries in the corresponding field-system log at the times of the pack changes (disk_pos entries do go back to zero; pattern of disk_pos values match DirList's of the actual packs [once they're identified]). In session 3/2008, Ur seems to have been able to change packs in the middle of a (skd) scan. Schedule issues: Gbps experiments that include NRAO stations solve the current 512 Mb/s limit by recording 1-bit sampling at the NRAO stations. However, this requires different skd files for the Mk4 & NRAO stations. We've not settled on a convention for handling the naming of the skd files. We do keep things straight off-line at JIVE (neither of the skd files would be directly suitable to drive correlation; we need to make a composite one using both). In session 3/2009, there were two more such experiments (GP047, GW019B), and some confusion arose about the skd file named gp047b.skd (the EVN-appropriate Gbps file). We'll try to be more pro-active in future situations when a skd-file name deviates from what's expected from the block-schedule to forestall confusion. Network firsts/highlights: 3/2008: First time the three Russian KVASAR stations (Svetloë, Zelenchukskaya, and Badary) participated in EVN observations (at S/X, C, and L bands). First Yebes40m participation at X and S/X. 1/2009: First Yebes40m participation at 5cm; also provides first SATA packs for correlation at JIVE (not yet correlated). First participation of Yamaguchi32 in EVN observations (5cm; not yet correlated). Irbene participation in the C-band NME; bit-encoding inferred from the sampler stats seemed off, no fringes seen. Correlator firsts/highlights: Every user experiment in session 1/2009 requires the full (32-board) correlator (the addition of a second MERLIN outstation as described in "Astronomical Features" moved the few 8-station continuum experiments to 9-station ones, no longer able to fit in a quarter-correlator configuration). Data quality: Urumqi had bouts of too-fast Gbps recording during the C-band session of 1/2009. This was seen (& repaired) previously at Cm, with the culprit being the clock generator chip on the Mark 5A I/O board. Most Ur Gbps during the L-band session appear okay (not yet correlated, but implied bit-rates from disk_pos entries in FS log are appropriate for Gbps recording). Jodrell had very low sampler stats in all LCP channels in C-band Gbps experiments from 1/2009 (haven't yet looked at L-band Gbps experiments). Problem not seen in the C- or L-band NMEs. Development of Astronomical Features: Recirculation at factors x2, x4, and x8 have been tested and shown to work (the maximum recirculation factor for any given job is 16MHz/BW_SB, where BW_SB is the subband bandwidth). The minimum integration time would increase by the recirculation factor. The recirculation-capabable version of the real-time processes run exclusively at 64 correlator frames per second. The first two user experiments requiring recirculation observed in session 1/2009 (not yet correlated). A second MERLIN out-station can be included in "unused" subbands in the Cm recording for >=256 Mbps dual-pol modes. Making use of this additional out-station capability up to now has entailed copying the Cm pack in order to correlate two stations separately (with the inherent need for additional disk-space at JIVE). We can now use the multi-cast techniques developed for e-VLBI observations using multiple out-stations to correlate both stations "directly" from the single Cm disk-pack, bypassing any disk copying. We have finished three experiments from session 1/2009 using this multi-cast technique for correlating the additional MERLIN out-station.