CORRELATOR SCIENCE OPERATIONS REPORT, EVN MkIV & SFXC DATA PROCESSORS AT JIVE EVN TOG MEETING, August 2011, Arecibo 21 August 2011 (statistics cover 27 January 2011 - 20 August 2011) Bob Campbell SCIENCE OPERATIONS Sessions and their Experiments The table below summarizes projects correlated, distributed, and released from 27 January to 20 August. The table lists the number of experiments as well as the network hours and correlator hours for both user and test/NME experiments. Here, correlator hours are the network hours multiplied by any multiple correlation passes required. User Experiments Test & Network Monitoring N Ntwk_hr Corr_hr N Ntwk_hr Corr_hr Correlated 35 282 378 16 44 44 Distributed 38 318 432 17 47 47 Released 36 299 408 13 34 34 The following table summarizes by session the user experiments with activity since the previous TOG meeting , with an additional column for experiments not yet distributed (entries = remaining to do / total). The table omits experiments from session 2/2011 abandoned by the PI prior to correlation (2 exps, 28 hr). N_to.corr Corr.hrs N_to.dist N_SFXC N_ToO session 3/2010(d) 0/18 0/232 0/18 2 session 3/2010(e) 0/1 0/11 0/1 1 Nov-Feb e-VLBI 0/12 0/72 0/12 3 session 1/2011(d) 0/12 0/172 0/12 3 session 1/2011(e) 0/3 0/52 0/3 Mar-May e-VLBI 0/9 0/58 0/9 3 Mar d-VLBI 0/1 0/3 0/1 1 session 2/2011(d) 13/20 203/265 17/20 7 2 Some landmarks: e-VLBI during disk sessions for user experiments continues as a normal mode of operation. Session 1/2011 saw the longest continuous e-EVN session (64.5hr including the preliminary test/clock-search period) and the longest single e-EVN experiment (48hr). 7 stations (Ef, Wb, Ys, Ur, Sv, Zc, Bd) provided Mark5B recordings exclusively in session 2/2011. On provided parallel Mark4(5A)/DBBC(5B) recordings in an L-band Gb/s user experiment from session 2/2011 (EM084B). Sh will provide Mark5B recordings starting from the August e-VLBI day, after a successful test earlier in the month. Km had its first Gb/s fringes, using the Mark4 DAS sent from Wb, in the X-band NME from session 1/2011. Later in separate tests using the Chinese digital back ends over June-July Sh, Ur, and Km achieved 2 Gb/s fringes. These 2 Gb/s tests also marked the first time that JIVE correlated 32 MHz subbands (on SFXC). First EVN fringes from VERA_Ishigaki-jima in session 1/2011 (ER026A at 5cm). The recent trend of more ToOs, more out-of-session observations, and a growing number of new stations/back-ends/observing modes to test looks set to continue or even accelerate going into the future. Astronomical Features: The current play-back line-up is 14 Mark5As, 2 Mark5Bs, 1 Mark5B+, and 7 Mark5Cs (possible to jump from one Mark5 'flavor' to another, with various selection rules/transition probabilities applying to the possible transmogrifications). The MkIV correlator is still limited to a maximum of 16 stations for a single pass. SFXC could instead accept 24 stations, with the further advantage of not having to worry about the Mark5-flavor of the participating play-back drives. We have correlated 7 user experiments on the SFXC software correlator, and session 2/2011 brings another 7. Our general priorities for the transition to SFXC focus on experiments needing a feature that the MarkIV doesn't adequately provide, and then those that could avoid multiple passes. For the time being, we're continuing to use the MarkIV for multi-epoch experiments that have already begun on the MarkIV. Altogether, the SFXC-correlated experiments include 6 puslar-gating, 5 cross-pol spectral-line, and one each of near-field satellite, wide field-of-view (multiple passes on MarkIV), and a 16-station global spectral-line. Proposals have been received for multiple phase-center and >16-station observations. Correlating real-time e-VLBI on SFXC is being addressed in various tests, including the August e-EVN day just before the TOG meeting. SFXC can now run at real-time for a 9-station, 512 Mb/s mode, but new hardware to double the physical size of its cluster is in-house and being installed. For large N_sta, the processing-time would scale roughly as N_sta**2; and would have a linear dependence on data rate. There is something like a log(N_frq.pt) dependence above 128-256 frequency points per SB/pol (below 128, the correlation takes place at 128 and is spectrally-averaged down internally to the desired N_frq.pt). The potentially larger processing factors could be mitigated to some degree by avoiding multiple passes and by more reliable unattended operation. After building up a reasonably good intuition about MarkIV problems and their symptoms/diagnoses/cures in post-correlation review and FITS-file preparation, we are beginning again with the output of SFXC. So far, things seem more straightforward from data-quality aspects, and we are adjusting the post-correlation/pre-FITS tactics and scripts to be more optimized for SFXC output data. NETWORK SUPPORT The automatic-ftp feature continues to be exercised in all NMEs. Stations send the specified portion of a scan directly to JIVE for correlation with SFXC. Correlation results go to a web page within a couple hours, and Skype chat sessions during the NME provide the station friends with even more immediate initial feedback. With 2-3 ftp fringe-test scans per NME, there is opportunity to find, fix, and verify a problem and its solution within a single NME. We continue to process all experiments via the pipeline, with results being posted to the EVN Archive. The pipeline provides feedback on stations' general performance and in particular on their gain corrections, and identifies stations/frequency bands with particular problems. Jun will present a more detailed calibration report in his presentations. After the PIs had deposited their session 2/2011 schedules, Mc noticed data-throttling in Gb/s experiments. We provided separate 512 Mb/s schedules, using 1-bit instead of 2-bit sampling, for remaining L-band Gb/s experiments in order to bypass this problem for the remainder of the session. No time since the previous TOG meeting has been found to look further into testing an ~80-MHz lowered Gb/s C-band frequency range in light of RFI and/or front-end limitations at KVAZAR stations. We hear unofficially from CRAF sources that the frequency range above 5000 MHz may become less favorable in the future (outside primary or secondary protection), so there may also be a long-term motivation for lowering the frequency range anyway. In recent sessions, there have been experiments going to two or three different correlators. There remain occasional problems with individual packs containing data intended for more than one correlator. USER SUPPORT The EVN Archive continues to provide web access to the station feedback, standard plots, pipeline results, and FITS files. Access and public-release policy remain the same. The archive machine continues to have 12.8 TB of dedicated disk space, with a buffer of another 1.8 TB that also houses the pipeline work area. In addition to the tape back-up of the Archive, we have a mirror to a machine physically in Westerbork. We now have 10.6 TB of FITS files in the Archive, a gain of 0.8 TB since the previous TOG report (26 jan). We continue to contact all PIs once the block schedule is made public, and to check over schedules posted to VLBEER prior to stations downloading them. This occupies occupies a great deal of time in the fourth to second weeks before the start of the session, but helps to prevent avoidable errors in the observations themselves. In this transition period where the stations may be changing back-ends and/or participating at new frequencies faster than we can accommodate their new characteristics into sched, we are providing PIs with plug-in setini sections and a separate stations.dat file to incorporate these improvements, and if necessary editing uploaded vex/skd files prior to releasing them to .latest (via Alastair). We've made a draft table showing station contacts for ToO proposals. The intention was to link to this from the "Proposing" part of the EVN Users' Guide, whose direct URL after following the Users' Guide link is: www.evlbi.org/proposals/prop.html There, the third bullet discusses ToOs (it now has a bit of vertical white space around it). The draft table is at www.evlbi.org/proposals/ToOtbl.html Currently, the Users' Guide says to collect this information from the EVN contacts page (thus the PI would have to click once per station from a table to get all the relevant e-mail addresses). A link to this table could replace the reference to the contacts page. Most stations have provided feedback -- I'll query Ar & Wz at the TOG.