European VLBI Network Technical and Operations Group Minutes of Meeting held at MDSCC, Robledo - 30th June 2003 Introduction ------------ Around 30 people attended the TOG meeting held at MDSCC on 30th June 2003. The following text summarises the most important aspects covered during the meeting. The Director of MDSCC welcomed the TOG to the site and wished them a successful meeting. Peter Poon (JPL) also welcomed the TOG and thanked MDSCC for hosting the meeting. Poon also gave a brief introduction to the DSN. 0. Approval and Last Minute Additions to Agenda ----------------------------------------------- The minutes of the last meeting were unanimously approved. Yebes asked for the addition of an agenda item - the frequency changes at Yebes for the 40-m (new) telescope 32 GHz band continuum. Parsley asked for the issue of the disposal of tapes to be added to the agenda. 1. Action Items from Last Meeting --------------------------------- A review of action items from the last meeting was made; ACTION ITEM: 6 & 18cm dual polarisation receivers at DSN antennas. A proposal had been submitted and was in progress. ACTION ITEM: Stations who wish to take advantage of the recorder tests to send test tapes to JIVE. These are usually done by most stations. ACTION ITEM: Torun BBC problems. 4 additional BBCs had been sent from Onsala and this has solved the problem. ACTION ITEM: CONWAY to investigate whether spare BBCs at SEST could be procured. Done (see above). ACTION ITEM: Schedule checking: VENTURI reported that she has in the past checked the Block Schedule. It was agreed that Gunn would take over this task. ACTION ITEM: GARRETT to investigate whether individual schedules can be checked by JIVE Support Scientists. This would probably be too much work for JIVE staff and would only be done if requested. An additional action item was added; CAMBELL to investigate whether schedules can be checked prior to observation since they are already checked prior to correlation. ACTION ITEM: REYNOLDS to check whether TSYS data is now available from DSN antennas and report back to POON and GARRETT. Reynolds reported that this information is normally available. Garcia reported that the logging was not working in the May 2003 session but will be working again by the end of the year. ACTION ITEM: Barrel rolling - several EVN telescopes now using this mode for geodesy, not yet supported at JIVE. CAMBELL was asked to clarify whether barrel-rolling is (will be, should be) available at JIVE. ACTION ITEM: BURGESS to repost information about availability of MkIV decoders. Done. ACTION ITEM: BURGESS to revise list of spares, in particular what should now be held locally/centrally as disk-based systems replace tape. Ongoing. ACTION ITEM: GARRETT to request POLATIDIS to make a new call for station input with respect to EVN STATUS sheet. Done. ACTION ITEM: MkIV Spare formatter - discussion to be led by VOLVACH, Alef, Himwich & Garrett with respect to moving spare formatter to Simeiz. The Directors were not keen on this idea. No further discussion. ACTION ITEM: HIMWICH/VAN LANGEVELDE to close the loop with respect to final continuous post-pass at the end of experiments, plus 2 minute read-back tests. This was in principle solved. ACTION ITEM: All stations to ensure feedback is provided via the logs, the EVN experiment feedback facility and jops@jive.nl. This is generally happening but stations were asked to continue providing feedback. ACTION ITEM: STATIONS to upgrade to the latest version of the FS (9.5.12). Done. ACTION ITEM: BURGESS to check whether VLBA racks are doing the right thing with respect to changing attenuation by inserting test signals and reporting back to EVNtech. Not done. ACTION ITEM: GARRETT to contact EVN scheduler and CBD to discuss short calibration experiments to be scheduled and included within the block schedule. Done. ACTION ITEM: GUNN to act as PI for these calibration experiments. Done. ACTION ITEM: REYNOLDS & HIMWICH to investigate the production of UVFLG files from FLAGR data written to logs. Done. ACTION ITEM: ALL STATIONS to ensure that the FS FLAGR can run at their telescope. This was apparently not working at all stations. ACTION ITEM: REYNOLDS to compare FLAGR data with correlated (drop-out) data in case there are latencies within antenna control systems. Done. ACTION ITEM: TUCCARI to contact LBA with respect to their experience with Digital Filters. Results not yet known. ACTION ITEM: GARRETT to set-up a small group to discuss the wide-ranging implications of employing disk-based recording systems within the EVN. Done. ACTION ITEM: Burgess to begin purchasing various components. Done. ACTION ITEM: STATIONS to contact MPIfR and JIVE for replacement heads if they have no spares locally. It was agreed that there was little point in replacing heads and stations with failed heads should switch to Mk5. 2. Observing Sessions --------------------- Alef asked whether stations were happy with the Block Schedule format and whether more information should be available. Stations agreed that the present format was acceptable. Alef asked whether stations were finding the scheduled calibration runs (CLXXX) useful. The response was a unanimous yes. Stations were also happy with the FTT (Fringe Test Tape) procedures now in place although Gunn requested that FTTs were not scheduled on a Monday since it was sometimes difficult to ensure system reliability immediately after a weekend. A Monday morning FTT did not allow for any necessary fault repairs. Alef asked whether the NME experiments should be conducted at 512 Mbps. Paragi commented that JIVE should decide what and how to test the network during NMEs and this usually takes into account the typical session experiment. Colomer suggested that ALL EVN stations should be included in NMEs. Paragi responded that this depends on the user load and whether an NME is done at a given frequency. JIVE requests NMEs at all frequencies from the scheduler and this is normally provided. Several people suggested that the NME should be the first experiment at a given frequency. This is done by the scheduler if possible. Garcia commented that Robledo would need 2 days notice to perform an NME and Paragi was encouraged to include this station. Paragi was asked to communicate these requests to the scheduler. Gunn encouraged stations to perform additional CLXXX-type calibration runs during convenient gaps in the schedule. 3. Reliability and Performance of the EVN ----------------------------------------- Paragi gave a short report on the reliability and performance of the EVN over recent sessions. All sessions had problems but overall the operation was good. A discussion was made of the reasons for failures at various stations during previous sessions. Progress was generally good towards solving these. Gunn reported that the problem with Cambridge single versus dual polarisation was essentially solved after recent tests. Steps had also been taken to prevent a repeat of the Lovell LO problem in November 2002. Paragi commented that a good deal was learnt during tests of data download using Mk5 systems (and PC-EVN). Almost real-time fringe tests were performed during the May 2003 session. It was agreed that JIVE should aim for performing near real-time fringe tests for all stations at all frequencies as soon as possible. Paragi said that the recording quality was mostly stable with odd problems. He requested stations to switch on read-back tests when drudging schedules and to check the FS scan checks after each scan. Paragi also brought up the issue of phasecal analysis. There was some discussion about phasecal analysis and the analysis of observed BBC bands in general. Ritakari described software developed at Metsahovi for the analysis of signals from BBCs. It was decided that Alef should request this software from Mujunen for use at other stations. Maccaferri was asked to consider putting the Metsahovi bandpass analysis software into the Fields System. Parsley was asked to enquire of Garrett whether JIVE could consider doing phasecal analysis to improve EVN reliability. Stations were encouraged to measure the band passes of BBCs and ensure the best are placed in the first 8 positions (ensuring the narrow band filters are in the first four). Graham also suggested that stations should check the sampler statistics in the BBCs. Graham was asked to send details of these test to EVNtech and Paragi was asked to encourage Urumqi to perform these tests. Reynolds was asked to schedule an NME to look at 16 MHz bandpasses and BBC sampler statistics. The subject of frequency agility was discussed at length. This was mostly driven by the recent experience of a user experiment that had very complex frequency changes between 18 and 21cm. Gunn commented that at JB this schedule took approximately eight man-hours to prepare. It was unanimously agreed that the EVN PC should do more to alleviate these problems, by requesting PIs to be specific about frequency setups/changes and perhaps communicating better with stations about what is possible/required. Gunn also mentioned that MERLIN had requested additional information from the EVN PC concerning setups for joint MERLIN/EVN experiments. Alef was asked to pass these comments to the EVN PC Chair. Paragi emphasised that stations must use the EVN feedback pages and NRAO's tape TRACK system. 4. Amplitude Calibration ------------------------ Conway gave an overview of the status of amplitude calibration within the EVN. He reported that in the last year there had been a lot of activity in this area and thanked Himwich for his efforts in the Field System implementation. Conway then reviewed the process of achieving good amplitude calibration, from running ONOFF, the production of reliable RXG files, the analysis of calibration data with GNPLT, the production of continuous TPI measurements during user experiments to the creation of ANTAB files for AIPS. Stations reported that they were happy with the scheduled CLXXX experiments for gathering calibration data. Gunn urged stations to use gaps in the observing session to gather as much data as possible. If those responsible for local calibration at stations are unfamiliar with suitable calibration sources for various frequencies, they were encouraged to seek the aid of a local astronomer. Conway can also provide advise on calibration sources. Conway presented some results of his measurements of VC non-linearity at Onsala. There was some discussion of these results. Several reasons for non-linearity were discussed and stations were encouraged to investigate the effects themselves. Conway and Graham were requested to produce a recipe for stations to follow for this investigation. Himwich added that stations should look at the use of the IFADJUST command in the Field System. A brief discussion was made of the GNPLT program for the analysis of gain curves and Tcal. Some stations, notably Torun, reported problems with the software. There is the possibility that some additional work will be carried out on the software, so stations were encouraged to fully investigate their implementation and running of the software as soon as possible. Any comments or problems should be reported directly to Himwich and Conway. Reynolds gave a brief talk concerning the production of ANTAB files. Various minor problems were reported. It was decided that, although ANTAB files would for the time being be produced centrally at JIVE by Reynolds, stations should produce their own ANTAB files for user experiments during the May 2003 session. Reynolds will circulate details of the necessary scripts soon. Stations are requested to run the scripts and report any problems in producing ANTAB files directly to Reynolds as soon as possible. The design of the VLBA radiometer was briefly discussed. Graham was charged with investigating the requirements for the use of this equipment for amplitude calibration. Finally, the TOG chairman recommended that people responsible for local calibration at EVN stations try to attend the TOW meeting at Haystack, which is currently scheduled for September 2003. The Chairman also recommended that the TOG should hold another amplitude calibration workshop at the next TOG meeting. 5. Mk5 ------ Smythe gave a status report on the Mk5 recording system. Details can be found at http://web.haystack.mit.edu/mark5/Mark5.htm. A discussion ensued concerning the experience of Mk5 users within the EVN. Alef requested Smythe to notify EVNtech about any changes or updates to Mk5 software etc. Graham summarised the results of EVN Mk5 tests during the May 2003 session. Mc, Ef, Nt, Wb operated correctly although Wb did not have the rack ID set properly. Smythe distributed rack ID kits to many stations. Nt did not do bank switching during these tests; reason unknown. A test with Jb late in the session at 22 GHz correlated at Bonn up to 1024 Gbps (32 and 64 track modes). Four stations tested positively at 512 Mbps and 1024 Mbps although Nt had some problems at 1024 Mbps. All indications are that Mk5 is working efficiently and correctly at those stations now with units. The change to Mk5 is really only hampered by the problem of disk purchase. There was some discussion about the change over to Mk5 operations. Gunn commented that Cambridge should be moved to Mk5-only as soon as possible and Alef made a similar request for Effelsberg. Before this takes place a confirmation that Mk5-only stations can be correlated is required from JIVE. Reynolds was unsure about the time-frame required and was asked to get clarification from JIVE. Alef reported that geodesists are moving to Mk5 sooner than expected (most by the end of 2003). It was suggested that the November 2003 session contains a 6-7 telescope high bit-rate test of the Mk5 systems. Also, that Cm should be Mk5-only and Effelsberg on disk AND tape during the November session. There was also some discussion of Mk5 logistics. Alef recommended that all stations put disk packs together locally as this experience may be required later for disk replacement. It was suggested that a session at the TOW meeting at Haystack could be devoted to this. It was also agreed that JIVE and Bonn start sourcing packing materials, i.e. foam inserts. Stations should purchase suitable boxes for their 8-packs. JIVE favours 2 packs per box and is busy sourcing these materials. JIVE was requested to report packing requirements to stations as soon as possible. Gunn complained about the old-fashioned glue-on labels now appearing on Mk5 disk-packs and agreed to source an alternative, if possible. 6. Field System Developments ---------------------------- Himwich gave a brief summary of recent Field System developments. Of importance to the EVN were mostly the changes related to amplitude calibration. There was some discussion of the implementation of FLAGR at stations. Reynolds was asked to provide Alef and Himwich with information about which stations are not running FLAGR correctly. Some further discussion was made of GNPLT and Himwich emphasised the need for stations to provide feedback on the program. Another topic discussed was the proposal to include LO setup commands (station specific) whilst drudging user experiments. Himwich asked stations to provide information about what is required locally. Himwich ended by drawing attention to the IVS TOW meeting at Haystack MIT at the end of September. 7. Sched Developments --------------------- Reynolds gave a brief summary of recent SCHED developments. Mots were minor but there has been some development for Mk5 use. There was a short discussion about the logistics of Mk5 schedules (e.g. disk changes, different disk sizes etc). Reynolds reported that these issues were in hand. Reynolds mentioned that stations would soon be asked to provide updated information on antenna slew rates, acceleration and settle times. Stations were requested to await the request from JIVE for this information. 8. e-VLBI Status ---------------- Dan Smythe and Steve Parsley reported to the TOG on recent results of eVLBI tests in the US and Europe. The past year has seen a significant increase in capabilities and plans are currently afoot for most EVN stations to get adequate fibre links to JIVE within the next year or so. Details of these issues are generally available on the web. 9. Spare Parts --------------- Wunderlich gave a short presentation concerning spare parts in the EVN. He said Paul Burgess had done a good job of collecting information and should be encouraged to continue this task. However, two years of efforts have failed to provide a satisfactory inventory. He explained that the situation is very bad; that even some components in the Mk4 formatter were no longer available due to progress in the technology. Whether spares are kept locally or centrally is no longer the issue since they are unavailable. The crucial issue is whether any stations has the components at all. For tape drives there were probably enough spares to last until the end of the tape era. Only one spare Mk4 formatter was available in Bonn. The most crucial components are Mk4 formatter FPGA's which are no longer available. Wunderlich also pointed out that there is no spare EVN decoder and that many of the BBC components are obsolete. Additional information is required from stations and correlators to ensure the life of units until new digital BBCs can be introduced. It was suggested that all stations go through their stocks and report to Paul Burgess what components they have available. 10. Station and Correlator Reports ---------------------------------- No time was given to individual station and correlator reports. These were distributed through EVNtech prior to the TOG meeting. 11. AOB ------- Alef directed the TOG to review reports from Tuccari concerning a recent visit to Bear Lakes, and a report on the development of digital BBCs. These were distributed separately. Also, a report on the water vapour radiometer was presented by Roy. Parsley commented that JIVE wanted to dispose of old thick tapes in its possession. It was suggested that JIVE contact stations individually to see if they want their tapes back, otherwise they will be destroyed. 12. Time and Place of Next Meeting ---------------------------------- No decision was taken concerning the date and location of the next TOG meeting. Summary of Action Items ----------------------- ACTION ITEM: CAMBELL to investigate whether schedules can be checked before observation since they are already checked prior to correlation. ACTION ITEM: CAMBELL was asked to clarify whether barrel-rolling is (will be, should be) available at JIVE. ACTION ITEM: BURGESS was asked to continue to revise list of spares, in particular what should now be held locally/centrally as disk-based systems replace tape. ACTION ITEM: STATIONS were asked to check whether they have the check procedures for performing read-back tests and to request these tests when drudging schedules. ACTION ITEM: PARAGI was asked to communicate with the scheduler and request that, if possible, FTT are not performed on a Monday, that NMEs should be done at all scheduled frequencies, that NMEs should, if possible, be the first experiment at a given frequency and to include Robledo in NMEs where appropriate. The TOG recognised that some of these issues were already considered by the scheduler. ACTION ITEM: ALEF should request software for the analysis of BBC spectra from Mujunen for use at other stations. ACTION ITEM: MACCAFERRI was asked to consider putting the Metsahovi bandpass analysis software into the Field System and include the process of Mk5 data extraction. ACTION ITEMS: STATIONS are encouraged to analyse BBC bandpasses and sampler statistics. ACTION ITEM: ALEF to ask JIVE to aim for performing near real-time fringe tests for all stations at all frequencies as soon as possible. ACTION ITEM: PARSLEY was asked to enquire of Garrett whether JIVE could consider doing phasecal analysis to improve EVN reliability. ACTION ITEM: STATIONS to ensure feedback is provided via the logs, the EVN experiment feedback facility and jops@jive.nl, and to use TRACK. ACTION ITEM: ALEF to pass to EVN PC Chair the comment that the PC should do more to alleviate problems due to frequency agility, by requesting PIs to be specific about frequency setups/changes and perhaps communicating better with stations about what is possible/required. MERLIN has also requested additional information for joint MERLIN/EVN experiments. ACTION ITEM: GRAHAM to send details of BBC sampler statistics tests to EVNtech. ACTION ITEM: PARAGI to encourage Urumqi to perform sampler statistics tests. ACTION ITEM: REYNOLDS to schedule an NME to look at 16 MHz bandpasses and BBC sampler statistics. ACTION ITEM: CONWAY and GRAHAM to provide stations with a recipe to investigate VC non-linearity and how to correct any serious problems. ACTION ITEM: STATIONS to investigate VC non-linearity and, if possible, adjust VCs, and report results to EVNtech. ACTION ITEM: STATIONS to provide feedback on GNPLT to Himwich and Conway. ACTION ITEM: REYNOLDS to circulate instructions and scripts for stations to produce test ANTAB files for user experiments in the May 2003 session. ACTION ITEM: GRAHAM to investigate the requirements for the use of the VLBA radiometer for amplitude calibration. ACTION ITEM: REYNOLDS to enquire at JIVE about time-frame for correlator to move to Mk5-only stations (partial and complete network). ACTION ITEM: PARSLEY to provide details of Mk5 8-pack packing requirements to stations as soon as possible (i.e. boxes and foam inserts). ACTION ITEM: REYNOLDS to provide Alef/Himwich with information concerning which stations are not running FLAGR properly. ACTION ITEM: STATIONS to provide Himwich with requirements for implementing automatic LO switching (e.g. 18 to 21 cm) whilst drudging user experiments. ACTION ITEM: REYNOLDS to request updates to slew rates, acceleration rates and Tsettle values from stations. ACTION ITEM: STATIONS to go through their stocks and report to Paul Burgess what components they have available. ACTION ITEM: JIVE to contact stations to see if they require their thick tapes back, otherwise these should be destroyed.