Geodetic eVLBI at Bonn Correlator

About e-VLBI

Electronic VLBI (e-VLBI) describes the transfer of observational VLBI data from radio telescopes to the correlators via high-speed network connections like fiber channel. Advantages of this technology are

higher potential data rates for greater sensitivity with the same antennas
faster turn-around time of experimental processing for use in scientific investigation
elimination of large and expensive media pool, that is currently being used.

Since the geodetic use of e-VLBI is restricted to data transfer as opposed to real-time data transfer and correlation, which is sometimes the case for astronomical observations, we refer to it as e-transfer in the following.

e-transfers at the Bonn correlator

A 10 GBps line connects the Bonn correlator to the high speed network via a commercial provider. Since late 2011, a firewall computer has been set up for the e-transfer servers.

Currently, the software used by most of the stations for transfers over high speed networks (≥ 1 Gbps) is jive5ab/m5copy, a VLBI data recorder software that allows high-speed VLBI data recording as well as fast and flexible data transfers. However, its developer was not totally satisfied with its functionality and upped the ante by delivering the etransfer server/client system (etc/etd), which enables to run server to server transfers via a client program by simply specifying two remote locations and has a built-in support for the usage of remote wildcards. The e-transfer tools support both TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) for transfers over high speed networks.

Besides we used to use Tsunami which is a another file transfer protocol that uses UDP data and TCP control for data transfers. It can be downloaded at http://tsunami-udp.sourceforge.net/. The project is based on original Indiana University 2002 Tsunami source code, but has been significantly improved and extended by Jan Wagner. As such, large portions of the program today are courtesy of the Metsähovi Radio Observatory.

TCP is the most commonly used protocol on the Internet. The biggest advantage of TCP is the so-called "flow control" which guarantees the delivery of the data that are transferred. Flow control determines when data needs to be re-sent, and interrupts the flow of data until previous packets are successfully transferred, i.e., the client re-requests the packet from the server until the whole packet is complete and identical to its original.

UDP is another commonly used protocol on the Internet. It offers speed and is much faster than TCP because there is no form of flow control or error correction. This main advantage is, however, at the same time its biggest disadvantage.

Tsunami combines both TCP and UDP; it offers data transmission with default priority for data integrity, but disabling retransmissions may as well enable rate priority. Communication between the client and server applications flows over a low bandwidth TCP connection. The bulk data is transferred over UDP.

Further alternatives are, e.g., UDT, or GridFTP . UDT is another UDP based application level data transport protocol designed for transferring huge datasets over high-speed wide area networks. It uses UDP to transfer bulk data with its own reliability and congestion control mechanisms (e.g., m5copy makes use of it). GridFTP is based on FTP, the highly-popular Internet file transfer protocol, optimized for high-bandwidth wide-area networks.

During the last years, the number of stations, that transfer their observational data via high-speed network connections to the correlators, has increased significantly. This necessitates some form of coordination since the transfers are mostly running on the same network connections and thus interfere mainly due to bandwidth limitations. In order to help coordinating e-transfers among correlators and stations, the Geodesy VLBI Group has set up a small set of scripts to show ongoing transfers on a website (List of Active Transfers). In addition to ongoing transfers, the storage capacity at the three IVS correlators in Washington, Haystack and Bonn is listed as well. It is important to point out that the website merely shows active transfers and works on a first come-first served base. An overall coordination of e-transfers concerning their importance and priority is still required and the transfer website should be regarded as a temporary solution.

The affore mentioned website is the front end to display information about current transfers and is located on the MPIfR web server. It is created by a Perl script running as CGI which reads the underlying database. The HTML page is static, there is no mechanism to automatically update the table. Therefore the page needs to be reloaded in order to see the latest status of transfers.

The most important information in the List of Active Transfers is the route on which the data are sent ("Sent from" and "Sent to") as well as the applied transfer rate and the port on which the transfer is running.

Future e-transfer in view of VGOS (VLBI2010 Global Observing System)

In the near furture we will have a data transfer from the network stations to the Correlator centers with combination of high-speed networks and high-data-rate disk systems. The rapid advance of both magnetic-disk technology and global high-speed network technology will be utilized in the next-generation system. A more complex discussion of this subject and the impacts on observing is included in the VLBI2010 agenda. Special sub-groups report on data-acquisition and transport [WG3-4] and on observing strategies [WG3-1]. A few important points are summarized here:

An array of antennas directly connected to the correlator via high-speed network provides the possibility for real-time or near-real-time processing to produce geodetic results in a matter of hours, which is particularly important to the rapid turnaround of EOP results.
Modern magnetic-disk-based recording systems will allow economical recording at rates to several Gbps. With ever expanding disk capacities, stations will be able to operate unattended for periods of a day or more at a time, needing attention only for disk-module changes, and thus enabling continuous observations even for sites not only connected via high-speed fiber.
The existing global grid of research and education networks spans all the continents except Antarctica at minumum rates of 10Gbps, and continues to rapidly improve in speed. This global grid may, with suitable agreements, be available for transmitting geodetic VLBI data.
High-speed network connections to antenna sites must be implemented for at least a substential subset of the sites. In many cases, sites are within a few km of a potential connection point, but some more remote sites are considerably more distant. Relatively inexpensive free-space optical links should be considered for some sites where fiber installation is otherwise too expensive or difficult.
All data collection and transmission interfaces and formats should adhere to the set of internationally agreed VLBI Standard Interface (VSI) specifications.

Top