The complex and large data set generated by the Gaia focal plane, consisting of astrometric, photometric and spectroscopic data, must be compressed without losses before being stored to the PDHU for its later transmission down to the Ground Station. This task will be carried out by the Video Processing Units (VPU).
Although preliminary studies indicated very promising results in this critical aspect of the mission, no detailed and realistic enough analysis was carried out yet for the data compression of Gaia. The Institute for Space Studies of Catalonia (IEEC), with the Spanish software company GTD as prime contractor, made a deep study of the problem at the request of ESA.
The Gaia Optimum Compression Algorithm (GOCA) TRP Study was successfully finished in January this year. Left panels illustrate the ratios achieved for SP1 packets, containing SM, AF and BP/RP data. The required compression ratio for these packets was 2.6 for faint stars, which is fulfilled in the most typical scenarios. The top figure clearly illustrates the dependence of the ratios on the star magnitude. A small group of "problematic packets" is circled in white, corresponding to truncated packets and to packets without BP/RP, most of which are generated when observing extremely dense areas.
Right panels illustrate the same results for SP2 (RVS) packets, where a required compression ratio of 2.5 is fulfilled as well. The dependence on the star magnitude is not so important in this case. The reason is the high amount of relative noise in the RVS measurements, which fixes a minimum entropy and hence a maximum achievable ratio. A clever pre-processing stage, including an adequate filtering, enables to take advantage of any redundancy in the data.
GOCA was successfully integrated with the rest of video algorithms and tested on a VPU prototype board, at EADS/Astrium premises in Tolouse, where less than 10% of the CPU was used in the worst of the cases (1400 SP1 plus 60 SP2 per second and per VPU). A completely lossless operation was also verified.
Some of the concepts developed during this study have been included in the final data compression algorithms developed by EADS/Astrium.
Images courtesy of the GOCA-IEEC team (Jordi Portell, Alberto G. Villafranca and Enrique García-Berro).
[A larger version of each of the four figures is available: top left image; top right image; lower left image; lower right image]