GIRO executable – first release
The purpose of sharing this version of the GIRO executable which provides little functionality is to verify that the handling of NetCDF files by the executable works properly on different Linux machines (both 32 bits and 64 bits).
In the presentation [GIRO_VITICCHIE] one finds the proposal to share a first version of the GIRO executable to perform a first test of:
• reading of the input NetCDF files (observation + spectral response function),
• computation of the observables, and
• writing of a “dummy” lunar calibration file in which only the observables will be stored while rest of the information is left at _FillValue (for the definition of observables please refer to [HLD_GIRO_IO]).
These are the main functionalities of the executable files available in this archive.
When unpacking the GIRO_firstrelease.tar.gz archive the user will find locally the following files and folders:
|
Name |
Type |
Description |
|
GIRO_firstrelease32.exe |
executable file |
GIRO executable to be used on 32 bits Linux machines |
|
GIRO_firstrelease64.exe |
executable file |
GIRO executable to be used on 64 bits Linux machines |
|
./data/ |
folder |
Folder containing three NetCDF lunar observation files from both MTSAT2 and MSG3 |
|
./SRFs/ |
folder |
Folder containing the NetCDF spectral response function files of MTSAT2 and MSG3 |
|
./results/ |
folder |
Empty |
|
readme.pdf |
.pdf document |
This document |
|
NetCDF.licence, zlib.licence, and hdf5.licence |
text files |
Copyrights of the static libraries of the executable |
Such folder structuring must be respected when using either executable file. In detail, in the folder in which the executable will be lauched the three folders data/, SRFs/ and results/ must be available. During a run, the GIRO executable looks for the spectral response function file of a satellite in the folder SRFs/ and then in the folder data/ for lunar observation files to analyse. The naming of the files stored in both folders must respect the format convention described in [HLD_GIRO_IO]. Sub-folders are allowed in both the folders data/ and SRFs/ with the exception made for subfolders whose name contains the code-name of a satellite (see next section for the description of the code-name).
A GIRO calibration run can be launched with the command line (for a 32 bits machine):
$ ./GIRO_firstrelease32.exe SAT_NAME
where SAT_NAME is the code-name of the satellite that is used in the naming of the spectral response function NetCDF file of the satellite and the lunar observation NetCDF files from the same satellite. For example:
SAT_NAME = MSG3 (argument of the command line)
W_XX-EUMETSAT-Darmstadt,VIS+IR+SRF,MSG3+SEVIRI_C_EUMG.nc (MSG3 SRF file in SRFs/)
W_XX-EUMETSAT-Darmstadt,VISNIR+SUBSET+MOON,MSG3+SEVIRI_C_EUMG_20140318140112_01.nc (lunar observation file in data/)
The GIRO executable can be launched for a single satellite. Each GIRO run works on all the channels of a satellite that can be analysed using the GIRO executable. The selection of the analysable channels of the satellite is performed by referring to their spectral extension/coverage. In detail, in order to be analysed, a channel must be completely included in the spectral range 0.3 – 2.5 µm. As a result of this process, a list of analysable channels is defined and the executable looks into each lunar observation file to read exclusively the data related to the analysable channels. This means that the user can store the data from all the channels of a satellite in a lunar observation file; the GIRO executable will select the analysable part of such file. Please note that this selection is done by comparing the names of the channels as found in the spectral response function file and the same names as found in the lunar observation file. From this stems that the naming and codes indicating the satellite, the instrument, and the instrument channels must be consistent in the spectral response function file and in the lunar observation files.
In order to get a help on how to run the GIRO executable the user can use the command line:
$ ./GIRO_firstrelease32 –-help
Also in the case of a wrong command line the help is printed on screen.
The files that have been included in the folders data/ and SRFs/ will allow the user to run the GIRO executable in the case he/she did not prepare any lunar observation NetCDF file. In detail, two different runs can be performed:
1. Run on three lunar observation files from MTSAT2 (i.e., a satellite whose instrument has one analysable channel).
2. Run on three lunar observation files from MSG3 (i.e., a satellite whose instrument has four analysable channels).
These files contain all the COMPULSORY, ADDITIONAL, and IMAGETTES data (see [GIRO_VITICCHIE], [GIRO_TAKAHASHI], and [HLD_GIRO_IO] for the definition of these data).
When launching:
$ ./GIRO_firstrelease32.exe MTSAT2
the executable will print the number of analysable channels for MTSAT2 (i.e., one) and the number of lunar observation files that have been found to be analysed (i.e., three).
The following messages will specify the paths to the files that are produced by the GIRO executable. Currently, two kinds of files are produced by the GIRO executable:
1. Sanity check files: sanity_check_’SAT_NAME’+’Instrument’+’Channel’.info
these files (one file per analysed channel) collect the results of the comparison between the observables as provided by the user and the observables are derived by the imagette processing of the GIRO executable (see sections 3.3 and 5.2 in [HLD_GIRO_IO]) for each analysed file. For example, in the file sanity_check_MTSAT2+Imager+VIS.info on finds:
W_JP-JMA-MSC,VISNIR+SUBSET+MOON,MTSAT2+Imager_C_RJTD_20100701062451_01.nc
Quantity User Imgt Processing
Observed Irradiance 0.000702360438230500 0.000702360438649869
Observed DC 15887136 15887136
DC Offset 50.515755395683456186 50.515755395683456186
Total number of pixels 82395 82395
W_JP-JMA-MSC,VISNIR+SUBSET+MOON,MTSAT2+Imager_C_RJTD_20110704163217_01.nc
Quantity User Imgt Processing
Observed Irradiance 0.000026484273576469 0.000026484273701312
Observed DC 924069 924069
DC Offset 48.963885088919290922 48.963885088919290922
Total number of pixels 9607 9607
W_JP-JMA-MSC,VISNIR+SUBSET+MOON,MTSAT2+Imager_C_RJTD_20130725035138_01.nc
Quantity User Imgt Processing
Observed Irradiance 0.001303947964100031 0.001303947964754193
Observed DC 27668717 27668717
DC Offset 50.745347698334967390 50.745347698334967390
Total number of pixels 116446 116446
In this example the observed irradiance as provided by the user and the observed irradiance as derived by the GIRO executable via the imagette processing are slightly different even though the procedure employed in the computation of these quantities is exactly the same. The difference stems from the fact that when storing the imagettes in the lunar observation NetCDF file the user has performed a rounding of the radiance values in the imagette. This rounding could not be applied to the digital count (DC) imagette; for this reason the Observed DC, the DC Offset and the Total number of pixels are exactly the same in the two columns (these are derived from the DC imagette). This example shows the usefulness of the imagette processing combined to the sanity checks to quantify the impact, for example, of a method of storing the data on the final results of the processing.
2. A single lunar calibration NetCDF file:
W_’wmo_LocationIndicator’,SATCAL+ROLOVISNIR,’SAT_NAME’+’Instrument’_C_’wmo_CCCC_Code’_’Observation_time_of_the_first_imagette_YYYYMMDDHHMMSS’_’major_version_number’.nc (for the content of this file refer to [HLD_GIRO_IO]).
When launching:
$ ./GIRO_firstrelease32.exe MSG3
the executable will print a total number of four analysable channels and of three lunar observation files to analyse. Also in this case the paths to the output files that are produced by the procedure are printed; in detail, five output files will be produced when analysing MSG3 data:
1. sanity_check_MSG3+SEVIRI+VIS006.info
2. sanity_check_MSG3+SEVIRI+VIS008.info
3. sanity_check_MSG3+SEVIRI+HRVIS.info
4. sanity_check_MSG3+SEVIRI+NIR016.info
5. W_XX-EUMETSAT-Darmstadt,SATCAL+ROLOVISNIR,MSG3+SEVIRI_C_EUMG_20130101145644_00.nc
i.e., four sanity check files and one lunar calibration file. Besides this, also three error messages will be printed during the processing:
COMPULSORY INFORMATION Irr_OBS MISSING IN FILE ‘file name’
These messages follow the creation of the sanity check file of the HRVIS channel (i.e., when the procedure examines the content of the HRVIS channel in the lunar observation files), and point out that no irradiance value has been provided by the user. This is a normal output that a user must expect when analysing data from MSG satellites in which not in all lunar observations the Moon is available in all the VISNIR channels simultaneously. This stems from the fact that the HRVIS (high resolution) channel covers only a part of the field of view covered by the VIS06, VIS08, and NIR16 (low resolution) channels. By checking the content of the sanity check file produced for the HRVIS channels the user will find the _FillValue adopted for the lunar calibration NetCDF files: -999.
In the case in which the user had prepared his/her own lunar observation files following the format specifications in [HLD_GIRO_IO], in order to run the GIRO_firstrelease on such data he/she will need to prepare the SRF NetCDF file of the satellite with which the data were acquired. To produce the SRF file please contact us and we will made the file available in the GSICS wiki page from which the GIRO_firstrelease.tar.gz has been downloaded.
Verifying that the GIRO executables run properly on your local machine is not the only goal of the exercise described in the previous section! We strongly recommend that the users spend some time in checking the content of both input and output NetCDF files. In detail, the user should compare the content of both the lunar observation files and the lunar calibration files against the content of the tables reported in [HLD_GIRO_IO]. It is worth pointing out that some additional/complementary information can be found in these NetCDF files (e.g., variable attributes like valid_min, valid_max); these were not described in the [HLD_GIRO_IO] for the sake of shortness. In spite of this, when producing his/her own dataset, the user should try to include these additional information in the NetCDF lunar observation files.
If the user is not familiar with the handling of NetCDF files this link will be useful: http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-tutorial.html
List of GUIs to read NetCDF files: http://nsidc.org/data/netcdf/tools.html
|
OS |
Linux 32 bits: gcc 4.6.2 Linux 64 bits: gcc 4.3.2 |
|
Required libraries/shared objects |
libm.so.6, libdl.so.2, and libc.so.6 |
|
Release |
First Release |
25th July 2014 |
|
|
Compilation Environment |
Linux 32 bits: gcc 4.6.2 Linux 64 bits: gcc 4.3.2 Zlib 1.2.8 HDF 5-1.8.13 NetCDF-4.3.2 |
|
Tested on |
Ubuntu 14.04 LTS 32 bits, openSUSE 11.1 64 bits, and openSUSE 12.1 32 bits |
|
|
Licences |
Please refer to the .licence files for the Copyrights of the static libraries used by the executable. |
|
Bartolomeo Viticchie |
bartolomeo.viticchie-at-eumetsat.int |
|
Masaya Takahashi |
masaya.takahashi-at-eumetsat.int |
|
Sebastien Wagner |
sebastien.wagner-at-eumetsat.int |
|
[HLD_GIRO_IO] |
https://gsics.nesdis.noaa.gov/pub/Development/20140624/GSICS_ROLO_HighLevDescript_IODefinition_wiki.pdf (please ignore the warning message) |
|
[GIRO_VITICCHIE] |
https://gsics.nesdis.noaa.gov/pub/Development/20140624/Lunar_Calibration_Toolbox_Implementation.pptx (please ignore the warning message) |
|
[GIRO_TAKAHASHI] |
https://gsics.nesdis.noaa.gov/pub/Development/20140624/20140624_Takahashi_LunarCal_DataFormt.pptx (please ignore the warning message) |