Data Server Configuration

This page contains all the information about configuration of a GSICS data server.

THREDDS Tree View on the Web Browser (as of Nov. 2014)

A case of the EUMETSAT THREDDS server on: http://gsics.eumetsat.int/thredds/catalog.html

  • folder GSICS Source Data
    • folder EUMETSAT (organization)
      • folder Meteosat Second Generation Satellites (satellite)
        • folder MSG3 SEVIRI Instrument (instrument)
          • folder W_XX-EUMETSAT-Darmstadt,VIS+IR+IMAGERY,MSG2+SEVIRI_C_EUMG_20141103222510.nc
  • folder GSICS Intermediate Data
    • folder JMA (organization)
      • folder Co-location Data (data type)
        • folder MTSAT2 Imager collocated with MetOpA IASI (file type)
          • folder W_JP-JMA-MSC,SATCAL+COLLOC+GEOLEOIR,MTSAT2+Imager-MetOpA+IASI_C_RJTD_20141030131759_20141030131919.nc
  • folder GSICS Products
    • folder EUMETSAT (organization)
      • folder Near Real Time Corrections (NRTC) (international data subcategory)
        • folder MSG2 SEVIRI calibrated against MetOpA IASI (monitored vs reference instrument)
          • folder Demonstration (product phase)
          • folder Pre-Operational
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-MetOpA+IASI_C_EUMG_20131017000000_preop_01.nc
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-MetOpA+IASI_C_EUMG_20131016000000_preop_01.nc
          • folder Operational
      • folder Re-Analysis Corrections (RAC) (international data subcategory)
        • folder MSG2 SEVIRI calibrated against MetOpA IASI (monitored vs reference instrument)
        • folder MSG3 SEVIRI calibrated against MetOpA IASI
          • folder Demonstration
          • folder Pre-Operational
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR,MSG3+SEVIRI-MetOpA+IASI_C_EUMG_20130110000000_preop_01.nc
          • folder Operational
        • folder MSG3 SEVIRI calibrated against MetOpB IASI (monitored vs reference instrument)

Upcoming Changes in the Near Future (planned in 2015)

Through the discussion at the 2014 GSICS annual meeting, web meetings, and gsics-dev, the GRWG decided to propose new THREDDS configuration to generate "Prime GSICS Correction" on 4 Nov, 2014 ( link to the discussion on the gsics-dev). The proposed configuration is shown as follows (new configurations are highlighted in red). Initial proposal was updated at the GDWG breakout session in the 2015 annual meeting: to add "-PRIME" to the GSICS Prime Correction file naming (right after monitored instrument name).

This change will be implemented before the GEO-LEO-IR products becom the first "Operational" GSICS products.

  • folder GSICS Source Data
    • folder EUMETSAT (organization)
      • folder Meteosat Second Generation Satellites (satellite)
        • folder MSG3 SEVIRI Instrument (instrument)
          • folder W_XX-EUMETSAT-Darmstadt,VIS+IR+IMAGERY,MSG2+SEVIRI_C_EUMG_20141103222510.nc
  • folder GSICS Intermediate Data
    • folder JMA (organization)
      • folder Co-location Data (data type)
        • folder MTSAT2 Imager collocated with MetOpA IASI (file type)
          • folder W_JP-JMA-MSC,SATCAL+COLLOC+GEOLEOIR,MTSAT2+Imager-MetOpA+IASI_C_RJTD_20141030131759_20141030131919.nc
  • folder GSICS Products
    • folder EUMETSAT (organization)
      • folder Near Real Time Corrections (NRTC) (international data subcategory)
        • folderMSG2 SEVIRI Prime GSICS Correction
          • folder Demonstration (product phase)
          • folder Pre-Operational
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-PRIME_C_EUMG_20131017000000_preop_01.nc
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-PRIME_C_EUMG_20131016000000_preop_01.nc
            • folder Operational
        • folder MSG3 SEVIRI Prime GSICS Correction
        • folder Individual References
          • folder MSG2 SEVIRI calibrated against MetOpA IASI (monitored vs reference instrument)
            • folder Demonstration (product phase)
            • folder Pre-Operational
              • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-MetOpA+IASI_C_EUMG_20131017000000_preop_01.nc
              • folder W_XX-EUMETSAT-Darmstadt,SATCAL+NRTC+GEOLEOIR,MSG2+SEVIRI-MetOpA+IASI_C_EUMG_20131016000000_preop_01.nc
            • folder Operational
      • folder Re-Analysis Corrections (RAC) (international data subcategory)
        • folder MSG2 SEVIRI Prime GSICS Correction
        • folder MSG3 SEVIRI Prime GSICS Correction
          • folder Demonstration
          • folder Pre-Operational
            • folder W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR,MSG3+SEVIRI-PRIME_C_EUMG_20130110000000_preop_01.nc
          • folder Operational
        • folder Individual References
          • folder MSG3 SEVIRI calibrated against MetOpB IASI (monitored vs reference instrument)
          • folder MSG2 SEVIRI calibrated against MetOpA IASI
          • folder MSG3 SEVIRI calibrated against MetOpA IASI
            • folder Demonstration
            • folder Pre-Operational
              • folder W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR,MSG3+SEVIRI-MetOpA+IASI_C_EUMG_20130110000000_preop_01.nc
            • folder Operational
          • folder MSG3 SEVIRI calibrated against MetOpB IASI

New Directory Structure is now Live (2016-06-23)

The directory structure update where PRIME and Individual References is now configured on the EUMETSAT GSICS Collaboration Server.

This directory structure will also be prepared for the CMA and NOAA-Nesdis Server (see attachments) ready for their administrators to review and install.

In the following are hyperlinks to the presentations and emails regarding this change.

Tools for mirroring GSICS Products among Collaboration Servers

To mirror GSICS products among Collaboration Servers, copying of the products between the servers was proposed by using a Wget script, and demonstrated to the group at 2017 GRWG/GDWG Annual Meeting ( meeting minutes).

The following are the scripts developed by GPRCs which operate Collaboration Server in March 2017. After the reviewing within GDWG, these scripts are expected to be installed and used at each Collaboration Servers.


In the following is the server configuration for administrators. Users of the GSICS products do not have to pay any attention to that.

THREDDS Server Templates and a Directory Creation Script (Obsolete - from 2016/06/23)

As discussed in the GSICS joint working group meeting in Daejeon in March 2011, the action GDWG05_45 (A THREDDS template to configured remote catalogue access and metadata pages is to be created and shared amongst GPRCs to configure their THREDDS applications.) is addressed by the contents of this tar file. The tar file contains the following:

README.txt
Explanation of the contents of the tar file.

catalogue.xml
This is the root catalogue template for the GSICS THREDDS server. Server administrators should use this version as a template for their servers. N.B. Remote catalogues are used for source and intermediate data.

makeGSICSdirectories.ksh
Utility script to create the agreed directory structure under the $THREDD_HOME/public directory.

threddsConfig.xml
The THREDDS configuration file where remote catalogue reference is set to true. i.e. <CatalogServices><allowRemote>true</allowRemote></CatalogServices>.

Common Directory Structure (Obsolete - from 2016-06-23)

All directory names must be lowercase.

GSICS Data server

  • folder upload (file upload directory)
  • folder THREDDS Base Directory
    • folder source
      • folder « organization » (data producer; example: "nesdis", "jma")
        • folder « instrument » (example: "msg2-seviri", "mtsat1r-jami")
    • folder intermediate
      • folder « organization » (data producer; example: "nesdis", "jma")
        • folder « international data subcategory » (example: "colloc")
          • folder « monitored instrument » (example: "goes11-imager")
            • folder « reference instrument » (example: "aqua-airs")
              (Further subdirectory organization subject to specific data.)
    • folder product
      • folder « organization » (data producer; example: "nesdis", "eumetsat")
        • folder « international data subcategory » (example: "rac")
          • folder « monitored instrument » (example: "goes11-imager")
            • folder « reference instrument » (example: "aqua-airs", combo reference: "aqua-airs-metopa-iasi")
              • folder demo
              • folder preop
Topic attachments
I Attachment Action Size Date WhoSorted ascending Comment
CMA_FileDlTool.zipzip CMA_FileDlTool.zip manage 1 K 27 Jan 2017 - 09:43 MasayaTakahashi Scripts to download files on Collaboration Servers developed by CMA
downlodEUMETSAT.shsh downlodEUMETSAT.sh manage 1 K 23 Mar 2017 - 22:00 MasayaTakahashi  
gsics-v1.iniini gsics-v1.ini manage 1 K 23 Mar 2017 - 22:01 MasayaTakahashi  
CMA_THREDDS_V28.0.zipzip CMA_THREDDS_V28.0.zip manage 235 K 24 Jun 2016 - 11:44 PeterMiu CMA GSICS Collaboration Server Products Configuration Files
GSICSserverTemplate.tartar GSICSserverTemplate.tar manage 20 K 10 May 2011 - 14:43 PeterMiu Templates and Utility for the GSICS Data and Product Servers.
NESDIS_THREDDS_V28.0.zipzip NESDIS_THREDDS_V28.0.zip manage 234 K 24 Jun 2016 - 11:44 PeterMiu NOAA NESDIS GSICS Collaboration Server Products Configuration Files
Topic revision: r31 - 23 Mar 2017, MasayaTakahashi
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSICS Wiki? Send feedback