GPPA Checklist for EUMETSAT DCC Product for Meteosat-9/SEVIRI/VIS0.6

Product
Phase

Step

GPAF

Task

Who

Due/
Done?

Submission

1 §I Product Provider Information Product provider

t1=2015-03-14

GPAF

ATBD

Sample Product files Available here

Daily File upload location

Imact and Validation Report

Example of validation were presented by Arata (4m ) and Fangfang in GSICS Newsletter ( Publication by Yu and Wu 2015, https://doi.org/10.1016/j.rse.2015.04.003)

§II.1 Brief Product Description
§II.2 Product Scope within GSICS
§II.3 Product function and performance specifications
§III.1.A.1 Adherence to GSICS file naming standard
§III.1.A.2 Adherence to GSICS file format convention
§III.1.B Preliminary ATBD
2 Submit GPAF and preliminary ATBD
to GCC Deputy for review to GPAT
Product provider t2=2015-09-15
3

Determine if GPAF filled out correctly.

Decide if product ABTD is adequate. Decide if product within GSICS scope.
GPAT

t2+6wk
Masaya and Manik Review. Includes Author response to the review
4 GPAT feedback given to Product provider GCC Director t3+1wk
5 Upload sample file to GSICS data server Product provider
GDWG Chair
t5=2016-03-19
NRTC
RAC
6 Determine if sample file follows GSICS conventions GDWG Chair t5+2wk
7 Remediate any GPAF issues according to GPAT feedback Product provider,
GCC Director
t4+20d
8 Once all issues resolved, product enters Demonstration Phase GCC Director t2+90d

Demonstration

9 Notify Exec Panel about Product entering Demonstration Phase GCC Director t8+2wk
10 Arrange routine upload of Product to a GSICS data server
Agree on file retention policy

GDWG Chair,
GRWG Chair,
Product provider
t8+1m
11 Notify GSICS users and invite feedback from them GCC Director t10
12 §III.1.B Complete full ATBD - Submit to GCC Director Product provider t8+3m
§III.1.C Product traceability to standards - Submit to GCC Director
13 Review submitted documents (ATBD and product traceability to standards)

GPAT
Users
t12+1m
14 Remediate any document issues according to the GPAT and user feedback Product provider t13+1m
15 §III.2.A Radiative transfer models Product provider t8+6m
§III.2.B Cal/val supporting measurements
16 Examine submitted documents
(radiative transfer models and cal/val supporting measurements)
GPAT t15+1m
17 Collect and disseminate user feedback regarding product's data usability and format GCC Director t8+6m
18 Remediate any document or data issues according to GPAT and user feedback

Product provider
GCC Director
t17+1m
19 Make consensus decision whether to continue Product acceptance process GPAT t18+2w
20 Notify Executive Panel of status of the Product's acceptance process GCC Director t19+2w - GCC - Director Notifcation to EP See here informing produc enters Demo Phase
21 §III.3.A Product Quality Documents (Uncertainty and Traceability) Product provider t8+10m
22 Examine the submitted Product Quality Document GPAT t21+1m
23 Remediate any document issues according to GPAT feedback Product provider t22+1m
24 Once all issues are resolved, Product enters Pre-operational Phase GCC Director t8+1y

Pre-operational

25 Send notification and GPAT Product recommendations to Exec Panel GCC Director t24+2w
26 Exec Panel review GPAT recommendations and send feedback to GCC Director Exec Panel t25+6w
27 GCC Director notifies Product provider about Executive Panel feedback GCC Director t26+2w
28 §III.2.C Analysis software documentation Product provider t24+3m
§III.2.D Product version control plan
§III.3.B Operations and distribution plan
§III.3.C Users' Guide
29 Examine submitted documents + Provide feedback
(product version control plan, operations and distribution plan, and user's guide)
GPAT t28+1m
30 Remediate any documentation and overall product issues following Exec Panel and GPAT feedback Product provider t29+1m
31 Review remediation material and decides if requirements are now satisfied.
Send final recommendation to the GCC Director.
GPAT t30+3w

Operational

32 GCC Director notifies Exec Panel that product has satisfied all requirements to enter Operational Phase GCC Director t31+1w

Review of GSICS-EUMETSAT SEVIRI- MODIS DCC Product

On 15 Sept 2015 EUMETSAT submitted to GCC SEVIRI- MODIS DCC Product. After an initial review Manik Bali @ GCC forwarded the product to GPAT members for a review.

As part of the submission the following documents were submitted to the GCC

For the Demo we decided to have 2( Masaya Takahashi and Manik Bali see here) instead of usual 3 by taking advantage of the fact that the product has been extensively discussed in the community and banks on published work,.

The following part of the GPPA is being followed at this time
  1. Fill out Sections I, II, III.1.A.1, and III.1.A.2 of the GSICS Product Acceptance Form (GPAF). Also, fill out Section III.1.B (ATBD) of the GPAF, but during the Submission Phase the ATBD needs only to be a preliminary version. The ATBD could be a journal article, technical memorandum or other documentation of the method used to make the product.
    • Who: Product provider
    • Done
  2. Submit the GPAF and preliminary ATBD for review to the GSICS Product Acceptance Team (GPAT) via the GSICS Coordination Center (GCC) Deputy.
    • Who: Product provider
    • Done
  3. Determine if the GPAF is filled out correctly and decide if the product theoretical basis is adequate and the product scope is within the GSICS domain.
    • Who: GPAT
    • Due: 6 weeks after the GPAF submission
    • Done
  4. GPAT feedback given to the Product provider.
    • Who: GCC Director
    • Due: 7 weeks from GPAF submission
    • Done
  5. Upload a sample file to a GSICS data server if GPAT feedback is affirmative.
    • Who: Product provider, GDWG Chairman
    • Due: One week after GPAT feedback regarding GPAF form submission
    • Daily File upload location
7. Remediate any GPAF issues according to the GPAT feedback.
  • Who: Product provider, GCC Director
  • Due: Within 20 days after sending GPAT feedback to the Product provider
  • Done
Author Response to initial submision ( see here )

1. ATBD ( After First review )

2. Product Available here

8. Once all issues are resolved, the product enters the Demonstration Phase .
  • Who: GCC Director in consultation with the GPAT
  • Due: Within 90 days of the GPAF submission
  • Pending
9. Upload a sample file to a GSICS data server if GPAT feedback is affirmative.
    • Who: Product provider, GDWG Chairman
    • Due: One week after GPAT feedback regarding GPAF form submission
    • Done
10. Determine if the sample file follows the GSICS netCDF and file naming conventions.
  • Who: GDWG Chairman
  • Due: 2 weeks after successful product sample file upload
11. Remediate any GPAF issues according to the GPAT feedback.
  • Who: Product provider, GCC Director
  • Due: Within 20 days after sending GPAT feedback to the Product provider
  • Done-Review Recommend Acceptance-Report stated here and Remediation
  • 12. Once all issues are resolved, the product enters theDemonstration Phase .
Who: GCC Director in consultation with the GPAT
    • Due: Within 90 days of the GPAF submission
    • Done Product enters Demo Phase: Notification to EP issued.

GPPA Exception

 

Typically Impact report and Validation report are required at the Demo Phase. The aim of these reports is to ensure that users of this product need this product within the producing agency and that the agency is backing the product.However this product needs to be seen in the light of its acedemic value. In the DCC special session in Tsukuba (Session 4j-4v here) the DCC method was discussed extensively by all agencies. Participants stated that developing a combined product towards intercalibration of visible channels in which DCC would be one of the methods is the way forward. In a combined approach DCC method main use would be to validate the cross calibration of other methods. Hence at the Demo state use of DCC in improving retrieval of downstream variables such as cloud height etc is not needed.

Example of validation were presented by Arata (4m ) and Fangfang in Publication by Yu and Wu 2015, https://doi.org/10.1016/j.rse.2015.04.003

Decision: Considering the acceptance of the product among agencies ( as seen in the 2016 Annual Meeting Session 4j-4v here ) and the commitment to provide Impact and Validation report at the next level of maturity(see below) , GCC Director Larry Flynn is using his discretion to waive this requirement at the Demo Phase.

Tim and Sebastien have indicated a plan at EUMETSAT to test the DCC product.

GDWG Chair Consent to Accept Product ( after reviewer comments were provided)

m_takahashi@met.kishou.go.jp

12:49 AM (10 hours ago)
to me, Larry, Peter, Sebastien, Tim, Dohy

Hello Manik,

Sorry for the delay in my response - I was on leave last week. I have
reviewed their GPAF, ATBD and product formats and the product is ready for
entering Demo-Phase.

I would like to congratulate EUMETSAT for your great progress on - the
first GSICS VNIR product!

Best regards,
Masaya

ohy Kim

12:42 AM (10 hours ago)
to me, Tim, Masaya, Peter, Sebastien, Larry

Dear Manik

It is great job of EUMETSAT, and of course I agee with its process in Demo phase if you need my consent.

Best regards,

Dohy

From: Tim Hewison
Sent: Friday, January 06, 2017 9:08 AM
To: Manik Bali - NOAA Affiliate
Cc: Larry Flynn; Sebastien Wagner
Subject: RE: Clarifications for DCC GSICS Corrections to enter Demo Phase

Hi Manik,

Thanks for following this up.

Just an update on the Impact Report. We discussed this with our Atmospheric and Imagery Applications group, who agreed to test impact of GSICS VNIR products once the warm channels are all inter-calibrated – because they use multiple channels in the Optimal Cloud Analysis algorithm. This means we need to have a prototype merged product for DCC+Lunar, which we expect to have by the end of this year. This means the report will not be available until at least mid 2018. I hope this is acceptable, as it is the best we can do (at least for any meaningful impact assessment).

Cheers,

Tim

Manik Bali - NOAA Affiliate <manik.bali@noaa.gov>

12/7/15

to Tim, Kenneth, Lawrence, Sebastien, Peter

Dear Tim, Ken,

Since you wish to have the Prime Ref officially in Demo before Christmas we This morning I and Larry reviewed the GPPA for the Prime Ref Product as we need to have a workable plan. So far we have not received any GPAT review of this product which puts us in a slightly tight situation as far as time is concerned.

Please find below (scroll down) the status of the steps in the GPPA for your product.You can see that GPAT review is still required.

Proposal now is that for the Demo we can go for 1 GPAT ( external review ) instead of usual 3. +

Considering that the agencies themselves are the first users of the products they generate, we can then go for a 'Internal EUMETSAT verification and validation document ( which I guess you might already be having)'. We can discuss on what you have and what parts of the internal review are yet to come as the product attains maturity.

Since it is a new product many of the GPAT might not be familiar with its scope /usage and could be the reason for not getting much response and I guess we would end up with many of the GPAT not coming forth with the review so we would go with external reviewers in due course.

External reviews also help advertise the product beyond the walls of the producer agencies and Larry has suggested to identify some of friends in AIIRS and IASI teams.

You can help us in that.

Let us know what you think of this.

Rgds

Manik

Reply from Tim Hewison

We can proceed on this basis for many product submissions, including our DCC products. These are based on an algorithm that has been developed communally within GSICS. As such it has already been reviewed by other GSICS members. However, we don\x92t have any written evidence of that review process, except minutes of the meetings.

However, we don\x92t have any such documents for the GSICS Prime Corrections. In this case, the algorithm was developed mostly by me. It is considered a higher risk product development, so the candidate products are generated from my prototype code running in a non-operational environment. Here we rely on peer review through GSICS to validate the approach. It is likely that other potential demonstration products will be in the same boat.

Topic attachments
I Attachment Action Size Date Who Comment
ATBD_for_EUMETSAT_GSICS_Inter-Calibration_with_MODIS_using_Deep_Convective_Clouds_1.docxdocx ATBD_for_EUMETSAT_GSICS_Inter-Calibration_with_MODIS_using_Deep_Convective_Clouds_1.docx manage 535 K 18 Feb 2016 - 20:22 ManikBali  
DCC Review Demo.docxdocx DCC Review Demo.docx manage 15 K 08 Dec 2016 - 17:53 ManikBali Review by Manik and Masaya. Includes author response
DCC Review and Remediation.docxdocx DCC Review and Remediation.docx manage 19 K 03 May 2017 - 15:49 ManikBali DCC Review and Remediation
EUMETSAT_SEVIRI_MODIS_DCC_acceptance_decision.docxdocx EUMETSAT_SEVIRI_MODIS_DCC_acceptance_decision.docx manage 25 K 08 May 2017 - 15:02 ManikBali GCC Director E-mail to EP
GSICS_GCC_GPAF_V01_2_VNIR_EUM_2.docdoc GSICS_GCC_GPAF_V01_2_VNIR_EUM_2.doc manage 317 K 18 Feb 2016 - 20:23 ManikBali  
W_XX-EUMETSAT-DarmstadtSATCALRACGEOLEOVNIRMSG2SEVIRI-AQUAMODIS_C_EUMG_20121231000000_demo_blend_v1_1.ncnc W_XX-EUMETSAT-DarmstadtSATCALRACGEOLEOVNIRMSG2SEVIRI-AQUAMODIS_C_EUMG_20121231000000_demo_blend_v1_1.nc manage 102 K 18 Feb 2016 - 20:23 ManikBali  
Topic revision: r12 - 08 May 2017, ManikBali
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSICS Wiki? Send feedback