CEDA Repository
Not a member yet
1250 research outputs found
Sort by
FIDUCEO project: (Microwave): Report on the MW FCDR: Uncertainty
No full textThis document is one of the five documents that make up the D2-2 report on “traceability chains for FCDRs”. Since the original project proposal our thoughts have refined and while this document describes the “sequence of measurement standards and calibrations that is used to relate a measurement result to a reference” (the VIM definition of a traceability chain), it is not presenting this in the form of a chain. This document provides an overview of the uncertainty analysis for the analysed sensors along with the methods to establish metrological traceability for the developed FCDRs. This document is specifically about the MicrowaveFCDR (MHS, SSM-T2, AMSU-B). The document D2-2a provides an overview of the purposes of these documents and explains the basis of the effects tables
FIDUCEO project: Product user guide – Upper Tropsheric CDRs from microwave sounders
No full textThis document describes the Upper Tropospheric Humidity (UTH) CDR(version 1.2) data files uploaded to CEMS in February 2019. The released data record is based on the Microwave “easy” FCDR version 4.1 (see Microwave FCDR PUG). The instantaneous observations from the FCDR are used to derive a spatio-temporal averaged data record, which contains monthly mean UTH and brightness temperature mapped to a regular latitude/longitude grid covering the tropical region with a spatial resolution of 1° x1°. It covers all mission years of SSMT2 on F11, F12, F14, F15, AMSU-B on NOAA15-17 and MHS missions (NOAA18, NOAA19, MetopA, Metop-B). This product user guide gives: 1. An overview of the specifications of the data record; 2. A description of the implementation of the retrieval processing chain; 3. Information on limitations of this current version of the data record; 4. Technical details on the format and on how to access the dat
EUSTACE Product User Guide and quick start guide, draft version 0.8
No full textThis Product User Guide (PUG) is meant as a practical guide for the data products of the EUSTACE (EU Surface Temperature for All Corners of Earth) project and to facilitate (potential) users in their exploitation of the EUSTACE datasets
FIDUCEO project: Report on MVIRI Aerosol demonstration dataset
No full textThis document forms the deliverable D5.8 to report on the climate data record (CDR) of aerosol optical thickness (AOT) as retrieved from the MVIRI fundamental climate data record (FCDR) [RD 1, RD 2, RD 3] using the Combined Inversion of Surface and AeRosol (CISAR) Algorithm [RD 4]. The primary objective of this data record is to assess and demonstrate how the recalibrated and uncertainty-quantified MVIRI FCDR can support improved retrieval of geophysical parameters. Of particular interest is the impact of in-flight reconstructed and spectrally degrading spectral response functions
NCAS CDAO Sky-Camera Time-Lapse Video for 2009-03-25 17:25 UTC (showing Altocumulus fluctus and Altocumulus undulatus clouds followed by Altocumulus lenticularis)
No full textA time-lapse video showing Altocumulus fluctus and Altocumulus undulatus clouds followed by Altocumulus lenticularis. The short-lived fluctus cloud feature is caused by Kelvin-Helmholtz wave activity, which is generated where the wind speed changes sharply with altitude. The initial telephoto image shows the characteristic breaking-wave pattern. This was taken (at 17:37 UTC) at the same location as the camera used to to generate the video sequence, but shows a narrower field of view. The wave activity is also revealed by the Altocumulus undulatus cloud elements, which form in parallel bands. There is evidence of mountain wave activity throughout the sequence and Altocumulus lenticularis becomes the dominant cloud type towards the end. These clouds remain stationary relative to the landscape rather moving with the wind. Occasional contrails and Cirrus clouds can be seen at a higher level (one of the contrails develops undulatus features). These remain illuminated for longer than the Altocumulus clouds as the sun sets. This video has been created from images taken by the Sky-Camera at the National Centre for Atmospheric Science (NCAS) Capel Dewi Atmospheric Observatory (CDAO) near Aberystwyth, UK - formerly known as the Natural Environment Research Council (NERC) Mesosphere-Stratosphere-Troposphere (MST) Radar Facility. The images are freely available under a UK Open Government Licence from http://tinyurl.com/nerc-mstrf-sky-camera/ . For more details visit http://mst.nerc.ac.uk
CEDA Annual Report 2018 - 2019
No full textThis annual report presents key statistics for the year past (2018 - 2019) as well as a series of snapshots of activity, expressed as short highlights and short reports
FIDUCEO project: Report on AVHRR aerosol demonstration dataset
No full textThis document forms the deliverable D5.8 to report on the climate data record (CDR) over Europe and North Africa of aerosol optical depth (AOD) as retrieved from the AVHRR fundamental climate data record (FCDR) using a simple dark field Algorithm [RD 4]. The primary object of this data record is to assess and demonstrate how the sophisticated uncertainty information contained in the AVHRR easy FCDR dataset be propagated through a retrieval algorithm
Long-term Archive Challenges: Enhancing Data Discovery via Multilevel Metadata Aggregations At Scale
No full textData archives require accurate, content-rich data catalogues that are fit-for-purpose to support meaningful data discovery. However, sourcing suitable high-quality metadata to populate catalogues at scale can be problematic when manual workflows are no longer able to cope. One solution is automated metadata harvesting directly from archive contents, drawing on technical solutions to Big Data challenges faced by rapidly evolving, petabyte-scale, heterogeneous archives. Yet other issues quickly arise, including: changes in, or lack of, metadata standards over time; missing or incorrect metadata; diversity of, and lack of interoperability between, formats and metadata conventions; and, changes in data availability over time. These are further compounded when dealing with historical archives and legacy systems stretching back decades before comprehensive end-to-end metadata harvesting workflows were envisioned.
The Centre for Environmental Data Analysis (CEDA) Archive has long-term archiving responsibility for the UK atmospheric, climate change and Earth observation communities. With highly heterogeneous deposits of both historic and rapidly growing fresh data holdings, spanning over 5,500 datasets, 200 million files and over 5 Pb of online storage, the CEDA Archive is no newcomer to the 4Vs of the Big Data challenge: Volume, Velocity, Variety and Veracity. Recently CEDA’s combined use of parallel archive and processing systems with noSQL Elasticsearch indexes has addressed many of these challenges. This has permitted file-level metadata such as parameter, spatial and temporal information to be indexed and then aggregated to populate most dataset records in CEDA’s ISO-driven data catalogue. However, there remain significant shortcomings in the quality and coverage of the metadata harvested via this pipeline (less than 50% of files return parameter information for example) that require additional, complementary approaches.
The key remaining challenges are how to address the following issues: where file-based metadata is incomplete or missing (e.g. unscannable file-formats; incomplete metadata); coping with incorrect file-based metadata (e.g. incorrect geo-temporal information; mapping between coordinate systems); dealing with removed data files; and, coverage for offline/external content. To address this a complementary YAML-driven ’Manual Metadata Store’ has been set up operating at the dataset records level in the CEDA data catalogue. This enables a manually maintainable, traceable, versioned alternate metadata source and splicing rules to determine how the information should be used in conjunction with automatically harvested metadata such as CEDA’s file-level index. Splicing options include: partial or complete replacement; appending of information; and, setting default information. Whilst this has allowed CEDA to begin addressing issues with automated metadata harvesting, it has also raised additional questions, especially as CEDA seeks to allow file-level faceted searches in the future, where file-derived metadata is known to be incorrect or missing. At what level should amendments to metadata be applied? The dataset record, file-level index or the original metadata themselves? And how to manage the conflicting pressures of limited resources, preserving original data integrity and the desire to deliver quality information at all levels and at scale. Essentially, the next issue to face can be summed up with the questions: ‘What is Truth? And at what level to convey it