2.3.2 Conventions
2.3.2.1 ENVISAT: General product structure
The following convention has been defined for
all ENVISAT products:
ASCII Header Conventions
The Main Product Header (MPH) and Specific
Product Header (SPH) of ENVISAT products
follow the following conventions:
- Headers use only ASCII characters.
- They are fixed size structures (i.e.,
the SPH may vary across products, but
within each product it is a fixed number
of bytes long).
- Each entry in the MPH and SPH will
follow a
keyword-value<units>-terminator structure.
KEYWORDS: Keywords are
limited to the set of ASCII characters which
include the capital English alphabet
[A...Z], and the numbers [0...9]. The
only other characters allowed in a keyword
is the underscore (_), and the equal sign
(=). A keyword is a single word, or
several words connected by underscore
characters, followed by an equal sign.
VALUES: All values are
expressed in ASCII format and follow
immediately after the equal sign in the
keyword (i.e., no white space in between
keyword and value). Values may be of two
classes: numeric values, or string-values.
Numeric values are those which would
normally be expressed as an integer or
floating point value. String values are
those values which would normally be
expressed in ASCII characters regardless
of their location in the product. String
values fall into two types: single-character
entries, and multi-character entries.
Multi-character entries must be placed
within double quotes (" ") in the
MPH or SPH. The string within these
quotes may use any of the allowable ASCII
character set. Single value characters do
not require quotation marks, but are limited
to the characters [A..Z], [a..z] and [0..9].
UNITS: The use of units is
required for numeric values unless the value
has no units or the unit type is inherently
obvious. For numeric values which do not
require units and string-values, the value
is followed directly by the terminator
character and no units entry is
included. When units are deemed necessary,
they are placed within angled braces (<
>) directly following the last
character of the value to which the units
apply. No white space is left between the
value and the first angled brace, nor is any
white space left between the first
angled brace and the first character of the
units expression. Finally, no white space is
left between the last character of the
units expression and the closing angled
brace. Within the braces, the units
expression may use any allowable ASCII
characters and be of any length.
TERMINATOR: The terminator
character is placed directly after the
closing angled brace of the units for
entries which have units, or directly
after the last character in the value for
entries which do not have units attached to
them. The terminator value for ENVISAT
products is the ASCII newline character
(character code 10 in See Decimal Value and
corresponding ASCII character). The use of
this terminator allows the MPH/SPH structure
to be displayed in an easily readable format
(one entry per line) on most UNIX text editors.
All ASCII string entries are left-justified
within the quotation marks. Therefore, if
the string is shorter than the number of
characters allocated for it, blank-space
ASCII characters are placed after the last
character in the string, but before the
closing quotation mark.
Note that in the data definitions in this
document, the notation is used to indicate
the inclusion of an ASCII blank-space
character (ASCII character 32).
|
Figure 2.12 Generalised product structure |
2.3.2.2 Main Product Header (MPH) The MPH has a fixed length. Its
format is identical for all ENVISAT products.
The major types of information found in the MPH
are:
- product identification (consolidated or
unconsolidated, processing stage,
reference document describing the product)
- data acquisition and processing
- sensing time (start, stop)
- geolocation (start, stop)
- orbit information (phase, cycle, orbit
number, state vector, state vector origin)
- auxiliary time tagging information
(UTC/SBT, leap indicator, clock step)
- product confidence data (PCD)
- product size/structure information
- check sum
Contents
The Main Product Header (MPH) identifies the
product and its main characteristics. The
Main Product Header is an ASCII structure
containing information needed for all
ENVISAT sensors. It is of fixed length and
format for all products. The MPH contains
the following major types of information:
Product Identification Information
This information includes the file name of
the product (which describes most of the
essential features of the product, such as
instrument, mode, and processing level), the
consolidation level of the product, and the
document ID number of the documentation
describing this product.
Information Regarding Data
Acquisition and Processing
This information identifies where the product
was acquired, where it was processed, when
it was processed, and what hardware/software
performed the processing.
Information on Time of Data
Included in these fields are the UTC start and stop time of
data sensing.
Information on ENVISAT Orbit and Position
These fields contain orbit positioning data
which allow one to determine the exact
position of the satellite at the time of sensing.
SBT to UTC Conversion Information
This data allows for precise conversion from
Satellite Binary Time (SBT), as stored in
Instrument Source Packets (ISPs), to the
conventional UTC time system.
Product Confidence Data
(See "Product
Confidence Data" above.) 2.3.1.4.
Product Size Information
These fields identify the size of various
structures within the product so that they
may be accurately interpreted. (See "Processed Product
Sizes and Coverage" 2.3.1.6. above.)
Format
All entries are left justified unless
otherwise stated (i.e., any spare characters
within an entry are included at the end of
the entry). If blank characters are
included at the end of a multi-character
string, the blanks are written before the
closing quotation mark, not after. The
fields of the MPH are presented below.
Note: the MPH is common to all sensors and
products. It is provided as a separate file
within the Data Formats 6.6.1. section,
for easy reference.
2.3.2.3 Specific Product Header (SPH) The Specific Product Header (SPH)
is defined for each product and provides
additional information, applicable to the whole
product, related to the processing. Its
length is fixed for a given product. It contains
one SPH descriptor (an ASCII string describing
the product), and at least one data set
descriptor (DSD). The SPH will follow an ASCII
keyword value format similar to the MPH. It can
contain:
- product descriptor
- time and geolocation of first and last
valid DSR
- calibration data
- instrument mode data
- version number of characterisation files
- version number of characterisation
look-up tables
- processing algorithms information
- product confidence data (PCD)
- annotations
- data set descriptors (DSDs)
The SPH will follow an ASCII
keyword-value<units>-terminator format
identical to that of the MPH. The detailed
SPH structure and contents are given in
the sections where each specific product is
described in the handbook.
2.3.2.4 Data Set Descriptors (DSD)
The Data Set Descriptors (DSD) are used to
describe an attached Data Set or to provide
references to external files relevant to the
current product (e.g., auxiliary data used
in processing but not included with the
product). There must be one DSD per Data Set
or per reference to an external file.
The DS may be a Measurement Data Set (MDS),
an Annotation Data Set (ADS) or a Global
Annotation Data Set (GADS).
Contents
All DSDs have the same format. The DSD is
contained within the SPH as shown in Figure2.12 "Generalised
product structure" above. As such,
the DSD is also in ASCII format. The DSD
contains information specific to a given
Data Set within the product. The general
contents of a DSD are shown in Table 2.6 below "General
DSD format."
Format
The structure of the DSDs will be the same
for all products and all instruments. The
ASCII format conventions are the same as
those used for the MPH and SPH. This
structure is referred to as "dsd"
throughout this handbook. The general format
is shown in Table 1 below:
|
Table 2.6 General DSD format
|
Field #
|
Description
|
Units
|
Byte length
|
Data Type
|
1
|
DS_NAME=
|
keyword |
8
|
8*uc |
quotation mark (")
|
-
|
1
|
uc |
Data Set Name
Name describing the data set.
Characters not used are blanked.
|
-
|
28
|
28*uc |
quotation mark (")
|
-
|
1
|
uc |
newline character
|
terminator
|
1
|
uc |
2
|
DS_TYPE=
note: the
"DSD Type" flag has
been combined with the "DSD
Attachment" flag by
allowing more possible
letters. |
keyword |
8
|
8*uc |
DS Type = M
if a Measurement DS is attached.
= A if an Annotation DS is
attached = G if a Global ADS is
attached = R if no DS is
attached (reference
DSD only) |
-
|
1
|
uc |
newline character
|
terminator
|
1
|
uc
|
3
|
FILENAME=
|
keyword |
9
|
9*uc |
quotation mark (")
|
-
|
1
|
uc |
External Product Reference
If the DS Attachment flag was
set to R this field contains
the name of the referenced
product using the standard
naming convention
of the MPH. If the DS
Attachment Flag was set to
A, M, or G, this field may
contain the name of the file
from which the Data Set was
copied, or it may be blank
(set to ASCII blank space
characters). For a product
which was supposed to
contain a data set or
reference to one,
but the file was
unavailable, the first 7
characters of this field may
be set to MISSING and the
rest blanked. If space for a
DSD has been set aside in
the SPH, but the DSD is not
used in the current product,
this field may be set to NOT USED.
|
-
|
62
|
62*uc |
quotation mark (")
|
-
|
1
|
uc |
newline character
|
terminator
|
1
|
uc
|
4
|
DS_OFFSET=
|
keyword |
10
|
10*uc |
DS Offset in bytes
Gives the position of the
first byte of the
corresponding DS with
respect to the whole
product. Set to 0 if no DS
is attached.
|
bytes |
21
|
Ad |
<bytes> |
units
|
7
|
7*uc
|
newline character
|
terminator
|
1
|
uc
|
5
|
DS_SIZE=
|
keyword |
8
|
8*uc |
Total Size of DS in bytes
Length in bytes of the Data
Set. Set to zero if no DS is attached.
|
Bytes |
21
|
Ad |
<bytes> |
units
|
7
|
7*uc
|
newline character
|
terminator
|
1
|
uc
|
6
|
NUM_DSR=
|
keyword |
8
|
8*uc |
Number of DSRs within the DS
Number of Data Set Records
within the DS, set to zero
if no DS is attached.
|
-
|
11
|
Al |
newline character
|
terminator
|
1
|
uc
|
7
|
DSR_SIZE=
|
keyword |
9
|
9*uc |
Length of the DSRs in
bytes Length of
each DSR if DSR length is
constant within the Data Set. 0
= no DSRs attached (i.e. no DS
attached) -1 = DSR length is
variable. |
Bytes |
11
|
Al |
<bytes> |
units
|
7
|
7*uc
|
newline character
|
terminator
|
1
|
uc
|
8
|
Spare (blanks)
|
ASCII |
32
|
32*uc |
newline character
|
terminator
|
1
|
uc
|
TOTAL
|
|
280
|
|
Example Data Set Descriptors
All DSDs must be the same size. In order to
clarify the use of DSDs within the product
structure, the 5 possible DSD contents are
shown explicitly below. All DSDs should
fall into one of the following categories.
Note that the symbol is used to denote the
ASCII blank space character (ASCII
character 32). Values given in the following
examples are for illustrative purposes only
and may not correspond to the true values.
DSD Pointing to a Data Set
If the DSD points to a Data Set actually
contained within the current product, the
contents of the DSD will follow the example
given below. The example given below
assumes that the ASAR SR/GR ADS is being
described, however the same format would be
used for any other ADS, MDS or GADS.
DS_NAME="SR/GRADS"
DS_TYPE=A
FILENAME=""
DS_OFFSET=+00000000000000012345<bytes>
DS_SIZE=+00000000000000006788<bytes>
NUM_DSR=+0000000002
DSR_SIZE=+0000003394<bytes>
DSD Referencing a File
If the DSD is referencing a file external to
the current product, the contents of the DSD
will follow the example given below. The
example below assumes that the ASAR
Processor Configuration file is being
referenced; however, the same format would
be used with any other file.
DS_NAME="ASARProcessorConfig.File"
DS_TYPE=R
FILENAME="ASA_CON_AXVPDK19990324_150411_19990325_123000_20001231_101413"
DS_OFFSET=+00000000000000000000<bytes>
DS_SIZE=+0000000000000000000<bytes>
NUM_DSR=+0000000000
DSR_SIZE=+0000000000<bytes>
DSD Not Used
In some cases, space may be allocated to a
DSD in an SPH, but the DSD is not used
during normal processing of the current
product either due to the nature of the
product or due to operator-selected options
during product processing. For example, room
may be allocated for an ADS, but the ADS
is not included normally in the current
product. In such cases, the DS_NAME and
DS_TYPE fields are filled as they normally
would be, but the FILENAME entry is set to
NOT USED as shown in the example below. The
remaining fields are set to values of zero.
The example below is for the ASAR SR/GR ADS
and assumes that we are describing an ASAR
Single-Look Complex product. This ADS is not
included in the ASAR SLC products; however,
space for it is included in the ASAR Image
Products SPH. Therefore, this is a good
example of a DSD which is not used for a
certain product. In this case, the DSD would
have the following format:
DS_NAME="SR/GRADS"
DS_TYPE=A
FILENAME="NOTUSED"
DS_OFFSET=+00000000000000000000<bytes>
DS_SIZE=+00000000000000000000<bytes>
NUM_DSR=+0000000000
DSR_SIZE=+0000000000<bytes>
Spare DSD
A spare DSD is simply 279 blank space
characters () followed by a newline
character. This structure is referred to as
"dsd_sp" throughout this handbook.
DSD for a Missing Data Set or File
If a Data Set was supposed to be included in
the product or an external file was supposed
to be referenced, but for some reason was
not, the DSD indicates this by setting
the FILENAME field to MISSING. Note that
this is different from the NOT USED DSD. The
MISSING DSD indicates that something
unexpected has happened, and a reference or
Data Set that was supposed to be included
has not been. In contrast, the NOT USED
DSD indicates that a file or Data Set is
simply not used for this product - nothing
unexpected has happened.
Note that for most products, if an auxiliary
file could not be read or a Data Set
produced, then the entire product will not
be produced. Thus the MISSING DSD will
only be used in special situations which
warrant its use.
The example below assumes the ASAR Chirp
parameter ADS was supposed to be in the
product, but was not produced for some
reason. The following DSD would thus be produced:
DS_NAME="CHIRPPARAMSADS"
DS_TYPE=A
FILENAME="MISSING"
DS_OFFSET=+00000000000000000000<bytes>
DS_SIZE=+00000000000000000000<bytes>
NUM_DSR=+0000000000
DSR_SIZE=+0000000000<bytes>
2.3.2.5 Data Set (DS)
Contents
The data set (DS) contains the
instrument's scientific measurements
and it is the actual data of interest. The
DS is composed of data set records
(DSR). There are as many DSRs as necessary
to complete a DS within a product and that
number will depend on the product type.
In addition, the size of Data Sets within a
product may vary. There are two types of
data sets: measurement data sets (MDS)
containing instrument data, known as MDSRs,
and annotation data sets (ADS) containing
auxiliary data, known as ADSRs. In addition,
ADSs may exist in two forms. The basic
ADS contains time-stamped ADSRs, which can
be used to relate the information to the
correspondingly time-stamped MDSRs.
Global Annotation Data Sets (GADS), however,
contain information which pertains to the
full product and thus each GADSR may not
be time-stamped.
Format
The Data Set is in a mixed-binary format.
This may consist of integers, floats,
characters (1-byte numbers), or ASCII values
and ASCII strings. Note that for ASCII
multi-character strings in the Data Sets,
quotation marks are not used to enclose the string.
A Data Set is composed of Data Set Records
(DSRs), as shown in Figure2.12 "Generalised
Product Structure." For Level 1B and
Level 2 products, the structure includes:
- The start time of the DSR in MJD 2000
(MJD) format.
- The DSR length (optional: include if DSR
size is variable within the Data Set).
- Quality Flag: for MDSRs, a signed
character is used to indicate the MDSR
quality. A value of -1 indicates the
MDSR is a blank MDSR (used for
Level 1B and Level 2 only).
- Attachment Flag: for ADSRs, a signed
character flag may be included to
indicate if corresponding MDSRs exist
for the ADSR (1 = error, no
corresponding MDSRs, 0 = no error). This
flag is used to identify large gaps in
the sequence of MDSRs. (To simplify
processing, this flag is only used
for geolocation ADSs (LADSs), and
Summary Quality ADSs (SQADSs). In all
other cases, it is always set to zero.)
In LADSs, this flag identifies
empty granules. An example is provided
in Figure2.13 "Example
of Attachment Flag Usage". If an
ADS corresponds to more than 1 MDS, the
attachment flag is evaluated for
each MDS in turn, then combined into 1
flag via a logical OR operation.
- For GADS, no time stamp or flag is required.
- The data itself.
For Level 0 data, the MDSRs
contain Annotated Instrument Source Packets
(AISPs) preceded by a time stamp (sensing
time) in MJD 2000 format.
|
Figure 2.13 Example of attachment flag usage |
For the format of the Level 0 SPH and MDSR
refer to the "Data Format(Chapter 6. )" section.
|