ESA Earth Home Missions Data Products Resources Applications
    24-Jul-2014
EO Data Access
How to Apply
How to Access
Index
Credits
AATSR Data Formats Products
Records
SST record 50 km cell MDS
BT/TOA Sea record 17 km cell MDS
ATS_TOA_1P_MDSR_conf
ATS_TOA_1P_MDSR_cl
ATS_TOA_1P_ADSR_pix
ATS_SST_AX_GADSR
Vegetation fraction for Land Surface Temperature Retrieval GADS
Topographic Variance data for Land Surface Temperature Retrieval GADS
Land Surface Temperature retrieval coefficients GADS
General Parameters for Land Surface Temperature Retrieval GADS
Climatology Variance Data for Land Surface Temperature Retrieval GADS
Level 0 SPH
Level 0 MDSR
Auxilliary Data SPH with N = 1
1.6 micron nadir view MDS
Summary Quality ADS
Scan pixel x and y ADS
Grid pixel latitude and longtitude topographic corrections ADS
Across-track Band Mapping Look-up Table
Configuration Data GADS
Processor configuration GADS
LST record 50 km cell MDS
Distributed product MDS
Level 2 SPH
SPH
10-arcminute mds
Limits GADS
Validation Parameters GADS
BT/TOA Land record 17 km cell MDS
General Parameters GADS
Temperature to Radiance LUT GADS
Radiance to Brightness Temperature LUT GADS
Medium/High Level Test LUT GADS
Infrared Histogram Test LUT GADS
11 Micron Spatial Coherence Test LUT GADS
11/3.7 Micron Nadir/Forward Test LUT GADS
11/12 Micron Nadir/Forward Test LUT GADS
Characterisation GADS
Browse Day_Time Colour LUT GADS
Browse SPH
Grid pixel latitude and longtitude topographic correction ADS
Level 2 SPH
Auxilliary Products
ATS_VC1_AX: Visible Calibration data
ATS_SST_AX: SST Retrieval Coeficients data
ATS_PC1_AX: Level-1B Processing configuration data
ATS_INS_AX: AATSR Instrument data
ATS_GC1_AX: General Calibration data
ATS_CH1_AX: Level-1B Characterization data
ATS_BRW_AX: Browse Product LUT data
Level 0 Products
ATS_NL__0P: AATSR Level 0 product
Browse Products
ATS_AST_BP: AATSR browse image
Level 1 Products
ATS_TOA_1P: AATSR Gridded brightness temperature and reflectance
Level 2 Products
ATS_NR__2P: AATSR geophysical product (full resolution)
ATS_MET_2P: AATSR Spatially Averaged Sea Surface Temperature for Meteo Users
ATS_AR__2P: AATSR averaged geophysical product
Frequently Asked Questions
The AATSR Instrument
Instrument Characteristics and Performance
In-flight performance verification
Instrument Description
Internal Data Flow
Instrument Functionality
AATSR Products and Algorithms
Common Auxiliary data sets
Auxiliary Data Sets for Level 2 processing
Instrument Specific Topics
Level 2 Products
Level 1B Products and Algorithms
Level 1B Products
Algorithms
Instrument Pixel Geolocation
Availability
The Level 0 Product
Differences Between ATSR-2 and AATSR Source Packets
Definitions and Conventions
Conventions
Organisation of Products
Relationship Between AATSR and ATSR Products
AATSR Product Organisation
Data Handling Cookbook
Characterisation and Calibration
Monitoring of AATSR VISCAL Parameters
Latency, Throughput and Data Volume
Throughput
Introduction
Heritage
Data Processing Software
Data Processing Centres
The AATSR Products User Guide
Image Gallery
Breakup of the Ross Ice Shelf
Land cover in the Middle East
Typhoon Saomai
Mutsu Bay, Japan
Deforestation in Brazil
Spatially Averaged Global SST, September 1993
Further Reading
How to use AATSR data
Why Choose AATSR Data?
Why Choose AATSR Data?
Special Features of AATSR
Principles of Measurement
Scientific Background
The AATSR Handbook
SST record 17 km cell MDS
Surface Vegetation class for Land Surface Temperature Retrieval GADS
1.6 micron forward view MDS
12 micron nadir view MDS
12 micron forward view MDS
Summary Quality ADS
Surveillance Limits GADS
Master Unpacking Definition Table GADS
1.6 micron Non-Linearity Correction LUT GADS
General Parameters GADS
Thin Cirrus Test LUT GADS
Fog/low Stratus Test LUT GADS
1.6 Micron Histogram
Browse MDS
ATS_CL1_AX: Cloud LUT data
Glossary
Pre-flight characteristics and expected performance
Payload description, position on the platform
Auxiliary products
Auxiliary Data Sets for Level 1B processing
Summary of auxiliary data sets
Calculate Solar Angles
Image Pixel Geolocation
Level 0 Products
Acquisition and On-Board Data Processing
Product Evolution History
Hints and Algorithms for Higher Level Processing
Data Volume
Software tools
Summary of Applications vs Products
Geophysical Coverage
Geophysical Measurements
ATS_TOA_1P_ADSR_sa
Visible calibration coefficients GADS
Level 1B SPH
LST record 17 km cell MDS
Conversion Parameters GADS
12 Micron Gross Cloud Test LUT GADS
ATS_PC2_AX: Level-2 Processor Configuration data
Level 2 Products
Hints and Algorithms for Data Use
BT/TOA Sea record 50 km cell MDS
BT/TOA Land record 50 km cell MDS
Level 2 Algorithms
Signal Calibration
Services
Site Map
Frequently asked questions
Glossary
Credits
Terms of use
Contact us
Search


 
 
 


2.6.1.1.6 Signal Calibration

Uncalibrated scan pixels are calibrated using calibration coefficients derived earlier. Pixel calibration uses these calibration coefficients to convert the pixel data to brightness temperature, in the case of the infra-red channels, or to reflectance in the case of the visible and near-visible channels.

In the case of the infra-red channels, this process makes use of look-up tables for the conversion of radiance to brightness temperature. Thus for each pixel and channel, the detector count from the source packet is converted to a radiance using the linear calibration law, and the resulting radiance is used to enter a look-up table from which the brightness temperature of the pixel is derived.

In the case of the visible channels, the linear calibration law gives a calibrated reflectance directly. Calibration parameters to be used by the current module for the visible channels are read from an input file.

Each channel within each view is calibrated independently, and separate calibration parameters (slope and offset) are used for odd and even numbered pixels within the scan.

If valid calibration parameters for a given scan, parity and channel are not available, or if the derived channel values are out of range, the pixel is flagged by an exceptional value. If the input channel count is negative, this implies that the channel is invalid; it has already been flagged by an exceptional value, and is output from the module unchanged.

2.6.1.1.7 Regrid Pixels

Calibrated AATSR pixels are regridded into co-located forward and nadir images, onto a 1 km grid (modified to allow for equal time sampling along-track), using the pixel x and y coordinates derived above.

2.6.1.1.8 Cosmetic Fill

Grid pixels which remain unfilled by the regridding process are filled by copying the value in their nearest unfilled neighbour. This process has the effect of approximately reconstituting original pixel sizes. Filling occurs only where actual pixels are large, and therefore widely spaced, but have been artificially forced into 1 x 1 km boxes. Nearest neighbour copying reverses this process and allows pixels to expand to a more representative size.

2.6.1.1.9 Determine Land-Sea Flag

Given the image pixel latitude and longitude, the surface type for each image pixel is derived using land/sea mask information from an external auxiliary file.

2.6.1.1.10 Cloud Clearing

The process of cloud-clearing, or the identification of cloud affected pixels, is accomplished by applying in turn a series of tests to the brightness temperature data in the 12, 11 and 3.7 µm channels, and to the reflectance data in the 1.6 µm channel. The pixel is flagged as cloudy if any one of the tests indicates the presence of cloud. Table 2.11 below lists the cloud clearing tests that are applied. All of the tests are of course conditional on the appropriate infra-red or 1.6 µm data being available.

Table 2.11 Cloud Clearing Tests
Test Comments
gross cloud test applied to nadir and forward views separately
thin cirrus test applied to nadir and forward views separately
medium/high level cloud test applied to nadir and forward views separately
fog/low stratus test applied to nadir and forward views separately
11 µm spatial coherence test applied to nadir and forward views separately
1.6 µm histogram test applied to nadir and forward views separately
11/12 µm nadir/forward test uses both views
11/3.7 µm nadir/forward test uses both views
infra-red histogram test applied to nadir and forward views separately

The tests are applied in the order in which they appear in table 2.11 . Although the order in which they are applied is not critical in all cases, some of the tests depend on the results of tests performed previously. The 1.6 µm test operates only on pixels not previously flagged as cloudy by the gross cloud test, the thin cirrus test or the 11 µm spatial coherence test (these are the other single view tests that operate on daytime data), and must therefore follow these. The infrared histogram test is always applied last, and only uses those pixels that have not been flagged as cloudy by any of the preceding tests.

The 1.6 µm test operates on daytime data only. The tests involving the 3.7 µm channel, on the other hand, are only applied to night-time data, because reflected solar radiation may be significant in this channel during the day. Those tests that involve only the 11 and 12 µm channels are applicable to both daytime and night-time data. Not all of the tests are implemented over land.

A series of cloud state flags is defined for each pixel and for the forward and nadir view separately. The flags are listed in table 2.12 .

Table 2.12 Cloud-clearing/land flagging flags (nadir or forward view).
# Meaning if set
0 Pixel is over land
1 Pixel is cloudy (result of all cloud tests)
2 Sunglint detected in pixel
3 1.6 µm reflectance histogram test shows pixel cloudy (day-time only)
4 1.6 µm spatial coherence test shows pixel cloudy (day-time only)
5 11 µm spatial coherence test shows pixel cloudy
6 12 µm gross cloud test shows pixel cloudy
7 11/12 µm thin cirrus test shows pixel cloudy
8 3.7/12 µm medium/high level test shows pixel cloudy (night-time only)
9 11/3.7 µm fog/low stratus test shows pixel cloudy (night-time only)
10 11/12 µm view-difference test shows pixel cloudy
11 3.7/11 µm view-difference test shows pixel cloudy(night-time only)
12 11/12 µm thermal histogram test shows pixel cloudy

These flags are set according to the results of the tests. Thus if one of the flags numbered 3 to 12 is set, this means that the corresponding test has indicated the presence of cloud. If on completion of the cloud-clearing sequence any of these flags is not set, it may mean either that the test did not indicate the presence of cloud, or that the test was not applied because suitable data was lacking.

Each test makes use of a look-up table of parameters with which the brightness temperature or reflectance data is compared. Where tests are applied to forward and nadir view images separately, the parameters may be defined separately for the two cases. More generally, the comparison parameters may depend on the air mass in the line of sight, and this is implemented by allowing the tabular parameters to depend upon the across-track band. For the purposes of cloud clearing and Level 2 processing, the (512 km) AATSR swath is imagined as divided into 10 bands parallel to the satellite ground track. These bands are numbered from 0 to 9, and each is 50 km wide except that the outer two are 56 km wide.

2.6.1.1.10.1 Gross cloud test

The gross cloud test uses the 12 µm channel, and is applied to single pixels for both day and night-time conditions. It is based on the principle that the temperature of the atmosphere falls with altitude, so that the brightness temperature of an optically thick cloud at high altitude will be significantly is less than the surface temperature. The test then consists of comparing the 12 µm brightness temperature with a threshold value that is set to be lower than the brightness temperature expected from the surface. If the 12 µm brightness temperature is lower than the threshold, the pixel is flagged as cloudy. The 12 µm channel is used because the optical depths of the clouds tend to be greater here than at shorter wavelengths.

Clearly the threshold value must be chosen with care. The appropriate threshold will depend on whether the pixel is over land or over sea, on the time of year, and on the geographic region. In principle the threshold might depend on angle of incidence; in practice the dependence of brightness temperature with incidence angle is sufficiently slow that it is sufficient to have a different set of thresholds for the forward and nadir views. Changes in the angle of incidence within the view (which might be applied by varying the threshold with across-track band) are not regarded as significant.

The choice of a suitable threshold is easier over sea than over land, since sea surface temperature in a given region varies only slowly during the year. Therefore the present algorithm applies the test to data over the sea only. The test uses a look-up table with values of the threshold tabulated at intervals of 1 ° of latitude for each view (forward and nadir), and for each calendar month.

Each pixel is examined separately. If the pixel is valid and if the 12 µm brightness temp

I(ir12, v; i, j) < gross_cloud_threshold eq 2.139

where gross_cloud_threshold is the appropriate threshold value from the 12 µm gross cloud test LUT 6.6.21. , for the latitude and month of observation, the pixel is flagged as cloudy.

2.6.1.1.10.2 Thin cirrus test

The thin cirrus test uses the differences between the measured brightness temperatures at 11 and 12 µm; this difference may be large over optically thin cirrus cloud owing to emissivity differences at the two wavelengths (Saunders and Kriebel 1988 Ref. [1.16 ] ).

The brightness temperature difference is calculated for each pixel and compared with a tabulated threshold value; if the difference exceeds the threshold, the pixel is flagged as cloudy.

The threshold for this test is a function of both the brightness temperature at 11 µm and the air mass in the line of sight; the latter is represented by the across-track band number. The look-up table then consists of values for each view separately, for each across-track band, and for temperature values at 1 K intervals between 250 and 310 K. In order to enter the table, a brightness temperature index is defined as the integer part of (T-250), in units of K.

The test uses a look-up table that defines a threshold thin_cirrus_threshold for the nadir and forward views separately, for each across-track band and for values of the brightness temperature index from 0 to 60 inclusive. It is applied to the forward and nadir views separately. For each pixel, if the difference

I(ir11, v; i, j) - I(ir12, v; i, j) > thin_cirrus_threshold(bt_index, band_no)eq 2.140

for the appropriate view, the pixel is flagged as cloudy. The appropriate value of the quantity thin_cirrus_threshold is taken from the thin cirrus cloud LUT 6.6.24. .

2.6.1.1.10.3 Medium/high level cloud test

This test uses the difference between the 3.7 and 12 µm channel brightness temperatures. This difference is calculated for each pixel and, if it is greater than a threshold value, the pixel is flagged as cloudy (Saunders and Kriebel 1988 Ref. [1.16 ] ). The threshold value in this case is a function of the 12 µm brightness temperature, and is tabulated with a resolution of 0.5 K. Separate tables are defined for the forward and nadir views.

The test uses a look-up table that defines a threshold med_high_level_thresh for the nadir and forward views separately, for values of the brightness temperature index from 0 to 120 inclusive. The brightness temperature index is defined as the integer part of (T-250), in units of K. The test is applied to the forward and nadir views separately. If the 3.7 and 12 µm brightness temperatures are valid, and the difference

I(ir37, v; i, j) - I(ir12, v; i, j) > med_high_level_thresh(bt_index)eq 2.141

for the appropriate view, then the pixel is flagged as cloudy. The test is only performed for night-time data; specifically, it is only performed for pixels on image rows such that at each end of the row, the band centre solar elevation for the view in question is less than 5°. The appropriate value of the quantity med_high_level_threshold is taken from the med/high level cloud LUT 6.6.23. .

2.6.1.1.10.4 Fog/low stratus test

At night the difference between the brightness temperatures at 11 and 3.7 µm may be used as a sensitive indicator of fog or low stratus cloud (Saunders and Kriebel 1988 Ref. [1.16 ] ), because the emissivity of such cloud is lower at 3.7 µm than at 11 µm.

The brightness temperature difference is calculated for each pixel and compared with a threshold; if the measured difference exceeds the threshold, the pixel is flagged as cloudy. The threshold depends on the air mass in the line of sight, and so different thresholds are defined for each across-track band, and for the nadir and forward view images separately. The test uses a look-up table that defines a threshold fog_low_stratus_threshold for the nadir and forward views separately, for each across-track band.

The test is only performed for pixels on image rows such that at each end of the row, the band centre solar elevation for the view in question is less than 5°, so excluding day-time data. For such pixels, if the difference

I(ir11, v; i, j) - I(ir37, v; i, j) > fog_low_stratus_threshold(band_no)eq 2.142

for the appropriate view then the pixel is flagged as cloudy. The appropriate value of the quantity fog_low_stratus_threshold is taken from the fog/low stratus LUT 6.6.22. .

2.6.1.1.10.5 Spatial coherence test (11 um)

The spatial coherence test actually comprises two tests, the small-scale and the large-scale tests. It is applied to the 11 µm image, and can be used for both day and night-time conditions. The 11 µm channel is used in preference to either the 3.7 µm or the 12 µm channel in this test because it is available in both day and night-time and, for ATSR and ATSR-2 operating with L-rate telemetry, its digitisation accuracy was better, or as good, as that of the other channels. The latter consideration does not apply to AATSR, but the test is unchanged to ensure continuity with the earlier instruments.

The small-scale spatial coherence test determines whether the variability of the 11 µm brightness temperature is greater than would be expected for clear conditions; if so, it assumes that cloud is present. To apply the test, the standard deviation of a 3 x 3 array of pixels is calculated and compared with a pre-defined threshold; cloud is flagged if the threshold is exceeded. The test can be applied over both land and sea surfaces, but different thresholds are used for the two cases. In particular a lower standard deviation threshold is used for land data at night than during the day, because the variability of the land surface temperature is found to be lower at night.

The test is only applied to 3 x 3 arrays of pixels that are all over sea, or all over land. It is not applied to mixed or coastal regions, where surface temperature variations may be large (Saunders and Kriebel 1988).

A refinement has been added to the basic small-scale test as defined by Saunders and Kriebel (1988) to allow for the possibility that the observed spatial variability may result from ocean fronts rather than cloud. This uses the difference between the 11 and 12 µm channels as a marker. It is assumed that this difference is insensitive to surface temperature variations, but will be sensitive to the presence of cloud. Thus if a group of 9 pixels have been flagged as cloudy by the basic test of spatial variability, they are further tested as follows. The average difference between the 11 and 12 µm channel brightness temperatures for the array of pixels is calculated, and compared with the same average calculated for the neighbouring cloud-free pixels. If the two averages differ by less than a pre-defined threshold, the pixels of the original group are 'unflagged'; that is, the flags representing the outcome of the test are reset to clear. The neighbouring cloud-free pixels in this case are the members of the eight surrounding 3 x 3 groups of pixels that have been flagged as clear by the basic spatial variability test.

The test is performed separately for the nadir and forward views. The nadir view 12 µm and 11 µm data are used in the nadir view test and the forward nadir view 12 µm and 11 µm data are used in the forward view test. It is assumed that the along-track size of a view is 512 km. That is, the orbit is imagined as tiled with 512 by 512 pixel images, each of which is treated separately. If an incomplete image is encountered at the end of data, the image is filled up to 512 by 512 pixels with unfilled pixels.

The small scale spatial coherence test uses a square array of 3 x 3 pixels.

The large-scale spatial coherence test is applied after the unflagging stage of the small-scale test has been completed, over open ocean only. It is logically a separate test, although it is defined as part of the same algorithm module, and as currently implemented only one flag is used to record the combined result of the small-scale and large-scale tests. Thus you cannot tell, if the flag is set, which of the two tests (if not both) has actually detected cloud.

The large-scale spatial coherence test is designed to detect low-lying marine stratiform cloud. The small-scale spatial coherence test is ineffective over uniform stratus cloud by reason of its very uniformity, and indeed this type of cloud is difficult to detect by any of the cloud tests, particularly if it is low-lying, so that the temperature differential between the cloud and clear sea is small.

The physical concept underlying the test is similar to that of the gross cloud test, that clouds are cooler than sea. However, in these circumstances it is difficult if not impossible to set a suitable threshold to differentiate between cloud and clear sea. This is particularly the case where there are significant gradients of SST; the physical basis of the test is that cloud is colder than clear sea, so that if a threshold were set that was valid at a particular place, then in the presence of temperature gradients, clear sea pixels would be rejected in colder regions, while in an area that was fully cloud covered, in warmer regions cloudy pixels might be passed as warmer than the threshold.

The detailed rationale for this test is described by Zavody et al (Zavody, A.M, Mutlow, C.T. and Llewellyn-Jones, D.T. Journal of Atmospheric and Oceanic Technology, 17, 595 – 615, 2000). The test attempts to overcome these problems by setting a dynamic threshold adapted to local conditions. The test works with rectangular areas of fixed size (the size is set by an auxiliary parameter, and in practice is 128 by 128 km), and the threshold used within a particular area is based on conditions in the neighbouring areas that surround it.

In order to set a realistic threshold for a given area, the clear sea pixels in the neighbouring areas are examined. In each area the maximum 11 micron BT of the cloud-free pixels in the area is determined. This is assumed to be representative of clear sea in the area, while the distribution of the BTs from the set of neighbouring areas is assumed to be representative of the distribution at 128 km resolution. It is then assumed that if the smallest of these representative temperatures is selected, the probability that the temperature of clear sea in the central region is lower than this is itself low. The dynamic threshold is then set on the basis of this temperature.

The large-scale spatial coherence test uses data from sub-areas of COH_AREA_SIZE x COH_AREA_SIZE. The value of COH_AREA_SIZE must be chosen so that it is a factor of 512. Its current value is 128 km.

The Small-Scale Spatial Coherence Test

The small and large-scale spatial coherence tests are applied to an image array of 512 by 512 pixels. Let the pixels be identified by indices i, j in the along-track and across-track directions respectively, where 0 ≤ i < 512 and 0 ≤ j < 512.

The small-scale spatial coherence test operates on arrays of 3 by 3 pixels, known as pixel groups. The 512 by 512 pixel image is divided into 171 by 171 such groups; let the groups be indexed by i_group and j_group, where 0 ≤ i_group < 171 and 0 ≤j_group < 171. In general, the pixel indexed by (i, j) falls in the group indexed by i_group = int[i/3], j_group = int[j/3].

However, because the image size is not a multiple of 3, this rule is modified at the edges of the image. The pixels that fall within the group defined by (i_group, j_group) are those whose indices satisfy

, ,
where is the smaller of (3i_group + 1, 509) and is the smaller of (3j_group + 1, 509). Thus the last group in each row consists of the pixels having j = 509, 510, 511. If we denote the set of pixels comprising the group indexed by (i_group, j_group) by G(i_group, j_group) then

G(i_group, j_group) = {i, j; , } where i 0 and j 0 are as defined as above. This implies that the last two groups in each row or column overlap, so that any pixel having either i = 509 or j = 509 falls into two adjacent groups.

The auxiliary parameters used in this test are listed in Table 1.1. They are taken from the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX.

Table 2.13 Auxiliary parameters used by small-scale spatial coherence test

Parameter

Value

SEA_MAX_DEV

20

LAND_DAY_MAX_DEV

150

LAND_NIGHT_MAX_DEV

100

COHERENCE_RESET_THRESH

10

The same parameters are used for both nadir and forward view image.

The test is performed separately for the nadir and forward view image. For each image, and for each group of 3 x 3 pixels within the image, do the following steps.

1) Determine the solar elevation corresponding to the across-track band containing the group.

2) Determine whether all the valid pixels in the group are over land or over sea. Set the group_land_flag if any pixels within the group are land pixels. Proceed to the next group if the set contains a mix of land and sea pixels, or if there are 2 or fewer valid pixels.

3) Calculate the mean and standard deviation of the 11µm brightness temperatures for the valid pixels in the group, excluding cosmetic fill pixels.

4) Select the appropriate threshold. If the pixel group is over sea set the threshold to SEA_MAX_DEV. If the group is over land, inspect the solar elevation angle determined in Step 1. If the solar elevation is greater than 5° set the threshold to the daytime value LAND_DAY_MAX_DEV; otherwise set the threshold to the night-time value LAND_NIGHT_MAX_DEV.

5) If the standard deviation is greater than the selected threshold, flag the group as cloudy and proceed to step 6, otherwise continue to the next 3 x 3 group of pixels.

6) In a second pass over the image, investigate whether the spatial variation may result from ocean gradients rather than cloud:

6.1) If 4 or more of the surrounding 8 pixel groups have been flagged as clear and the group contains more than 2 valid pixels proceed to Step 6.2; otherwise move to the next group.

6.2) Calculate the average difference between the 11 and 12 micron channel brightness temperatures for the central (cloudy) group and for those of the surrounding groups that are clear. If the absolute value of the difference between the averages for cloudy and clear groups is less than COHERENCE_RESET_THRESH, then clear the flags for all pixels of the group.

The Large-Scale Spatial Coherence Test

For the large-scale test, the image is sub-divided into n by n square sub-areas whose size is defined by an auxiliary parameter COH_AREA_SIZE, such that n = 512/COH_AREA_SIZE. In practice COH_AREA_SIZE = 128, so that there are 16 sub-areas of nominal size 128 by 128 km. The sub-areas may be indexed by i_sub, j_sub, 0 ≤ i_sub < n and 0 ≤j_sub < n.

The large-scale spatial coherence test operates the pixel groups, not on individual pixels. Because the sub-area size is not divisible by three, the relationship between the pixel groups and the sub-areas is complex. The groups for the sub-area indexed by i_sub, j_sub, are defined by the limits

jx1 = INT((j_sub x COH_AREA_SIZE)/3)

jx2 = INT(((j_sub + 1) x COH_AREA_SIZE)/3) - 1

iy1 = INT((i_sub x COH_AREA_SIZE)/3)

iy2 = INT(((i_sub + 1) x COH_AREA_SIZE)/3) – 1.

Here INT denotes the integer part function.

Pixel groups are associated with the sub-areas as follows. The set of groups that corresponds to the sub-area indexed by i_sub, j_sub comprises those groups whose indices i_group, j_group satisfy

(i_group, j_group) ∈ A(i_sub, j_sub),

where the set of index pairs A(i_sub, j_sub) is formally defined by

A(i_sub, j_sub) = {(i_group, j_group): jx1 ≤ j_groupjx2;
iy1 ≤ i_groupiy2}

Since the number of groups is not a multiple of n, the sets A(i_sub, j_sub) do not each contain the same number of groups.

The large-scale test uses the additional auxiliary parameters listed in Table 1.2. They are taken from the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX. The detailed structure of the algorithm is as follows.

Table 2.14 Additional auxiliary parameters used by the large-scale spatial coherence test

Parameter

Value

COH_AREA_SIZE

128

COH_FRACTION_PASSED

0.005

COH_ADJ_THRESH_LAND

400

COH_ADJ_DIF_LAND

0.20

COH_AREA_DIF_NV

25

COH_AREA_DIF_FV

35

COH_MIN_DIF_NV

-15

COH_MIN_DIF_FV

-15

COH_AREA_THR_NV

200

COH_AREA_THR_FV

250

CLOUDY_BOX_THRESH

-1

1. Flag groups in coastal regions

1) By reference to the group_land_flag determined at Step 2 of the small-scale spatial coherence test, flag all groups near to land, and all land groups. For each group set an extended_land_flag(j_group, i_group) if

ΣΣgroup_land_flag(i, j) > 0

where the summations extends over indices in the ranges of

j_group - 3 < i < j_group + 3,

i_group - 3 < j < i_group + 3.

i and j must also be in the range of 0 to 170.

2. Find maximum brightness temperature (BT) and BT differences in sub-area

2) For each sub-area, find the group(s) with the highest average 11-mm brightness temperature, and the average (11-µm – 12-µm) brightness temperature difference(s) of these:

2.1) Find the highest average 11mm brightness temperature in the sub-area (max_bt_11) and its/their coordinates. Use only valid groups, where a valid group is one that satisfies the following conditions:

  • its extended_land_flag is not set;
  • it has passed the small-scale coherence test; that is, its group_cloud_flag is not set;
  • it has not been shown to contain significant cloud by any previous test;
  • it contains at least 3 natural pixels having valid 11 micron and 12 micron brightness temperatures and so is able to contribute to the computation of the average (11 micron - 12 micron) brightness temperature difference at 2.2) below. (A natural pixel is one that is neither cosmetically filled nor unfilled). In the following the number of such pixels is denoted by valid_pixel_pairs.

The flag previous_tests(j_group, i_group) is determined as follows:

If the auxiliary parameter CLOUDY_BOX_THRESH = -1, the flag is determined by inspecting only the central pixel of the group; previous_tests(j_group, i_group) is set if the pixel (i, j) was flagged by the gross cloud test, the thin cirrus test, the medium-high level cloud test, or the fog – low stratus test, where i is the lesser of (3i_group + 1, 510) and j is the lesser of (3j_group + 1, 510); i and j index the central pixel of the group.

If CLOUDY_BOX_THRESH > 0, the value of previous_tests(j_group, i_group) is determined by inspecting all pixels of the group: it is set if (n_cloudy ³ CLOUDY_BOX_THRESH), where n_cloudy is the total number of pixels from the group defined by the index set in G(i_group, j_group) that have been flagged as cloudy by one of the above cloud tests.

Then max_bt_11 is maximum value of the average 11 micron brightness temperature (determined at Step 3 of the small scale spatial coherence test above) over the set of indices

(j_group, i_group) ∈ Avalid (j_sub, i_sub)

The set Avalid (j_sub, i_sub) is the set of indices of valid groups within the sub-area, and is defined by

Avalid (j_sub, i_sub) = {j_group, i_group: (j_group, i_group) ∈ A(j_sub, i_sub)
and NOT group_cloud_flag(j_group, i_group)
and NOT extended_land_flag(j_group, i_group)
and NOT previous_tests(j_group, i_group)
and (valid_pixel_pairs(j_group, i_group) ³ 3)}

Set

sub_area_max_11(j_sub, i_sub) = max_bt_11.

2.2) Calculate the average brightness temperature difference(s) T 11T 12using valid pixel pairs (as defined above) for the groups with max_bt_11).

If a single group has the average 11µm brightness temperature value of max_bt_11 then assign its brightness temperature difference value to sub_area_di(j_sub, i_sub) f.

If more than one group in the sub-area has the same max_bt_11 value then find the highest of these differences, and assign this value to sub_area_dif(j_sub, i_sub).

2.3) It is possible that there are no valid groups; that Avalid is an empty set. In this case sub_area_max_11(j_sub, i_sub) and sub_area_dif(j_sub, i_sub) should each be set to an exceptional value (≤ 0) to ensure that they are ignored in Step 5 below.

3. Identify sub-areas containing land

3) Set the land_sub_area flag to 1 for all sub-areas in which one or more groups are over or near land. The flag is set if

Σextended_land_flag(j_group, i_group) > 0

where the sum includes all index pairs (j_group, i_group) ∈ A(j_sub, i_sub).

4. Identify valid sub-areas

4) Set the flag valid_sub_area_flag to 1 for all sub-areas for which the number of clear sea groups is greater than a fraction COH_FRACTION_PASSED of the total number, i.e.

S (1-group_cloud_flag(j_group, i_group)) x (1-extended_land_flag(j_group, i_group)) /
(COH_AREA_SIZE/3)2 > COH_FRACTION_PASSED,

where the sum includes all (j_group, i_group) ∈ A(j_sub, i_sub),

and sub_area_dif(j_sub, i_sub) is greater than COH_MIN_DIF. COH_MIN_DIF should take the value COH_MIN_DIF_NV for the nadir view, or COH_MIN_DIF_FV for the forward view, as appropriate.

5. Determine and apply dynamic threshold

5) Process each sub-area in turn. For invalid sub-areas, set the 11mm brightness temperature threshold (threshold_11) to 32000 cK i.e. to an unrealistically high value, so that all the sea pixels in these sub-areas are flagged, as cloudy and omit Steps (5.1) to (5.7). For sub-areas that are valid, determine threshold_11 as follows (Steps (5.1) to (5.7)):

5.1) Select the up to 9 sub-areas (valid or not) centred on the one being investigated. These are the sub-areas whose indices fall in the set

S(p, q) = {(j_sub, i_sub): 0 ≤ j_sub < n, 0 ≤ i_sub < n,
p - 1 ≤ j_subp + 1, q - 1 ≤ i_subq + 1}

where p, q, 0 ≤ p < n, 0 ≤ q < n, are the indices of the sub-area currently under consideration.

5.2) Set the flag land_in_areas if one or more of the sub-areas selected has land, that is if, for any (j_sub, i_sub) ∈ S(p, q), land_sub_area(j_sub, i_sub) is set.

5.3) Find bt_dif_max, the highest value of sub_area_dif, for valid selected sub-areas:

bt_dif_max = max(sub_area_dif(j_sub, i_sub)),

over the set of all index pairs (j_sub, i_sub) ∈ S(p, q) that identify valid sub-areas. (At least one sub-area, the central one, must be valid, so this operation is always possible.)

5.4) Calculate the difference threshold given by:

difference_threshold = COH_AREA_DIF*(1 + land_in_areas x COH_ADJ_DIF_LAND)

COH_AREA_DIF should take the value COH_AREA_DIF_NV for the nadir view, or COH_AREA_DIF_FV for the forward view, as appropriate.

Identify those values of sub_area_max_11 for which the corresponding sub_area_dif is within difference_threshold of bt_dif_max found in Step 5.3, that is, for which

sub_area_dif(j_sub, i_sub) > bt_dif_max - difference_threshold.

5.5) Find the lowest maximum brightness temperature of those sub-areas identified in step 5.4. Set lowest_max_bt to the minimum value of sub_area_max_11(j_sub, i_sub) over the set of index pairs (j_sub, i_sub) ∈ S(p, q) for which the sub-area is valid, and sub_area_dif(j_sub, i_sub) satisfies the test in Step 5.4.

5.6) Calculate threshold_11 for the sub-area investigated:

threshold_11 = lowest_max_bt - COH_AREA_THRESH
- land_in_areas x COH_ADJ_THRESH_LAND.

COH_AREA_THRESH should take the value COH_AREA_THR_NV for the nadir view, or COH_AREA_THR_FV for the forward view, as appropriate

5.7) If only one valid sub-area was selected at step 5.4, and the land_in_area flag is set, lower threshold_11 by 200 cK.

5.8) Flag all sea pixels in the sub-area investigated in those groups for which the average 11 micron brightness temperature (determined at Step 3 of the small scale spatial coherence test above) is below threshold_11 determined in Steps 5.1 to 5.7 above.

2.6.1.1.10.6 Histogram test (1.6 um)

2.6.1.1.10.7 11/12 um and 11/3.7 um nadir/forward tests

Atmospheric absorption in the 11 and 12 µm window channels is dominated by water vapour. The effect is stronger in the 12 µm channel than in the 11µm channel and hence the differences in brightness temperatures in the two channels also vary in an almost linear way with the atmospheric water vapour loading.

For either channel, the total effect is also roughly proportional to the amount of water vapour in the path, and hence it is greater for the forward-view than for the nadir view.

It follows from the above that there is a fairly well defined relationship between, for example, the nadir view 11 µm minus 12 µm brightness temperature differences and the nadir view minus forward view 11 µm brightness temperature differences. Both differences increase with atmospheric water.

The relationships derived for cloud-free atmospheres do not necessarily hold if there is a uniform cloud layer in the path. Also, owing to the stereoscopic effect, clouds with spatial dimensions of less than roughly their height above sea level affect the brightness temperature from a sea pixel in only one of the views and hence the relationship between the two differences is even more likely to differ from that for the clear atmosphere.

The test, therefore, consists of comparing the observed 11 µm (nadir-view - forward-view) differences with those expected on the basis of the relationship (derived from modelling) given the (11 µm - 12 µm) differences in the nadir view. If the discrepancy between the two is significant (greater than some threshold), the pixel is flagged as cloudy.

The effects of clouds on the 3.7 µm channel differ from those in the 11 and 12 µm window channels and hence at night it is useful to do the test with the 3.7 µm brightness temperature. For the case of clear atmospheres, the nadir-view minus forward view differences are small in this case, and vary in a non-linear way with the 3.7 µm minus 11 µm brightness temperature differences. A quadratic relationship is used to test the data.

These tests are not reliable over land for the following reasons:

  • The pixels are only collocated at sea level hence, over land, the brightness temperatures measured in the nadir view and forward view do not necessarily originate on the same land pixel. As land temperatures can have strong spatial variability over distances of a few kilometers, the brightness temperature differences in the two views from non-collocated pixels is often meaningless.
  • The emissivities for the different channels vary with the nature of the terrain and hence there is no unique relationship between the differences in two channels and the differences in the two views.

2.6.1.1.10.7.1 11/12 um nadir/forward test

The nadir-forward tests (Zavody et al, 2000) are based on the idea that for clear sea pixels there should be a correlation between the difference in brightness temperature between the nadir and forward view measurements at 11 microns and the difference between the nadir view 11 and 12 micron brightness temperatures, since both depend on the atmospheric characteristics, principally the water vapour loading. This relationship has been determined using radiative transfer modelling applied to an ensemble of atmospheric profiles. For each pixel the difference T11 - T12 is determined, and used to predict the view difference using this relationship. The actual view difference is compared with the prediction, and if they differ by more than a threshold, the pixel is assumed to be cloudy.

This test is only applied to sea pixels. It loops through the pixels in the 10 across-bands of an image and checks the relationship between the measured brightness temperature differences for each pixel, using the appropriate parameters and threshold. It uses the nadir view 12 µm and 11 µm brightness temperatures, and the forward view 11m brightness temperatures.

For each sea pixel the following steps are executed

1) The (11um - 12um) BT difference dif_11_12 = T 11 - T 12 is derived.

2) The 11um (nadir-view - forward-view) BT difference dif_nv_fv is derived.

3 Using the coefficients appropriate for the across-track band in which the pixel falls, the expected view difference is computed:

exp_dif_nv_fv = a 0 + a 1 x dif_11_12

4) If the difference abs( exp_dif_nv_fv - dif_nv_fv ) is greater than the threshold value, the pixel is flagged as cloudy.

The coefficients a 0 and a 1 for each across-track band, and the threshold value, are taken from the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX.

2.6.1.1.10.7.2 11/3.7 um nadir/forward test

This test is similar to the preceding test but uses the 37 micron and 11 micron brightness temperatures instead of the brightness temperatures in the 11 and 12 micron channels, and is used at night when the 3.7 micron data are available. The theoretical relationship between the view difference and the nadir view channel difference is slightly more complex in this case than in the preceding test, being quadratic rather than linear.

The test uses the nadir view 11 µm and 3.7 µm brightness temperatures and the forward view 3.7 µm brightness temperatures with the appropriate parameters and threshold. Like the preceding test, it is only applied to sea pixels.

The test is applied to all sea pixels measured at night, for which the 3.7um channel is available. For each such pixel the following steps are executed.

1) The (3.7um - 11um) BT difference dif_37_11 = T 37 - T 11 is derived.

2) The 3.7um (nadir-view - forward-view) BT difference (dif_nv_fv) is derived.

3) Using the coefficients appropriate for the across-track band in which the pixel falls, the expected view difference is computed:

exp_dif_nv_fv = a 0 + (a 1 + a 2 x dif_37_11) x dif_37_11

4) If the difference abs(exp_dif_nv_fv - dif_nv_fv) is greater than the threshold value, the pixel is flagged as cloudy.

The coefficients a 0, a 1 and a 2 for each across-track band, and the threshold value, are taken from the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX.

2.6.1.1.10.8 Infrared histogram test

This is the last of the cloud tests performed and it is applied only to those pixels that have passed all the previous cloud tests. It uses the 11 µm and 12 µm channels, and relies mostly on the following:

1. The difference between brightness temperatures measured in different channels is almost entirely determined by the atmospheric characteristics.

2. The atmospheric variability is usually small over the area of an AATSR image product. It follows that even when strong temperature gradients are present in the sea, the brightness temperature differences form a fairly tight cluster for the clear pixels.

3. Low stratus clouds, the most likely contamination to have escaped detection by the other tests, affect the brightness temperatures differently from the clear atmosphere. There are two effects:

a. The infrared radiation detected originates from a height above sea level and hence the lowest layer of the troposphere is only partially traversed by the radiation. As it is this layer that contains most of the atmospheric water vapour, which is the major source of the difference between the 11 µm and 12 µm brightness temperatures, these differences are reduced over clouds.

b. Low clouds have a slightly higher emissivity in the 12 µm channel than in the 11 µm channel. It follows that the proportion of reflected cold radiation is less for the 12 µm channel as for the 11 µm channel and hence the 12 µm brightness temperature over this type of cloud, compared with the 11 µm brightness temperature, is not as cold as over the clear sea.

The 12 µm brightness temperature is almost always lower than the 11 µm brightness temperature owing to the water vapour absorption. Both the above effects reduce the difference found over clear sea.

4. If the water vapour column amounts are not uniform over an image, and the sea surface temperature is constant, then the lowest brightness temperatures originate from pixels having the highest water vapour loadings and hence the highest (11 µm minus 12 µm) brightness temperature differences. It follows that, in this case, the brightness temperatures and the brightness temperature differences are negatively correlated for clear pixels and positively correlated for cloudy pixels. (N.B. Although this is generally true there are exceptions. A positive correlation in SST with water vapour loading, or lower tropospheric air warmer than the SST, can reduce the correlation or even change its sign.)

The test relies mostly on effects 1 - 3 listed above, with the correlation effect used as an additional check.

The histogram test operates on a 512 by 512 pixel image segment, so the orbit is divided into image segments of this size and imagined as 'tiled' with 512 row image segments. If (at the end of an orbit or at the end of a segment in stripline processing) the data is insufficient to form a complete image segment of 512 rows the histogram can be based on an incomplete image segment. Because histograms are made up of those pixels that are valid sea pixels, if an image is largely over land, one has an incomplete image anyway. Provided the number of pixels exceeds a minimum threshold number for this test, the test can be applied.

The auxiliary parameters used in this test are listed in Table 3. They are taken from the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX.

Table 2.15 Auxiliary parameters used by the infrared histogram test

MIN_FOR_11_12_HISTOGRAM

100.00

PEAK_FRAC_MIN

0.51

LATITUDE_THRESHOLD

40.0

SECOND_LOW_FRACTION

0.50

HALF_WIDTH_M_NV

0.5

HALF_WIDTH_B_NV

25.0

HALF_WIDTH_M_FV

1.0

HALF_WIDTH_B_FV

15.0

MAX_DIF_AVE_CHAN_1

100.0

MAX_DIF_PEAK_CHAN_1

50.0

RATIO_B

4.0

IR_SPREAD_NV

60.0

IR_SPREAD_FV

80.0

SLOPE_MAX_ALLOWED

10.0

IR_PEAK_MIN

25000.0

Determination of the Difference Threshold

Thus the overall objective of the algorithm is simple, to determine a dynamic threshold against which the brightness temperature differences can be compared to differentiate cloudy from clear pixels. The determination of this threshold is based on the histogram of the distribution of the differences in brightness temperature between the 11 and 12 micron channels. In the presence of undetected cloud, the histogram is expected to be the superposition of two peaks, representing the distribution of the brightness temperature differences of clear sea and of cloudy pixels respectively.

The algorithm attempts to identify the presence of the two peaks, and to identify which one represents clear sea. In the simplest terms, the clear sea pixels should show a narrower and more regular distribution, at a higher brightness temperature difference, than the cloudy pixels. However, the peaks cannot be distinguished by their relative height, since the height of each peak depends on the number of contributing pixels, which is arbitrary and simply reflects the area of clear sea or undetected cloud as appropriate. Moreover, the peaks may overlap.

Therefore the peaks are tested against various criteria to identify a valid peak, that is, a peak that shows the characteristics characteristic of the distribution of clear sea pixels. If a valid peak is identified, a threshold is set on its lower edge, and all pixels having a brightness temperature difference less than this threshold are flagged as cloudy.

The procedure involves the following stages.

1 A histogram representing the frequency distribution of the brightness temperature difference (T 11 - T 12) is prepared over all the sea pixels that have not been flagged as cloudy by any previous test. At the same time, for each histogram bin, the cumulative brightness temperature in the 12 micron channel is calculated over all the pixels that contribute to each bin.

The histogram is generated of the brightness temperature difference T diff = (T 11 - T 12) in the range -20.0 K 蠄 T diff < 80.0 K with a bin size dT = 0.1 K. If the histogram bins are indexed by k (0 蠄 k < 1000) then the lower limit of bin k corresponds to T = k x dT - 20.0 K. The corresponding histogram value h(k) represents the number of clear sea pixels for which

k x dT - 20.0 蠄 (T 11 - T 12) < (k + 1) x dT - 20.0

The cumulative brightness temperature for bin k is S T 12, where the sum is over all contributing pixels that fall in bin k.

2 The maximum value of the histogram is found, and its position is taken to represent the principal peak of the histogram. The extent of this peak is determined, and the parameters, height, width and position that describe the peak are calculated.

The maximum of the histogram defines the principal peak, referred to in the following as the major peak, and corresponds to the mode of the frequency distribution. If the bin index corresponding to the maximum is peak_interval[MAJOR], then the maximum value is

peak_value[MAJOR] = h(peak_interval[MAJOR]).

The extent of the peak is determined by finding the first local minimum or zero value, whichever is found first, on either side of the peak. The indices of these bins are lower_limit[MAJOR] and higher_limit[MAJOR].

3 The peak is tested against certain criteria to determine whether or not it is valid, that is, possesses the characteristics likely to represent clear pixels.

The peak is flagged as invalid if it corresponds to a difference value less than zero, that is, peak_interval[MAJOR] < 200, unless the absolute latitude at the centre of the image segment exceeds a threshold LATITUDE_THRESHOLD (currently set at 40 degrees) or the number of pixels peak_value[MAJOR] exceeds a threshold IR_PEAK_MIN (currently 25000). Otherwise it is flagged as valid.

If the histogram represents a night-time scene and if the peak has passed the previous tests, so is flagged as valid, a further test is applied to verify that the histogram is not the sum of a clear and a cloudy histogram and hence excessively broad.

Firstly, a more accurate value of the histogram peak is determined, by fitting a quadratic curve to the peak value and the adjacent histogram values on either side. If the histogram peak and its neighbours are

h0 = h(peak_interval[MAJOR] - 1)

h1 = h(peak_interval[MAJOR])

h2 = h(peak_interval[MAJOR] + 1),

then the quadratic function is

h(x) = 0.5x(x-1)h0 - (x - 1)(x + 1)h1 + 0.5x(x + 1)h2

= 0.5x^2(h0 - 2h1 + h2) + 0.5x(h2 - h0) + h1,

where x is a normalised abscissa co-ordinate measured from peak_interval. The maximum of this function occurs at

x = 0.5(h0 - h2)/(h0 - 2h1 + h2).

Thus provided the denominator of this expression is not zero, which can only occur if all three ordinates are the same, improved values of the histogram peak and its position are found. The improved peak value exact_peak_value is obtained by evaluating equation () at the value of x given by equation (), while the new position is

exact_peak_interval = (peak_interval + x)

ir11_ir12_diff_at_peak = (exact_peak_interval - 200)*dT

Next, the width of the histogram at half-height is computed to sub-bin accuracy by interpolation between the two histogram values bracketing the 50% peak values defined by 0.5* exact_peak_value and differencing the abscissae. A half-width threshold is computed from

half_width_threshold = M x ir11_ir12_diff_at_peak + B,

where M and B are constants defined in the auxiliary file of cloud LUT data, ATS_CL1_AX. (These constants have different values for the forward and nadir views.)

The peak is flagged as invalid if

half_width[MAJOR] > half_width_threshold.

4 A new histogram is determined by removing the peak just found (the major peak) by setting to zero the histogram values in the open interval between the upper and lower limits of the peak determined in (2) above. The maximum of the new histogram is found, and its position defines the secondary (or minor) peak. This is tested and flagged in the same way as the major peak.

The new maximum represents the largest secondary peak, and will be designated the minor peak in the following. The same validity tests are applied to it as are described above for the major peak.

6 The two peaks are now further tested in relation to one another. These tests may result in a hitherto valid peak being flagged as invalid. In detail the tests are as set out in the detailed description of the algorithm below, but the main new element here is to use the averaged 12 micron brightness temperatures in the peak to refine the discrimination; if the mean brightness temperature for the rightmost peak exceeds that for the other by more than a threshold MAX_DIF_PEAK_CHAN_1, the left-hand peak is flagged as invalid. At the same time, if the peaks overlap, the upper limit of the left-most peak is refined by extrapolation, and the lower limit of the rightmost peak (which may for the basis of the threshold determination) is then adjusted upwards to match the extrapolated limit. (This makes for a more conservative detection.)

7 The difference threshold is now set. If only one peak is valid, the difference threshold is set to the lower limit of the valid peak. If both peaks are flagged as valid, that at the lower abscissa is selected and its lower limit is used. If neither peak is valid, all pixels are flagged as cloudy.

8 The correlation property described in paragraph 4 of section (2.6.1.1.10.8) above is used to validate and, in the case of night-time scenes, refine the threshold just derived. The average 12 micron brightness temperature associated with each histogram box is calculated, using

Tmean(k) = sigma (T 12)/h(k), where the sum is over all pixels contributing to box k. (The calculation uses the histogram associated with the chosen peak; thus if the minor peak was valid and selected, the histogram after removal of the major peak is used.) The histogram bin corresponding to the maximum brightness temperature is identified, and the slope of the averaged brightness temperature with Bt difference is computed. If this slope exceeds a value SLOPE_MAX_ALLOWED, all pixels are flagged as cloudy.

If the histogram represents a night time scene, the average 12 micron BT are used to refine the threshold just derived. If the position of the chosen histogram peak is to the right of the BT maximum, the threshold is increased until the histogram value at the threshold exceeds a given fraction (1/RATIO_B) of that at the position of the maximum BT. If the position of the chosen histogram peak is to the left of the BT maximum, the threshold is increased until either the histogram peak is reached, or a bin is found at which the 12 micron average exceeds the highest average BT less MAX_DIF_AVE_CHAN - 1 and the slope is lower than SLOPE_MAX_ALLOWED.

9 Finally, all so far clear sea pixels having a brightness temperature difference less than the threshold are flagged as cloudy. If the number of clear pixels that remain is less than the threshold MIN_FOR_11_12_HISTOGRAM, these pixels are also flagged as cloudy.

The detailed algorithm follows.

a) The following steps (b) to (m) are applied to the nadir and forward views separately. Some of the cloud detection parameters provided in the auxiliary file of Cloud Look-up Table Data ATS_CL1_AX differ for the two views, and where applicable those appropriate to the view being processed should be used.

b) The 11um minus 12um brightness temperature differences (T 11 - T 12) are calculated for those pixels that have not been flagged by any of the previous cloud tests and are not over land. The histogram of these differences is generated with a bin size of 0.1 K. For each histogram bin the cumulative brightness temperature in the 12 micron channel is calculated for all the pixels that contribute to each bin.

c) If the number of pixels contributing to the histogram is less than the auxiliary value MIN_FOR_11_12_HISTOGRAM, all the pixels are flagged as cloud no further processing is applied to this view. (If the scene is less than 512 rows in length, the above threshold is scaled in proportion.)

d) The position of the histogram peak, and the histogram value at the peak, are determined. If possible, the position of the peak of the distribution is derived to sub-box accuracy by fitting a quadratic function to the histogram values at the peak and in the two adjacent boxes, and computing the position and ordinate of the maximum of the quadratic. The peak determined at this step will be termed in the following the major peak.

e) The extent of the peak is determined by finding the position of a local minimum or of an empty histogram box on each side of the peak.

e.1) Starting at the histogram peak and working to the left, inspect each histogram bin until either a bin value of zero or a local minimum is found.

e.2) Check if possible that the trough found is genuine by testing that the histogram value at the local minimum found in e.1 is greater than 10% of the peak value and that the value in the second bin from the minimum in the direction away from the peak is not lower than the local minimum value. If neither of these conditions is true then the search is continued as in e.1. (This step is done only once.)

e.3) The higher limit is found in a similar way but working to the right of the peak (to higher bin numbers).

Note: It is possible, although unlikely, that an upper or lower limit cannot be found at this step. In this case the present implementation will terminate the processing if the current view here.

f) If the number of pixels contributing to the histogram peak is less than the auxiliary value IR_PEAK_MIN, the brightness temperature difference at the histogram peak is less than zero, and the absolute latitude of the image centre is greater than 40 degrees, then the peak is flagged as INVALID. If any of these three conditions do not apply, the peak is flagged as VALID.

Note: the current value of IR_PEAK_MIN is 25000. If the peak value of the histogram is greater than this, the peak will be flagged as VALID.

g) The average 12um brightness temperature for the pixels that fall in the histogram bin containing the most pixels (the histogram peak) is computed from the cumulative sum calculated at step (b).

h) If the major peak is VALID, and the solar elevation at the centre of the scene is less than 5 degrees, so that the histogram represents a night-time scene, the following tests are applied to verify that the histogram is not the sum of a clear and a cloudy histogram and hence excessively broad.

h.1) The width of the histogram at half-height is computed to sub-box accuracy by interpolation, using the two values bracketing the 50% peak values.

h.2) A half-width threshold is calculated from the auxiliary parameters appropriate to the view. These parameters define the threshold as a linear function of the (11um - 12um) difference value at the histogram peak.

h.3) If the half width found at (h.1) exceeds than half-width threshold from (h.2) the peak is flagged INVALID.

i) A new histogram is generated by setting to zero the histogram values in the open interval between the upper and lower limits of the major peak. The peak of this histogram, which is a secondary peak of the original histogram, and which in the following will be called the minor peak, is found and tested for validity as follows. If the number of pixels contributing to the new histogram is less than threshold used in Step (c), the minor peak is flagged as INVALID. Otherwise:

i.1) Steps (d) to (h) are repeated to determine the position of the minor peak and its limits.
If the histogram peak is at a difference value less than zero, and the (absolute) latitude of the image centre is greater than 40 degrees, then flag the minor peak as INVALID.

i.2) The minor peak is flagged as INVALID if any of the following conditions apply:

i.2.1) The difference between the higher and lower limits is 20 cK or less;

i.2.2) The major peak is highly likely to be cloudy, and the minor peak is not sufficiently distinct from the major peak:
- The box number of the minor peak is at a greater difference value
than that of the major peak, AND
- The average 12um BT at the minor peak is greater than that
at the major peak by more than threshold MAX_DIF_PEAK_CHAN_1,
AND
- the lower limit of the minor histogram is less than the higher limit of the major histogram, AND
- the value of the major peak at its higher limit is greater than half the minor peak

i.2.3) The minor peak is highly likely to be cloudy:
- The box number of the minor peak is smaller than that of the major peak, AND
- The average 12um BT at the minor peak is less than that at the major peak by more than threshold MAX_DIF_PEAK_CHAN_1

i.3) It is possible that there are no empty histogram boxes between the minor and major histogram peaks. In this case, if the major peak is likely to be cloudy, an extrapolation is used to find the intersect of the falling edge of the major peak and the x axis to refine the limits.

(i.3.1) If the minor histogram is valid and to the right of the major, the higher limit of the major histogram is adjusted by extrapolation:

(i.3.1.1) If the lower limit of the minor histogram is less than the higher limit of the major, then it is reset to the higher limit of the major histogram.

i.3.1.2) If the difference between the box numbers for the major peak and the high interval for the major peak is greater than 3 then extrapolate using the major peak box number and value, and the box number and value of the box at (high interval minus 1).

i.3.1.3) If the difference between the box numbers for the major peak and the high interval for the major peak is not greater than 3 then extrapolate using the major peak box number and value, and the box number and value of the box at high interval.

(i.3.1.4) If the lower limit of the minor histogram is less than the adjusted higher limit of the major, then it is reset to the higher limit of the major histogram. If this adjustment results in a lower limit that is not less than the minor peak position, the lower limit is further reset to the peak position - 1.

i.4) If the minor peak is at a higher difference value than the major peak, AND its average 12um brightness temperature is greater by threshold MAX_DIF_PEAK_CHAN_1 than that of the major peak, then flag the major peak as INVALID.

(i.4.1) If the minor peak is invalid and is to the left of the major histogram

(i.4.1.1) Re-adjust by crude extrapolation the minor high limit using the values of the histogram mode and the bin second to the left of the old limit

(i.4.1.2) Re-adjust by crude extrapolation the minor high limit using the values of the histogram mode and the bin immediately adjacent to the old limit

(i.5) If the minor peak is to the right of the major peak and its average brightness temperature is at a certain value greater than the average brightness temperature of the major peak, then flag the major peak as invalid

j) Choose between the lower limits of the major and the minor peaks, if necessary:
If both peaks are VALID then choose the lower value.
If there is only one VALID peak then choose its lower limit.
If there is no VALID peak then flag all pixels as cloudy and exit.

k) If the lower limit was successfully determined in (i) then check the way the 12um brightness temperature varies with the (11um minus 12um) difference. Under normal conditions, it decreases as the difference increases.

k.1) Average the 12um brightness temperature values for the pixels in each
(11um minus 12um) difference histogram box

k.2) Find the (11um minus 12um) difference value that has the highest average 12um brightness temperature.

k.3) Compute the slope of the (average 12um BT) with respect to the (11um minus 12um) difference, at this point.

k.4) If the slope exceeds threshold SLOPE_MAX_ALLOWED then flag all pixels as cloudy, and exit.

k.5) At night, use the 12um average BTs to tighten the lower limit, if necessary.

k.5.1) If the peak position from the (11um minus 12um) histogram is at a higher difference value than the difference value from k.2 then, starting with the lower limit calculated above, increase its value until the histogram reaches a certain fraction (1/RATIO_B) of the peak value.

k.5.2) If the peak position from the (11um minus 12um) histogram is at a lower difference value than the difference value from k.2 then

k.5.2.1) Generate slopes of delta(12um BT) / delta(11um-12um BT) for each non-empty histogram box

k.5.2.2) Starting with the lower limit calculated above, increase its value until, at lower limit, the (12um average plus threshold MAX_DIF_AVE_CHAN_1) value exceeds the highest average 12um brightness temperature determined in (k.2), and the value of the slope here is less than threshold SLOPE_MAX_ALLOWED

l) Flag as cloudy all formerly clear pixels with a (11um minus 12um) difference value less than the value of lower limit, determined above.

m) If the number of clear pixels remaining is less than a threshold, then flag all the remaining clear pixels as cloudy.

m.1) Calculate the number of clear) pixels remaining;

m.2) If the total is less than threshold value MIN_FOR_11_12_HISTOGRAM then flag all clear pixels as cloudy.

2.6.1.2 ATS_AST_BP

2.6.1.2.1 Physical Justification

The AATSR BROWSE product is a false-colour image generated from the AATSR data, and is intended to provide the user with a quick look view of the image contents. The browse product shows a sub-sampled image at 4 km resolution; the sub-sampling scheme consists of selecting every fourth pixel on every fourth image row.

Two different schemes are used to define the browse product. In the illuminated part of the orbit a coding scheme is used to derive a 3-colour composite image from the nadir view brightness temperature and reflectance images from the GBTR product. For the night-time arc the 11 µm channel brightness temperatures are used to produce a monochromatic infra-red image. Transition processing applied to the day time data at low solar elevations minimises the visual discontinuity at the transition between these forms.

2.6.1.2.1.1 Day-time browse product

Several different channel combinations might be used to generate the daytime browse product, but for the sake of compatibility with existing systems run at ESRIN, the scheme used in the IONIA AVHRR browser described by Melinotte and Arino (1995) has been adopted.

This scheme generates a three-colour composite of two AATSR visible channels and one infra-red channel; the red channel is derived from the 0.67 µm channel, the green channel from the 0.87 µm channel, and the blue band is the inverted 11 µm channel (i.e. light is cold and dark is hot). These data are translated via look-up tables (one for each colour) to corresponding values of red, green and blue (RGB). In the IONIA scheme these LUTS have been chosen to enhance certain features of the data. They are basically a series of linear ramps between specified knots. Each colour table is interpolated between the knots (so that there may be discontinuous changes of gradient across the knots).

2.6.1.2.1.2 Night-time browse product

The night-time browse product is based on the AATSR 11 µm channel. The same look-up table is used to generate a grey-scale image from the single infra-red channel.

2.6.1.2.1.3 Transition processing

The day-time browse product is a false colour image, but the night-time processing generates a grey-scale image. The transition between day-time and night-time processing occurs at the point at which the solar elevation at the centre of the image scan is 5 degrees, that is, the day-time algorithm is only applied to scans for which the solar elevation at nadir exceeds 5 degrees. In order to reduce the visual discontinuity between the two types of image, transition processing is applied to scans at solar elevations between 5 and 6 degrees. The RGB colour triplet is transformed into a different colour representation, the HSV (Hue, Saturation, Value) model. The saturation is scaled by a factor that varies linearly with solar elevation from unity at 6 degrees to zero at 5 degrees (the transition point), and the HSV colour triplet so modified is transformed back into the RGB colour space.

The effect is to reduce the saturation of the day-time image as the day-night transition is approached, so that at the transition point the saturation is zero and the image is monochromatic.

2.6.1.2.2 Algorithm Description

The nadir view data is extracted from each granule in turn until the complete orbit has been processed. The steps described below are applied to each granule.

2.6.1.2.2.1 Image sub-sampling

The browse product is generated from a sub-sampled image obtained by selecting every 4th pixel from the nadir image from every 4th image record in granule. In the following we denote the image pixel value (brightness temperature or reflectance) in each wavelength channel by I(ch , view; i´, j´), where the codes ch and view indicate the wavelength channel and view respectively, and ; i´, , index the image pixels within the granule: i´ ∈{0, 1, ... 31} indexes the image rows counting from the first row of the granule, and ∈{0, 1, ... 511} indexes the pixels within each row.

Thus the sub-sampled image arrays are

IB[ir11; i, j] = I(ir11, n; 4i, 4j)                       }
IB[v670; i, j] = I(v670, n; 4i, 4j)                   }                         
IB[v870; i, j] = I(v870, n; 4i, 4j)
eq 2.143
                   

for i ∈ {0, 1, ... 7} and j ∈  {0, 1, ... 127}. (Note: n indicates nadir view.)

2.6.1.2.2.2 Day time processing

Day time processing generates a false colour composite from 2 visible channels plus one infrared. This processing is applied to each scan of the sub-sampled image for which the solar elevation at centre of the scan is ≥ 5.0 degrees.

2.6.1.2.2.2.1 Generation of RGB Values

Initial red, green and blue channel intensities are derived from the 0.67 µm reflectance, 0.87 µm reflectance and 11 µm brightness temperature respectively, using the pre-defined reflectance/temperature lookup tables from the auxiliary Browse Daytime Colour LUT GADS 6.6.14. .

R[i, j] = Fred {IB[v670; i, j]}                            }
G[i, j] = Fgreen {IB[v870; i, j]}                         }
B[i, j] = Fblue {IB[ir11; i, j]}                            }
eq 2.144

Here the functions Fred , Fgreen and Fblue are the colour conversion functions defined by the lookup tables. For each colour (e.g. red) the look-up table provides a set of N_<colour> coefficient pairs (knots) as follows:

V_ref(index) = reference brightness temperature/reflectance, as appropriate;

coeff(index) = corresponding colour value;eq 2.145

for i = 0, N_<colour> - 1.

The functions Fred , Fgreen and Fblue are defined by piecewise linear interpolation between the knots in the corresponding tables; interpolated values are rounded to nearest integer. If the input value V is out of range of the table, the result is set to

coeff(0) if VV_ref(0)eq 2.146

and to

coeff(N_<colour>-1) if V > V_ref(N_<colour>-1) eq 2.147

Note that the blue conversion table applied to the 11 µm channel includes an inversion; the LUT gives light for T_MIN and dark for T_MAX.

These calculations are usually performed only if the three channel values are valid. However, since saturation of the 11 micron channel is relatively common in day-time data over land, the processing is modified in this case. If the 11 micron channel is saturated, the blue channel value is set equal to zero (B[i, j] = 0), while the red and green values are calculated as above.

2.6.1.2.2.3 Night-time processing

Night time processing is based on use of the 11 µm channel information. It is applied to each scan of the sub-sampled image for which the solar elevation at the centre of the scan is < 5.0 degrees

Provided it is valid, the 11 micron is converted into a colour value using the same look-up table Fblue as is used for the same channel in daytime processing, and the red, green and blue channels are each set to this value.Thus

R[i,j] = G[i,j] = B[i,j] = Fblue{IB[ir11;i,j]} eq 2.148

The result is a monochromatic image in which the temperature values are inverted, so low temperatures appear in light tones.

2.6.1.2.2.4 Transition processing

In order to reduce the visible discontinuity between the day and night time images, transition processing is applied to scans having solar elevations between 5 and 6 degrees. This processing comprises transforming the RGB colour triplet into a different colour model; the HSV (Hue, Saturation, Value) coding. The saturation is multiplied by a scale factor that varies linearly with solar elevation from 1 at 6 degrees to 0 at the transition point of 5 degrees, and the HSV colour triplet so modified is transformed back into RGB colour space.

The three colour values R, G, B can be regarded as the co-ordinates of a point in a three-dimensional colour space. As with any co-ordinate system, these co-ordinates can be transformed into other representations, representing different colour models. One such representation is the HSV (Hue, Saturation, Value) model.

2.6.1.2.2.4.1 The RGB to HSV transformation

The transformation from the RGB to the HSV representation is as follows. (Reference: Foley, J.D., van Dam, A., Feiner, S.K. and Hughes, J.F. Computer Graphics, Principles and Practice, 2nd Edition, Addison-Wesley, 1990.) The RGB colour values, normalised to lie in the range (0, 1), are represented by r, g, b. Let the maximum and minimum values from the triplet r, g, b be represented by max and min respectively. Then

v = max      }
s = (max - min)/max if max ≠ 0}
s = 0 if max = 0 }
eq 2.149

Note that the saturation s is zero if and only if all three colour values r, g, b are the same. The hue h is undefined if s = 0 , otherwise its value depends on which of the three values r, g, b is the greatest.

delta = max - min eq 2.150

h = 60 x (g - b)/delta if (r = max) }
h = 60 x (2 + (b - r)/delta) if (g = max) }
h = 60 x (4 + (r - g)/delta) if (b = max) }
eq 2.151

The hue h is considered to be an angular co-ordinate representing the angular position of the colour in the conventional colour circle, and the multiplication by 60 converts h into units of degrees. If h as given by the above equations is negative, it is converted to a positive number in the range 0 to 360 by the addition of 360.

2.6.1.2.2.4.2 The HSV to RGB transformation

The inverse transformation from the HSV to the RGB representation is as follows. (Reference: Foley, J.D., van Dam, A., Feiner, S.K. and Hughes, J.F. Computer Graphics, Principles and Practice, 2nd Edition, Addison-Wesley, 1990.)

If s = 0 then

r=g=b = v. eq 2.152

Otherwise the transformation depends on which 60 degree sector h lies in. Let i be the largest integer less than or equal to (h/60). Then

f = h - i eq 2.153

p = v x (1 - s) eq 2.154

q = v x (1 - (s x f)) eq 2.155

t = v x (1 - (s x (1 - f))) eq 2.156

(r, g, b) = (v, t, p) if i = 0 eq 2.157

(r, g, b) = (q, v, p) if i = 0 eq 2.158

(r, g, b) = (p, v, t) if i = 0 eq 2.159

(r, g, b) = (p, q, v) if i = 0 eq 2.160

(r, g, b) = (t, p, v) if i = 0 eq 2.161

(r, g, b) = (v, p, q) if i = 0 eq 2.162

Note that although for clarity we have presented the transformations in terms of normalised colour values, the values s and h are calculated as ratios, and so the transformation remains reversible under an arbitrary positive scaling of the RGB values. Thus it is not actually necessary to rescale the values in the implementation, even though the IONIA colour tables give values in the range 0 to 255.

2.6.1.2.2.4.3 Pixel processing in the transition region

The transition processing is then as follows. For each scan for which the solar elevation ε at the mid-point lies in the range 5 ≤ ε ≤ 6 degrees, the pixel RGB colour values from the day-time processing are converted to the HSV system using the above transformation.

The weight factor

w = ε - 5, eq 2.163

where (0 ≤ w ≤ 1), is calculated, and the saturation is scaled using

= w x s. eq 2.164

The triplet (h, , v) is transformed back to the RGB system as above, and the resulting RGB colour triplet for each pixel represents its value in the browse image.


Keywords: ESA European Space Agency - Agence spatiale europeenne, observation de la terre, earth observation, satellite remote sensing, teledetection, geophysique, altimetrie, radar, chimique atmospherique, geophysics, altimetry, radar, atmospheric chemistry