***************************************************************************** * You can find a PDF version of this file from * * http://hdfeos.org/software/aug_eos5/Usage_Of_Aug_Tool.pdf * ***************************************************************************** Contents 1. Introduction of Augmentation Tool 2. Description of Mapping File 3. How to Augment an HDF-EOS5 Grid File 3.1. Overview 3.1.1. Default Option 3.1.2. Index Option 3.1.3. File Option 3.2. Examples 3.2.1. Single Grid 3.2.1.1. Default Option 3.2.1.2. Index Option 3.2.1.3. File Option 3.2.2. Multiple Grids 4. How to Augment an HDF-EOS5 Swath/Zonal Average File 4.1. Overview 4.1.1. Default Option 4.1.2. Index Option 4.1.3. File Option 4.2. Examples 4.2.1. Single Swath/Zonal Average 4.2.1.1. Default Option 4.2.1.2. Index Option 4.2.1.3. File Option 4.2.2. Multiple Swaths/Zonal Averages 1. Introduction of Augmentation Tool This command-line tool can be used to augment an HDF-EOS5 file so that the netCDF-4 APIs can read the augmented file. The HDF-EOS5 objects we support are swath, grid and zonal average. In order to accommodate as many as diversified users, this tool provides three different options: Default Option, File Option and Index Option. Users can specify any of them by providing command-line parameters. Default Option augments a grid file by calculating latitude and longitude of this file. This option is useful for grid files that only have 2-dimensional data fields in a horizontal plane (See section 3.1.1). This option may also benefit users who do not have detailed knowledge of their files (See section 4.1.1). It is easier to manipulate, but the downside is that a user has no control over the augmentation process. So we only recommend this option for the users who do not have detailed knowledge of their files. File Option relies on the external information a user provides. It should apply to most users (See section 3.1.3 and 4.1.3). This option is especially useful for the users who have detailed knowledge of their files. Although users need to provide additional information, they have more comprehensive control over the augmentation process and thus the augmented file should meet users' expectation. So we strongly recommend users to use this option if possible. Index Option provides an alternative solution for users who cannot use File Option due to the lack of the detailed knowledge of their files or do not want to use Default Option (See section 3.1.2 and 4.1.2). Generally speaking, File Option can meet the vast majority of users' needs. However, a user needs to provide a mapping file to fulfill the augmentation process for this option. Therefore we first introduce the concept of the mapping file. Since Augmentation Tool handles an HDF-EOS5 zonal average using the same approach as an HDF-EOS5 swath, we merge the discussion of the zonal average with the swath. So the rest discussion on Augmentation Tool is divided into two parts: grid and swath/zonal average. We first discuss grid, then swath/zonal average. Each part starts with a brief introduction, followed by examples. All examples can be found under ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NASAHDFTOOLfiles/aug_eos5/sample_files_for_doc/. 2. Description of a Mapping File The mapping file plays an important role in the augmentation process for File Option. It consists of information on how to create the HDF5 dimension scales. For a single HDF-EOS5 grid or swath or zonal average, a mapping file is arranged into three columns separated by spaces as follows: Column 1 Column 2 Column 3 0|1|2 || Table 1: Structure of a mapping file For Column 1, a user should fill in number 0, 1, or 2. For Column 2, a user should provide a valid dimension name obtained from the original HDF-EOS5 file based on the user’s knowledge. For column 3, a user should either provide a valid data field name obtained from the same HDF-EOS5 file or provide the name of an ASCII file that stores the scale of the dimension listed in column 2 or leave it blank. Generally only the following three combinations are allowed in a row for a mapping file. For an HDF-EOS5 grid, if the dimension name is XDim or YDim, only number 1 is allowed for column 1. See details at section 3.2.1.3. Column 1 Column 2 Column 3 0 1 2 Table 2: Combinations of options of a mapping file For all combinations, users should provide a dimension name in column 2. As shown in row 2 of table 2, if the number is 0 for column 1, column 3 should be the data field that stores the scale of the dimension listed in column 2. As shown in row 3 of table 2, if the number is 1 for column 1, column 3 should be the name of a file that contains the scale of the dimension listed in column 2. This file is an ASCII file with the values arranged in a single column as follows: ... 11.111 12.22 ... It should be mentioned that the number of values must match the corresponding dimension size; otherwise, the augmentation tool will throw an exception. As shown in row 4 of table 2, if the number is 2 for column 1, column 3 should be blank. The scale of the dimension listed in column 2 is filled with the natural numbers starting from 1. In order to be consistent with the current implementation of netCDF-4, the augmentation tool fills a dimension scale with zeros if it is not listed in column 2. For a file containing multiple HDF-EOS5 objects, each object (grid, swath etc.) name should be provided in a mapping file. The following shows an example of this case. Supposed that an HDF-EOS5 file contains two swaths, the name of the first swath is Swath1 and the name of the second swath is Swath2, the mapping file should be like: swath name: Swath1 information for the augmentation of Swath1 swath name: Swath2 information for the augmentation of Swath2 The information immediately following a swath name is used to augment that swath. Be aware that swath name: must be preceded before the real name of that swath. For a grid, grid name: must be preceded before the real name of the grid. Since a zonal average object is handled similarly as a swath object, so for a zonal average object, swath name: instead of zonal average name: must be preceded before the name of the zonal average object. Finally, it should be mentioned that users do not need to provide any dimension information in a mapping file if that dimension is not used by any data field. Users can add some comments in a mapping file to increase readability. Any line starting with # is considered as comments. The Augmentation Tool ignores such a line. 3. How to Augment an HDF-EOS5 Grid File 3.1 Overview To start with the discussions, let us explain what XDim and YDim are. For an HDF-EOS5 grid file, XDim is the dimension created automatically by the HDF-EOS5 grid interface. It is the dimension of longitude if the projection is geographic. YDim is also created automatically by the HDF-EOS5 grid interface and corresponds to latitude if the projection is geographic. 3.1.1 Default Option At the command prompt, typing the following command: aug_eos5 Augmentation Tool can figure out how to fill the dimension scales of XDim and YDim and add them correspondingly. Other dimension scales are also added, but filled with zeros. 3.1.2 Index Option At the command prompt, typing the following command: aug_eos5 -i All dimension scales are added and filled with the natural numbers except for XDim and YDim. XDim and YDim are handled the same way as Default Option (See 3.1.1). 3.1.3 File Option At the command prompt, typing the following command: aug_eos5 -f Dimension scales are added. XDim and YDim are handled the same way as Default Option (See 3.1.1) if the mapping file doesn't provide any information for XDim and YDim; otherwise they are handled together with other dimensions according to the mapping file. The concept of the mapping file is fully explained in section 2. 3.2 Examples 3.2.1 Single Grid Assume that the single grid file name is single_grid.he5. 3.2.1.1 Default Option At the command prompt, entering the following command to augment single_grid.he5: aug_eos5 single_grid.he5 As stated in 3.1.1, XDim and YDim are filled with the data values calculated through HDF-EOS5 APIs. The other dimension scale is filled with zeros. 3.2.1.2 Index Option At the command prompt, entering the following command to augment single_grid.he5: aug_eos5 -i single_grid.he5 XDim and YDim are handled the same way as Default Option (See 3.1.1). The other dimension scale is filled with the natural numbers. 3.2.1.3 File Option Since this option requires the knowledge of the mapping file, please first check the description of the mapping file in section 2. Constructing the following mapping file (single_grid_mapping_file.txt): 0 ZDim Pressure then entering the following command: aug_eos5 -f single_grid_mapping_file.txt single_grid.he5 XDim and YDim are handled the same way as Default Option (See 3.1.1). ZDim is filled with the values obtained from Pressure instead. Pressure is a data field in the HDF-EOS5 file. Obtaining the dimension scale from a field inside an HDF-EOS5 file is the most common case a user may encounter. For some reason, if any dimension scale needs to be obtained from an outside ASCII file, a user should add the following line for that dimension in a mapping file 1 dim_name data_file data_file is an ASCII file storing the values of a dimension scale field. It should be mentioned that generally the latitude and longitude should be calculated via HDF-EOS5 APIs. Under such circumstance, the mapping file doesn't need to provide any information about XDim and YDim. Be aware that currently the augmentation tool neither supports the retrieval of the latitude and longitude from data fields in an HDF-EOS5 grid file nor supports filling the latitude and longitude with the natural numbers. 3.2.2 Multiple Grids For Default Option and Index Option, the way to use the augmentation tool for multiple grids is the same as that for single grid except that the tool augments each grid independently. One should be aware that the dimension scales of XDim and YDim are created under each grid accordingly. For File Option, according to section 2, the grid name should be provided for each grid in a mapping file. The following example shows how to construct such a file. Assume the file to be augmented is two_grids.he5. It contains two grids, NadirDayGrid and NadirNightGrid. Each of them has a dimension nLevels. Assume users want to fill the corresponding dimension scale with the values from a data field, number 0 should be specified at the first column. The dimension name, nLevels, should be specified at the second column. The data field name, Pressure, should be specified at the third column. The following shows the layout of such a mapping file (two_grids_mapping_file.txt): grid name: NadirDayGrid 0 nLevelsi Pressure grid name:NadirNightGrid 0 nLevels Pressure To augment two_grids.he5, simply entering the following command in a command prompt: aug_eos5 -f two_grids_mapping_file.txt two_grids.he5 4. How to Augment an HDF-EOS5 Swath/Zonal Average File 4.1 Overview Since swath/zonal average files store latitude and longitude in data fields, the special handling of dimensions like XDim and YDim for grids is not needed. 4.1.1 Default Option At the command prompt, typing the following command: aug_eos5 Dimension scales are added and filled with zeros. 4.1.2 Index Option At the command prompt, typing the following command: aug_eos5 -i Dimension scales are added and filled with the natural numbers. 4.1.3 File Option This option should be used for most users. The dimension scales should be obtained from fields in an HDF-EOS5 file if possible. We strongly recommend users to augment the file in this way. At the command prompt, typing the following command: aug_eos5 -f Dimension scales are added and filled with the values obtained according to the given information in the mapping file. The concept of the mapping file is fully explained in section 2. 4.2 Examples 4.2.1 Single Swath/Zonal Average The main example file is single_swath.he5 with two dimensions: NDim and ZDim. 4.2.1.1 Default Option At the command prompt, entering the following command to augment single_swath.he5. aug_eos5 single_swath.he5 All newly added dimension scales, NDim and ZDim, are filled with zeros. 4.2.1.2 Index Option At the command prompt, entering the following command to augment single_swath.he5: aug_eos5 -i single_swath.he5 All newly added dimension scales, NDim and ZDim are filled with the natural numbers. 4.2.1.3 File Option Since this option requires the knowledge of a mapping file, please first check its description in section 2. We hope users to follow this example to augment their files if possible. Constructing the following mapping file (single_swath_mapping_file.txt) first: 0 NDim Latitude 0 ZDim Pressure then entering the following command: aug_eos5 -f single_swath_mapping_file.txt single_swath.he5 Two dimension scale fields, NDim and ZDim, are added. NDim and ZDim are filled with the values obtained from Latitude and Pressure respectively. Latitude and Pressure are data fields in the HDF-EOS5 file. Obtaining the dimension scale from a field inside an HDF-EOS5 file is the most common case a user may encounter. To describe other combinations of this option, we provide another example file single_swath_2.he5. This file has three dimensions: nTimes, nLevels and nCloudTypes. The dimension scales should be obtained from fields in the HDF-EOS5 file if possible. We strongly recommend users to augment the file in this way. Here is a mapping file for this case. 0 nTimes Time 0 nLevels Pressure Entering the following command: aug_eos5 -f single_swath_2_mapping_file.txt single_swath_2.he5 Three dimension scales, nTimes, nLevels and nCloudTypes, are added. nTimes and nLevels are filled with the values obtained from the data fields of Time and Pressure in the HDF-EOS5 file respectively. In this example, there is no information on how to handle nCloudTypes in the mapping file. The augmentation tool currently fills nCloudTypes with zeros in order to be consistent with the current implementation of netCDF-4. If the dimension scale of nCloudTypes should be filled with the natural numbers, a user should add the following line in a mapping file. 2 nCloudTypes For some reason, if any dimension scale needs to be obtained from an outside ASCII file, a user should add the following line for that dimension in a mapping file 1 dim_name data_file data_file is an ASCII file storing the values of a dimension scale field. 4.2.2 Multiple Swaths /Zonal Averages For Default Option and Index Option, the way to use the augmentation tool for multiple swaths/zonal averages is the same as that for single swath/zonal average except the tool augments each swath/zonal average independently. Dimension scales are created under each swath/zonal average accordingly. For File Option, according to section 2, the swath/zonal average name should be provided for each swath/zonal average in a mapping file. The following example shows how to construct such a file. Assume that the file to be augmented is two_swaths.he5. It contains two swaths: Swath1 and Swath2. As we suggested before, the dimension scales should be obtained from fields in an HDF-EOS5 file if possible. The following shows the layout of such mapping file (two_swaths_mapping_file.txt): swath name: Swath1 0 NDim Latitude 0 ZDim Pressure swath name:Swath2 0 NDim Latitude 0 ZDim Pressure To augment two_swaths.he5 with File Option, simply entering the following command in a command prompt: aug_eos5 -f two_swaths_mapping_file.txt two_swaths.he5