| Both sides previous revision
Previous revision
Next revision
|
Previous revision
Next revision
Both sides next revision
|
user_area:generating_lookup_tables [2013/10/21 16:33]
josef |
user_area:generating_lookup_tables [2013/10/22 13:39]
josef |
| It is sometimes required to generate lookup tables which, for example, contain radiances as function of one or more variable atmospheric parameters. Besides the wavelength loop, there is no looping over input parameters implemented in ''uvspec'', so that ''uvspec'' can not create lookup tables on its own. However, several scripting languages are available that can used to build looping-capable scripts around ''uvspec''. These scripts repeatedly call ''uvspec'' with varying input parameters and use the ''uvspec'' results to build up a lookup table. | It is sometimes required to generate lookup tables which, for example, contain radiances as function of one or more variable atmospheric parameters. Besides the wavelength loop, there is no looping over input parameters implemented in ''uvspec'', so that ''uvspec'' can not create lookup tables on its own. However, several scripting languages are available that can used to build looping-capable scripts around ''uvspec''. These scripts repeatedly call ''uvspec'' with varying input parameters and use the ''uvspec'' results to build up a lookup table. |
| |
| In the following we show and explain example scripts coded in [[http://en.wikipedia.org/wiki/Bash_%28Unix_shell%29|Bash]]. The scripts generate lookup tables with simulated radiances for a satellite channel in the solar spectral range as function of one and two variable atmospheric parameters. These scripts create lookup tables in ASCII format. The scripts can be adapted, e.g. for other geometries or other sets of variable parameters etc. | In the following we show and explain example scripts coded in [[http://en.wikipedia.org/wiki/Bash_%28Unix_shell%29|Bash]]. The scripts generate lookup tables with simulated radiances for a satellite channel in the solar spectral range as function of one and two variable atmospheric parameters. The lookup tables are in ASCII format. The provided scripts can be adapted, e.g. for other setups or other sets of variable parameters etc. |
| |
| |
| ====== Script for one variable parameter ====== | ====== Script for one variable parameter ====== |
| First we describe the approach for looping over one variable input parameter. An example script with some required additional files is available {{:user_area:generating_lookup_tables:script_1para.tar.gz|here}}. (This example is for libRadtran version > 1.7) | First we describe an approach for looping over one variable input parameter. An example script with some required additional files is available {{:user_area:generating_lookup_tables:script_1para.tar.gz|here}}. (This example is for libRadtran version > 1.7) |
| |
| Basically, there are three steps required for creating lookup tables using ''uvspec'': | Basically, there are three steps required for creating lookup tables using ''uvspec'': |
| You can create a template for the ''uvspec'' input file like you would do for any other ''uvspec'' input file, e.g. starting from an example in the ''examples'' folder or from the GUI. The main difference is that you have to replace the parameter you want to vary by a placeholder. In the provided example, we vary the aerosol optical thickness, thus the template file ''uvspec.inp.template_1para'' contains the following line | You can create a template for the ''uvspec'' input file like you would do for any other ''uvspec'' input file, e.g. starting from an example in the ''examples'' folder or from the GUI. The main difference is that you have to replace the parameter you want to vary by a placeholder. In the provided example, we vary the aerosol optical thickness, thus the template file ''uvspec.inp.template_1para'' contains the following line |
| aerosol_modify tau set <AOT> | aerosol_modify tau set <AOT> |
| with the placeholder ''<AOT>'' for the variable aerosol optical thickness. In principle, a placeholder could be any unambiguous string, but for clarity we prefer using placeholders starting with "<" and ending with ">". Note, that all other input options are fixed, so the lookup table is created for this specific setup. | with the placeholder ''<AOT>'' for the variable aerosol optical thickness. In principle, a placeholder could be any unambiguous string, but for clarity we prefer using placeholders starting with "<" and ending with ">". Note, that all other options are fixed in ''uvspec.inp.template_1para'', with the consequence that the lookup table is created for this specific setup. |
| |
| For easier processing of the results it is advisable to use the input option | For easier processing of the radiance output it is advisable to use the input option |
| output_user uu | output_user uu |
| With this option, uvspec outputs only the radiances at the viewing angles, specified by the ''umu'' and ''phi'' input options. | With this option, ''uvspec'' outputs only the radiances at the viewing angles, that are specified by the ''umu'' and ''phi'' input options. |
| |
| ==== Step 2: Create a script ==== | ==== Step 2: Create a script ==== |
| After the input template has been created, the looping script needs to be adapted. In our example, the script is the file ''batch_uvspec_1para''. | After the input template has been created, a looping script needs to be created. In our example, the script file ''batch_uvspec_1para'' is provided. |
| |
| The list of the variable input parameters needs to be created. This list is stored in the variable ''parameter_list'' in our example. There are different possibilities to create such list. For example, it can be read from the external file ''list_aot'' using | The list of the variable input parameter values needs to be created. This list is stored in the variable ''parameter_list'' in our example. There are different possibilities to create such a list. For example, it can be read from an external file ''list_aot'' using |
| parameter_list=`cat list_aot` | parameter_list=`cat list_aot` |
| The ''`command`'' syntax in Bash is used to call an external command, whereby the output from the external command is redirected to the script. | The ''`command`'' syntax in Bash is used to call an external command, whereby the output from the external command is redirected to the script. |
| | Here, the output from the ''cat'' command, which outputs the content of the file ''list_aot'', is redirected into the variable ''parameter_list''. |
| As an alternative, the parameter values may be directly entered with | As an alternative, the parameter values may be directly entered with |
| parameter_list="0.0 0.1 0.2 0.5 1.0" | parameter_list="0.0 0.1 0.2 0.5 1.0" |
| sed -i s/\<${parameter_name}\>/${parameter}/g ${uvspec_input_file} | sed -i s/\<${parameter_name}\>/${parameter}/g ${uvspec_input_file} |
| echo ${parameter} `${uvspec_command} < ${uvspec_input_file}` >> ${output_file} | echo ${parameter} `${uvspec_command} < ${uvspec_input_file}` >> ${output_file} |
| is executed. First it copies the template input file to the actual temporary input file. | is executed. The first command copies the template input file to the actual temporary input file. |
| The second command replaces the placeholder by the actual input parameter value using the external ''sed'' tool. The ''sed'' syntax for the replacement of a string in a file is "''sed -i s/search_string/replacement/g file''". As the "<" and ">" characters have a special meaning in Bash, they require a preceding "\" here. | The second command replaces the placeholder by the actual input parameter value using the external ''sed'' tool. The ''sed'' syntax for the replacement of ''search_string'' in a file by a ''replacement'' is "''sed -i s/search_string/replacement/g filename''". As the "<" and ">" characters have a special meaning in Bash, they require a preceding "\" here. |
| In the third line the ''uvspec'' tool is executed as an external command and its output is directed, together with the parameter value, to the lookup table file. | In the third line the ''uvspec'' tool is executed as an external command and its output is directed, together with the parameter value, to the lookup table file. |
| |
| ==== Step 3: Testing and running ==== | ==== Step 3: Testing and running ==== |
| The script can be started now. The script in our example is executed on the command line by | The script can be started now. The script in our example is executed on the command line with |
| |
| ./batch_uvspec_1para | ./batch_uvspec_1para |
| * add the execute flag to the script with "''chmod +x batch_uvspec_1para''" | * add the execute flag to the script with "''chmod +x batch_uvspec_1para''" |
| * enter the correct path to the ''uvspec'' executable in ''batch_uvspec_1para'' | * enter the correct path to the ''uvspec'' executable in ''batch_uvspec_1para'' |
| * enter the correct path to the libRadtran data directory in ''uvspec.inp.template_1para'' | * enter the correct ''data_files_path'' in ''uvspec.inp.template_1para'' |
| |
| If your script ran without complains check out the contents of the lookup table. It is named ''lookup_table_1para.txt'' in our example. | If your script ran without complains check out the contents of the lookup table. It is named ''lookup_table_1para.txt'' in our example. |
| The provided bash script adds a header line containing the viewing directions for each column. | The provided bash script has added a header line containing the viewing directions for each column. |
| The lookup table should look like this: | The lookup table should look like this: |
| # AOT, umu 0.2 phi 0, umu 0.2 phi 60, umu 0.2 phi 180, umu 0.5 phi 0, umu 0.5 phi 60, umu 0.5 phi 180 | # AOT, umu 0.2 phi 0, umu 0.2 phi 60, umu 0.2 phi 180, umu 0.5 phi 0, umu 0.5 phi 60, umu 0.5 phi 180 |
| 0.002 4.474126053e+01 3.749518204e+01 4.862616730e+01 3.000129890e+01 2.839014435e+01 3.465411758e+01 | 0.002 4.474126053e+01 3.749518204e+01 4.862616730e+01 3.000129890e+01 2.839014435e+01 3.465411758e+01 |
| ... | ... |
| There is a plotting script (requiring Python Matplotlib) provided in the example and called at the end of the Bash script. It plots the radiance as function of the variable parameter for the specified viewing directions. The AOT-dependence for the provided example is shown here: | There is a plotting script (requiring Python Matplotlib) provided in the example and called at the end of the Bash script. It plots the radiances as function of the variable parameter for the viewing directions specified in the template file. The AOT-dependence for the provided example is shown here: |
| |
| {{:user_area:generating_lookup_tables:plot_1para.png?800|}} | {{:user_area:generating_lookup_tables:plot_1para.png?800|}} |
| | |
| | This plot is for a satellite channel around 680nm (~10nm width), ''albedo 0.1'', ''sza 60'', ''phi0 0'' with OPAC desert aerosol. The radiances are given in mW/m^2/sr integrated over the channel response function. |
| |
| ====== Script for two variable parameter ====== | ====== Script for two variable parameter ====== |
| The script described above can be modified for processing more variable input parameters. In the following we briefly show how to extend it by a second variable parameter; the resultant data files are available {{:user_area:generating_lookup_tables:script_2para.tar.gz|here}}. | The script described above can be modified for processing more variable input parameters. In the following we briefly show how to extend it by a second variable parameter; the modified files are available {{:user_area:generating_lookup_tables:script_2para.tar.gz|here}}. |
| |
| We select the sun zenith angle as the second variable input parameter. So we have to replace the fixed sun zenith angle in the template by a placeholder: | We select the sun zenith angle as the second variable input parameter. So we have to replace the fixed sun zenith angle in the template by a placeholder: |
| sza <SZA> | sza <SZA> |
| |
| In the Bash script a further list of parameters is added. The two lists are now named ''parameter1_list'' and ''parameter2_list''. A for-loop for the solar zenith angle is added also. The ''sed'' tool needs to be called twice for each set of input parameters to replace both placeholders from the template. We have added the second parameter to the output, so that the second column in the lookup table now is the solar zenith angle; the file looks like this: | In the Bash script a further list of parameters is added. The two variables with the list of parameter values are now named ''parameter1_list'' and ''parameter2_list''. ''parameter2_name'' is now SZA. A for-loop for the solar zenith angle is added also. The ''sed'' tool needs to be called twice for each set of input parameters to replace both placeholders by the actual values. We have added the second parameter to the output, so that the second column in the lookup table now is the solar zenith angle; the file should look like this: |
| # AOT, SZA, umu 0.2 phi 0, umu 0.2 phi 60, umu 0.2 phi 180, umu 0.5 phi 0, umu 0.5 phi 60, umu 0.5 phi 180 | # AOT, SZA, umu 0.2 phi 0, umu 0.2 phi 60, umu 0.2 phi 180, umu 0.5 phi 0, umu 0.5 phi 60, umu 0.5 phi 180 |
| 0.00 0 5.674485779e+01 5.674485779e+01 5.674485779e+01 5.222948837e+01 5.222948837e+01 5.222948837e+01 | 0.00 0 5.674485779e+01 5.674485779e+01 5.674485779e+01 5.222948837e+01 5.222948837e+01 5.222948837e+01 |
| 0.00 4 5.631779480e+01 5.646367645e+01 5.710541916e+01 5.175418854e+01 5.192774200e+01 5.251840973e+01 | 0.00 4 5.631779480e+01 5.646367645e+01 5.710541916e+01 5.175418854e+01 5.192774200e+01 5.251840973e+01 |
| ... | ... |
| There is also a plotting script provided to illustrate the lookup table for two variable parameters. In the provided example the result looks like this: | There is also a plotting script provided to illustrate the radiances in the lookup table for two variable parameters. In the provided example the result looks like this (note the different color scales): |
| {{:user_area:generating_lookup_tables:plot_2para.png?1000|}} | {{:user_area:generating_lookup_tables:plot_2para.png?1000|}} |
| |
| Further parameters could be added analogously by modifying the template and script. | Additional variable parameters could be added analogously by modifying the template and script. (Plotting for more than two variable parameters however may not be as straightforward ;-)) |
