20050922:DODS/THREDDS confusion

[David Wojtowicz]

OK,

 Here's what I have come up with concerning the discussion in this 
thread.   Thanks for considering these potential changes.  A pointer to 
the patch is at the end of this message.

***** Summary of Modified Functionality:

 1) All containerized attributes are now "flattened" into the flat 
namespace.  A colon seperates the container name from the contained 
attribute name.  Since all container attributes are removed by this 
process, the netCDF API no longer chokes from encountering one, and the 
former contents of the containers are now available to access in flat 
namespace if the user cares to do so.

 2) Because containerized attributes containing strings, are not quoted, 
the code that removes the quotes is modified to ignore the case where 
there is no quotes.

 3) In the case of arrays of strings, the array is translated to an 
array of characters of  the next higher rank, with the extra dimension's 
size and name being determine as follows:

 a) If DODS:strlen attribute exists  (or varname:DODS:strlen in the case 
of translated structures)  it supplies the size and DODS:dimName 
supplies the name

 or

 b) otherwise, the STRING_ARRAY_SIZE constant is used as the size and 
varname-chars is used as the dim name.

The extra dimension  is added to the global dimension list (if not 
already existing).   If the name already exists for some reason, but 
with a different size, an alternate name is used.

4) Single scalar strings are treated in a similar manner.


***** Code Changes:

I attempted to make code changes using a similar coding style to the 
source file that was being edited (i.e. using Stroustrup style on files 
written in this style). All changes are in libnc-dap.

NCConnect.cc/.h: 
 
 -- Removed compue_string_attr_length(),  moved it to nc_util.cc

 -- In NCConnect::init_remote_source(), added code to call 
flatten_attributes() for each variable (as well as global ones for which 
it was already being done).

-- added an alternate version of flatten_attributes, that modifies an 
attribute table in place rather than returning a new one for more 
efficient support of above change.

-- changed NCConnect::var_info() to call get_dods_str_dim() (added 
below) to determine added dimension name for string arrays instead of 
just making it name+"-chars".

-- changed NCConnect::parse_array_dims() and 
NCConnect::parse_string_dims()  to use get_dods_str_dim() determine 
added dimension name for string arrays, and to add it to the global 
dimension list via new function NCConenct::add_dim_if_new()

-- added NCConnect::get_dods_str_dim() to centralize code to determine 
extra dimension size and name (as described in functionality section above)

-- added NCConnect::add_dim_if_new() to centralize code to add newly 
created dimensions to global dim list.



nc_util.cc/.h:
 
 -- Moved compute compute_string_attr_length() from NCConnect.cc to 
here, made it available to other classes.   This allows it to be called 
from multiple locations in  NCConnect.cc and  Dattr.cc.   Changed code 
to not anticipate quote removal if string is not quoted.

-- Modified copy_attributes() to use properly copy non-quoted string 
attributes

--  Added alternate version of attributes_list_to_table() to modify a 
pre-existing attribute table in support of alternate 
NCConnect::flatten_attributes()

NCArray.cc: 

-- Removed code in NCArray::extract_values() to deal with strings 
seperately.  Translation code has already made them character arrays.

-- Changed NCArray::flatten() to set translated flag.

NCStr.cc:

--  NCStr::extract_values() now uses determined string length (as found 
earlier in get_dods_str_dim) instead of STRING_ARRAY_LEN.

Dattr.cc:

 -- Replaced two instances of code to compute string length with call to 
compute_string_attr_len() in nc_util.cc

NCAccess.h:

 -- Added provision to store strdim via NCAccess::set_strdim() and 
NCAccess::get_strdim().   This is set in NCConnect::get_dods_str_dim() 
and used in NCStr::extract_values().



***** Testing:

 My primary test vehicle was using dncdump on data from the motherlode 
TDS.   I was not able to get the code in unit_tests to even compile 
(even after making the link as described in README).  

 One case I have not well tested was a scalar string variable.   (I 
don't run a TDS myself and so was not able to generate such a test 
easily.... maybe it is easy, I just haven't looked into the server side 
of things much)

I ran dncdump through valgrind, and did not turn up any errors or memory 
leaks, except those that seem to be a minor problem in ncdump itself.


***** To Do: (that I need help with)

 - Further testing

 - Adding code to deal with constriants placed on variables from the URL 
as described by James previously.


***** Patches

The following collection of patches represent the above changes as 
applied to SVN rev 13358.

 http://www.atmos.uiuc.edu/~davidw/libnc_dap_str_patch.gz


Thanks.

-----
David Wojtowicz, Sr. Research Programmer, Sysadmin
Dept of Atmospheric Sciences / Computer Services
University of Illinois at Urbana-Champaign
davidw at uiuc.edu  (217) 333-8390


>