Dapper in-situ conventions spec available

[Joe Sirott]

John Caron wrote:
>>>
>>> 3. has_data attribute.  to use the spec effectively, particularly in 
>>> a time series sense, we have found that any parameter in the inner 
>>> sequence show always be there, but often it will not be observed 
>>> while other parameters were.  Rather than having to look at the data 
>>> itself to see if it is totally missing, do we want a "has_data" 
>>> attribute required?
>>
>> Yes, this is an issue that we too have encountered with our Web 
>> interface to Dapper. The problem is that the DDS contains the union 
>> of all of the variables in the dataset, even if a given variable is 
>> only measured in a few time series or profiles. One example,  is the 
>> World Ocean Database 2005. A chlorophyll variable is contained in the 
>> DDS, but very few of the millions of profiles in the database 
>> actually contain measurements for this variable. So any user who is 
>> looking for chlorophyll measurements will have to sift through many 
>> profiles before finding a profile with valid data.
>>
>> Adding a has_data as an optional variable attribute sounds like a 
>> good idea to me.
>
>
> presumably you would want to be able to select on has_data ?
... or any other attributes. Can't do this in current version of Dapper, 
though.

>
> another possibility that ive seen used somewhere is that you show N 
> generic data variables, and add N fields that say which variable this 
> sample is, eg:
>
>   Sequence {
>    time;
>    float sample1, sample2, ...
>    String sample1Name, sample2Name, ...       String unit1, unit2, ...
>   } obs; 
> Better would be if you basically had an enumerated type:
>
>  Structure{
>    int id;
>    String name, long_name, units ...
>   } quantity[nq];    
>   Sequence {
>    time;
>    float sample1, sample2, ...
>    int quantity1, quantity2, ...  // quantity id    } obs; 
> If you can select on the quantityi values, you can winnow out the 
> chlorophyll data.
> I realize this is a bit complicated.
> John
Seems like it would be a lot easier to just tag a variable with an 
attribute ... also would be easier to postprocess a dataset that doesn't 
already provide the information (as is true of most in-situ datasets)...

- Joe