Sequences, Lists, and Relational Constraint Expressions

[Joe Sirott]

Hi John,

I don't know if there are /any /OpenDap server implementations that 
fully implement the selection constraints as described in the OpenDap 
spec. For instance, if I try and constrain a string variable with a 
netCDF server I get a parse error (although I could be doing something 
wrong). In practice, it's difficult to see how a server /could/ 
practically implement all of the mandated constraints -- imagine having 
to search a database of several million profiles with several 
constraints on variable values. Even though such a server would be 
obeying the "letter of the law," a timeout after five minutes is 
probably worse behavior for a client then an immediate error message.

The practical alternative is to use some convention. Dapper already uses 
an ad-hoc convention that might be useful to you (as Steve mentioned) 
since it does identify the coordinate variables and allows constraints 
on those variables. 


John Caron wrote:
>
>
> Steve Hankin wrote:
>> Hi John,
>>
>> This dilemma is the result of a mismatch between the "discipline 
>> neutral" generality of the OPeNDAP DAP and the relatively focussed 
>> data models that Unidata is working with.  The obvious way to address 
>> this is through conventions -- just as the CF conventions are already 
>> needed in order to map the generality of netCDF onto the Common Data 
>> Model.
>>
>> It seems to me that the DAPPER conventions, if fleshed out, could 
>> address the problem that you pose.  Fully fleshed out DAPPER 
>> conventions would clarify the process for identifying the independent 
>> (coordinate) variables, georeferencing, the requirement (or not) for 
>> a 2-level Sequence, the scope of structures that would be permitted 
>> inside the sequences, the adoption (or not) of CF standard names, 
>> etc.   Through the conventions one could also impose restrictions (or 
>> convey information to clients) on the types of relational operations 
>> that are allowed and/or the variables to which they can be applied.
>
> Hmmm, I dont think you could use conventions to alter the semantics of 
> the data access protocol. That is, operations will still be allowed, 
> but they may fail because they exceed resource limits.
>
> The problem is we want to model our data with variable length arrays, 
> and right now we only have Sequences to do that, which require 
> supporting RCE on all fields. We could move to supporting a List data 
> type, but that has no RCE on any fields. I want to support RCE on some 
> fields of a variable length array, because I can only do that 
> efficiently on some of the fields. (This is true even with an RDB, 
> namely the fields that you have indexed).
>
> More abstractly, this is an example where I (ideally) want to express 
> what opendap calls are practical. The client should be able to exmine 
> the DDS and decide how to best get the info it wants. Because of the 
> generality of opendap (i assume following relational theory) i cant do 
> that.
>
> So you are suggesting that instead of indicating "what is practical" 
> in the DDS, we should do it at some layer above that, eg the "Dapper 
> convention layer". Thats probably what we have to do if we cant do it 
> in the DDS. I wanted to see if anyone else had other ideas before we 
> follow that path, though.
>
>
>>
>>    - steve
>>
>> =================================================
>>
>> John Caron wrote:
>>
>>> Hi James, et al:
>>>
>>> One problem I have in implementing opendap sequences is that you 
>>> have to support relational constraint expressions (RCE) on all of 
>>> the fields of a sequence. This only scales if you use a DBMS. I 
>>> would prefer to support RCE on some of the fields, eg the space and 
>>> time coordinates and station ids.
>>> Im wondering if anyone has this problem, and if there is anything to 
>>> do about it?
>>>
>>