OPeNDAP/THREDDS Directory Views

[Nathan Potter]

Greetings,

I would like to get some feedback about one of the OPeNDAP "services"  
and some possible changes to (or elimination/replacement of) the  
service.

OPeNDAP Directory/Contents

OPeNDAP servers offer a "presentation view" of each collection of  
datasets on the server. Typically this has been accessed by using a  
browser to open a URL ending in "/". The OPeNDAP sever returns an  
HTML document that the browser renders in human readable form.  In  
server 3 this view was generated by the Apache web server.

Server 3 example: http://test.opendap.org/opendap-3.7/nph-dods/data/nc/

With Hyrax we have integrated THREDDS catalogs into our server.  
THREDDS has it's own presentation view which is different from the  
OPeNDAP presentation view.

Hyrax OPeNDAP presentation view: http://test.opendap.org:8080/opendap/ 
data/nc/contents.html
Hyrax THREDDS presentation view: http://test.opendap.org:8080/opendap/ 
data/nc/catalog.html


Unfortunately this is already creating problems:

1) The THREDDS view may contain additional datasets that are not  
visible in the OPeNDAP view. (Data providers may add datasets to the  
catalog that are not associated with data served by the BES)

2) If the THREDDS catalog.xml doesn't contain a datasetScan element  
for each of the datasets/directories at the top level of the server  
then the omitted ones will not appear in the THREDDS view (and  
catalog). This means that the OPeNDAP view may include datasets that  
the THREDDS view does not.

3) The OPeNDAP view differentiates between files that the server  
recognizes as data and ones it does not. The dataset links in the  
OPeNDAP view lead to the HTML data request form for data, and will  
simply return the contents of the file if the server doesn't  
recognize it as a dataset that it can handle. The THREDDS view is  
built using the assumption that everything returned in a datasetScan  
is data. So - THREDDS views don't really have a way to include README  
files - they show up as a dataset and drilling down leads to broken  
links and not to access to the file.


- I can fix (1).

- Fixing (2) is really a configuration problem that has to be solved  
by each data provider.

- Fixing (3) is much harder, and begs the question: Why support 2  
views at all?



** OPTION 1 **
Drop the OPeNDAP presentation view. If we can live with just the  
THREDDS view then I don't need to fix (1). Problem (2) will be  
addressed more readily by data providers because the THREDDS catalog  
will need to be correctly configured for the server to produce  
THREDDS catalogs and views. The draw back to this is that we would  
loose the ability to serve "flat files" - README, index.html,  
info.html, images, etc.


** OPTION 2 **
Fix (1) and live with (2) and (3), broken links and all.


** OPTION 3 **
Work with the THREDDS group and woo them to the idea that (3) is a  
problem and that the THREDDS API's (and catalogs?) need to modified  
to solve (3). I suspect we could do it by adding a different  
"service" for the flat files. The CrawlableDataset interface would  
have to modified so that it could be used to identify which files  
were to be served using which service. When the datasetScan elements  
are accessed the catalog get built with the serviceName pointing to  
the OPeNDAP service for data, to the file service for files that are  
not data... But I am speculating as I haven't spoken our esteemed  
THREDDS colleagues.



What do people think?

Is OPTION 1 viable?

Other ideas/options/thoughts/opinions?


Nathan




= 
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852