OPeNDAP/THREDDS Directory Views

[Pan, Jerry Yun]

This was a question in my mind for a while too, so I am jumping on this
one.

My suggestion is this: the data providers maybe better off having the
freedom on this interface themselves.  I think it would be great if I
could chose (1) leave it as it is, a default template,  (2) OLFS only
(but with information how to get the catlog xml),  (3) use thredss
interface only, or (4) custom the interface freely, arange positions,
layout, add/delete additional contents.  It would be nice if I could
customize all the pages.

If we don't use servelet to spit out the htmp and instead use JSPs, this
desiered behavior may be easy to achieve.  There are also frameworks
like Struts or Java Server Face could help on this.

Thanks.

-Jerry



-----Original Message-----
From: owner-opendap-tech at unidata.ucar.edu
[mailto:owner-opendap-tech at unidata.ucar.edu] On Behalf Of Nathan Potter
Sent: Thursday, April 12, 2007 5:52 PM
To: James Gallagher; Daniel Holloway; OPeNDAP Tech
Cc: Nathan Potter; John Caron; Ethan Davis
Subject: OPeNDAP/THREDDS Directory Views


Greetings,

I would like to get some feedback about one of the OPeNDAP "services"  
and some possible changes to (or elimination/replacement of) the
service.

OPeNDAP Directory/Contents

OPeNDAP servers offer a "presentation view" of each collection of
datasets on the server. Typically this has been accessed by using a
browser to open a URL ending in "/". The OPeNDAP sever returns an HTML
document that the browser renders in human readable form.  In server 3
this view was generated by the Apache web server.

Server 3 example: http://test.opendap.org/opendap-3.7/nph-dods/data/nc/

With Hyrax we have integrated THREDDS catalogs into our server.  
THREDDS has it's own presentation view which is different from the
OPeNDAP presentation view.

Hyrax OPeNDAP presentation view: http://test.opendap.org:8080/opendap/
data/nc/contents.html
Hyrax THREDDS presentation view: http://test.opendap.org:8080/opendap/
data/nc/catalog.html


Unfortunately this is already creating problems:

1) The THREDDS view may contain additional datasets that are not visible
in the OPeNDAP view. (Data providers may add datasets to the catalog
that are not associated with data served by the BES)

2) If the THREDDS catalog.xml doesn't contain a datasetScan element for
each of the datasets/directories at the top level of the server then the
omitted ones will not appear in the THREDDS view (and catalog). This
means that the OPeNDAP view may include datasets that the THREDDS view
does not.

3) The OPeNDAP view differentiates between files that the server
recognizes as data and ones it does not. The dataset links in the
OPeNDAP view lead to the HTML data request form for data, and will
simply return the contents of the file if the server doesn't recognize
it as a dataset that it can handle. The THREDDS view is built using the
assumption that everything returned in a datasetScan is data. So -
THREDDS views don't really have a way to include README files - they
show up as a dataset and drilling down leads to broken links and not to
access to the file.


- I can fix (1).

- Fixing (2) is really a configuration problem that has to be solved by
each data provider.

- Fixing (3) is much harder, and begs the question: Why support 2 views
at all?



** OPTION 1 **
Drop the OPeNDAP presentation view. If we can live with just the THREDDS
view then I don't need to fix (1). Problem (2) will be addressed more
readily by data providers because the THREDDS catalog will need to be
correctly configured for the server to produce THREDDS catalogs and
views. The draw back to this is that we would loose the ability to serve
"flat files" - README, index.html, info.html, images, etc.


** OPTION 2 **
Fix (1) and live with (2) and (3), broken links and all.


** OPTION 3 **
Work with the THREDDS group and woo them to the idea that (3) is a
problem and that the THREDDS API's (and catalogs?) need to modified to
solve (3). I suspect we could do it by adding a different "service" for
the flat files. The CrawlableDataset interface would have to modified so
that it could be used to identify which files were to be served using
which service. When the datasetScan elements are accessed the catalog
get built with the serviceName pointing to the OPeNDAP service for data,
to the file service for files that are not data... But I am speculating
as I haven't spoken our esteemed THREDDS colleagues.



What do people think?

Is OPTION 1 viable?

Other ideas/options/thoughts/opinions?


Nathan




= = =
Nathan Potter                        ndp at opendap.org
OPeNDAP, Inc.                        541.752.1852