From cermak at sfos.uaf.edu  Sun Jun  8 11:18:28 2008
From: cermak at sfos.uaf.edu (Rob Cermak)
Date: Sun, 8 Jun 2008 10:18:28 -0800 (AKDT)
Subject: [Opendap] beslistener segfaults: 64-bit CentOS 5.1
Message-ID: <60076.216.67.45.40.1212949108.squirrel@ssl.sfos.uaf.edu>

Linux penguin.sfos.uaf.edu 2.6.18-53.1.21.el5 #1 SMP Tue May 20 09:35:07
EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

BES: 3.6.1
libdap: 3.8.0-svn

netcdf_handler-3.7.9
dap-server-3.8.5
dap-2.2.6

olfs-1.4.1

We see periodic segfaults with the beslistener on x86_64 system.  Two
different systems with it showing these:

Jun  8 05:13:13 r61-christina kernel: beslistener[24542]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 05:13:13 r61-christina kernel: beslistener[25038]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 05:13:19 r61-christina kernel: beslistener[25041]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 05:13:19 r61-christina kernel: beslistener[24551]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 09:00:09 r61-christina kernel: beslistener[22287]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 09:19:58 r61-christina kernel: beslistener[26163]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 09:24:30 r61-christina kernel: beslistener[27041]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4
Jun  8 09:40:56 r61-christina kernel: beslistener[29429]: segfault at
0000000000000024 rip 00002aaaab3d8344 rsp 00007fff30f78df0 error 4

It isn't tied to any specific file type or corrupt file.  Using the
python module pydap, if a segfault occurs, the module fails:

Traceback (most recent call last):
  File "./mk_xml", line 176, in ?
    wspd = wspdvar[:]
  File
"/usr/local/lib/python2.4/site-packages/dap-2.2.6-py2.4.egg/dap/dtypes.py",
line 411, in __getitem__
    return self.data[key]
  File
"/usr/local/lib/python2.4/site-packages/dap-2.2.6-py2.4.egg/dap/proxy.py",
line 117, in __getitem__
    start = data.index('Data:\n') + len('Data:\n')
ValueError: substring not found

If we re-run the script, the problem may or usually does not re-occur.  
The one time it did reoccur, it was accessing some other variable.

Traceback (most recent call last):
  File "./mk_xml", line 146, in ?
    rh = rhvar[:]
  File
"/usr/local/lib/python2.4/site-packages/dap-2.2.6-py2.4.egg/dap/dtypes.py",
line 411, in __getitem__
    return self.data[key]
  File
"/usr/local/lib/python2.4/site-packages/dap-2.2.6-py2.4.egg/dap/proxy.py",
line 117, in __getitem__
    start = data.index('Data:\n') + len('Data:\n')

Any pointers on things to try via SVN?  or a new release of items
forthcoming?

Now that we know where to look, if we can dig up some more log
information to track this down, I will forward it along.

Thanks,
Rob

Data Management
Alaska Ocean Observing System