In my recent post entitled Oracle Database 11g Database File System (DBFS) Is Not A Popular Topic. Yet., I discussed how I use Google search terms as a way of monitoring what sorts of problems people are running into with new technology such as (DBFS).
This is part one in a series of posts where I’ll offer some tips regarding the more common DBFS-related problems one might encounter in early testing. The first I’ll tackle is “Transport endpoint is not connected.”
Error Text: Transport Endpoint is not connected.
This error string occurs when attempting to access a mounted DBFS file system. The error condition arises when there is no dbfs_client currently executing that is associated with the file system.
In the following boxes I’ll show two examples of how this error can be encountered. First, I’ll show that I have a DBFS file system mounted and that an attempt to change directories to the mount point results in “Transport endpoint is not connected”, such as:
$ mount | grep dbfs dbfs on /data type fuse (rw,nosuid,nodev,max_read=1048576,default_permissions,allow_other,user=oracle) $ cd /data -bash: cd: /data: Transport endpoint is not connected
A simple ps command shows there is no dbfs_client process executing:
$ ps -ef | grep dbfs_client oracle 1694 21941 0 09:49 pts/11 00:00:00 grep dbfs_client
This error condition is quite simple to rectify. You simply need to restart the dbfs_client process:
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib $ export ORACLE_SID=DBFS1 $ $ ORACLE_HOME/bin/dbfs_client dbfs@ -o allow_root,direct_io /data < /opt/oracle/dbfs/passwd.txt & [1] 2501 $ fuse: bad mount point `/data': Transport endpoint is not connected
Sorry, I had to be sneaky in order to show one of the other conditions that raises the error. So, yes, to rectify this error condition you do have to restart dbfs_client, however, you need to first unmount the dead mount. You’ll see in the following box that I use fusermount –u to unmount and then execute dbfs_client and without any further action I can list the contents of the file system.
$ fusermount -u /data $ $ORACLE_HOME/bin/dbfs_client dbfs@ -o allow_root,direct_io /data < /opt/oracle/dbfs/passwd.txt & [1] 2931 $ Password: $ ls -l /data/FS1/TEST total 0 -rw-r--r-- 1 oracle oinstall 0 Dec 30 13:39 511b -rw-r--r-- 1 root root 0 Dec 30 13:40 test.out
Dead Mount, Living DBFS Instance.
If you look in the boxes above you’ll notice that dbfs_client was not executing (the real root of the problem) but the instance was running. Since dbfs_client is an OCI program all I had to do was kill it to create this scenario. If, however, you suffer this error while dbfs_client is executing then I recommend involving support because that would be a bug.
Noted on an exadata one dbfs is down on one node but the other is up. Neither show the dbfs_client as running.
the good node:
[oracle@ex01db01 ~]$ ps -ef | grep dbfs
oracle 3916 25145 0 15:31 pts/6 00:00:00 grep dbfs
oracle 24870 1 0 Sep16 ? 00:01:41 /sbin/mount.dbfs /@xiwprdfs.local /dbfsmnt -o rw,noexec,nosuid,nodev,direct_io,allow_other,wallet
[oracle@ex01db01 ~]$
[oracle@ex01db01 ~]$ df -k /dbfsmnt
Filesystem 1K-blocks Used Available Use% Mounted on
dbfs 104932992 102824424 2108568 98% /dbfsmnt
[oracle@ex01db01 ~]$
the bad node:
[oracle@ex01db05 ~]$ ps -ef | grep dbfs
oracle 15892 1 0 15:04 ? 00:00:00 /sbin/mount.dbfs /@xiwprdfs.local /dbfsmnt -o rw,noexec,nosuid,nodev,direct_io,allow_other,wallet
oracle 27925 8715 0 15:33 pts/1 00:00:00 grep dbfs
[oracle@ex01db05 ~]$
[oracle@ex01db05 ~]$ df -k /dbfsmnt
df: `/dbfsmnt’: Input/output error
[oracle@ex01db05 ~]$
Boil down the problem. Execute the dbfs_client mount command by hand without wallet then work your way back up the “feature stack.”