本文标题:Direct NFS Client【本站如从外站转载,会注明出处】
本文链接:http://www.validba.net/2009/10/direct-nfs-client/
oracle11g增强了NFS,oracle通过升级ODM(Oracle Disk Manager) Library在Oracle kernel中支持NFS. 采用DIRECT的方式直接对NFS设备进行操作.突破了之前依赖OS Kernel读写NFS的限制.Oracle10g已经支持在RAC环境使用NFS,但都是以传统的方式进行读写,没有性能上的提高, 11G改变这一切.
OS NFS和Direct NFS区别在于,OS NFS以串行的方式操作NFS,也就是说同一个NFS client只能建立一个到NFS Server的连接.如果NFS Client 发出了多个请求,NFS Server是通过串行的方式处理每个请求,并且这一TCP连接是在NFS Client mount nfs 时已经建立好的;Oracke Direct NFS 可以同时建立多个Channel到NFS Server,并行的操作NFS设备,同时Oracle Direct NFS 通过async I/O的方式读写 NFS设备.可以参考White paper
下面参考Direct NFS Client的配置:
- 配置NFS Server
这里的NFS serer是一个用freenas和pc服务器搭建的nas服务器. 启用了NFS服务.
# cat /var/etc/exports
/mnt/home -alldirs -mapall=root -network 128.0.0.0 -mask 128.0.0.0
/mnt/oraclebackup -alldirs -mapall=root -network 128.0.0.0 -mask 128.0.0.0
/mnt/dbbank -alldirs -mapall=root -network 128.0.0.0 -mask 128.0.0.0
- 配置Oracle Direct NFS Client
数据库版本:
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
PL/SQL Release 11.1.0.6.0 - Production
CORE 11.1.0.6.0 Production
TNS for Linux: Version 11.1.0.6.0 - Production
NLSRTL Version 11.1.0.6.0 - Production
配置oranfstab文件,启动Direct NFS,首先要在$ORACLE_HOME/dbs下配置oranfstab文件,这个文件里面配置了NFS Server的地址以及相关的disk share信息,默认情况下此文件是不存在的,可以创建一个.
[ora11@dbtest ~]$ more /oracle/product/11.1/dbs/oranfstab
server:freenas
path:192.168.203.200
path:192.168.203.201
export:/mnt/home
mount:/nas_temp
uid:65534
gid:65534
这个几个参数不难理解,oracle为了确保NFS的可用性,每个NFS SERVER下的每个NFS目录最大可以支持四个path,在第一paht failer时,Oracle可以选用后面的path继续操作NFS设备.
配置oranfsodm.默认情况下oracle没有使用nfsodm.ODM介绍详见
Metalink:815085.1
http://www.oracle.com/technology/deploy/availability/pdf/odm_wp.pdf
http://www.oracle.com/technology/deploy/availability/pdf/nitin_ODM.pdf
[ora11@dbtest ~]$ cd $ORACLE_HOME/lib/
[ora11@dbtest lib]$ ls -l *odm*
-rw-r--r-- 1 ora11 oinstall 44670 Aug 1 2007 libnfsodm11.so
lrwxrwxrwx 1 ora11 oinstall 14 Oct 26 15:37 libodm11.so -> libodmd11.so
-rw-r--r-- 1 ora11 oinstall 8841 Aug 1 2007 libodmd11.so
这里oracle 默认使用libodmd11.so,现在修改使用libnfsodm11.so
cp libodm11.so libodm11.so.cp
rm libodm11.so
ln -s libnfsodm11.so libodm11.so
mount nfs磁盘
su - root
[root@dbtest ~]# mount -t nfs 192.168.203.200:/mnt/home /nas_temp -o _netdev,rw,rsize=32768,wsize=32768
[root@dbtest ~]# more /etc/mtab
/dev/hda2 / ext3 rw 0 0
none /proc proc rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
usbfs /proc/bus/usb usbfs rw 0 0
/dev/hda1 /boot ext3 rw 0 0
none /dev/shm tmpfs rw 0 0
/dev/hda5 /oracle ext3 rw 0 0
/dev/hda3 /u01 ext3 rw 0 0
/dev/hda7 /usr ext3 rw 0 0
/dev/mapper/data-datavg /datavg ext3 rw 0 0
/dev/mapper/data-datavg1 /datavg1 ext3 rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
configfs /config configfs rw 0 0
ocfs2_dlmfs /dlm ocfs2_dlmfs rw 0 0
/dev/mapper/data-volumn /volumn ocfs2 rw,_netdev,datavolume,nointr,heartbeat=local 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0
192.168.203.200:/mnt/home /nas_temp nfs rw,_netdev,rsize=32768,wsize=32768,addr=192.168.203.200 0 0
重新启动数据库
su - ora11sqlplus / as sysdba
shutdown immediate
startup
测试nfs
在Oracle 读写NFS的过程中,可以通过V$DNFS_SERVERS,V$DNFS_FILES,V$DNFS_CHANNELS,V$DNFS_STATS四个视图来观察nfs的执行情况
create tablespace nas datafile '/nas_temp/nas.dbf' size 2000M;
观察nfs的使用
SQL> select * from v$dnfs_servers;
ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX
---------- -------------------- -------------------- ---------- ---------- ---------- ----------
1 192.168.203.200 /mnt/home 751 2049 32768 32768
SQL> select * from v$dnfs_files;
FILENAME FILESIZE PNUM SVR_ID
------------------------------ ---------- ---------- ----------
/nas_temp/nas.dbf 2097160192 32 1
SQL> select * from v$dnfs_channels;
PNUM SVRNAME PATH CH_ID SVR_ID SENDS RECVS PINGS
---------- -------------------- -------------------- ---------- ---------- ---------- ---------- ----------
9 192.168.203.200 192.168.203.200 0 1 0 0 0
9 192.168.203.200 192.168.203.200 1 1 0 0 0
26 192.168.203.200 192.168.203.200 0 1 0 0 0
30 192.168.203.200 192.168.203.200 0 1 0 0 0
32 192.168.203.200 192.168.203.200 0 1 571 4454 0
34 192.168.203.200 192.168.203.200 0 1 0 0 0
37 192.168.203.200 192.168.203.200 0 1 0 0 0
这里可以发现oracle在表空间创建期间创建多个channel,但只有30一个channel在读写.看看并行情况下,nfs的表现:
SQL> create table a tablespace nas pctfree 0 as select * from dba_objects;
Table created.
SQL> insert into a select * from dba_objects;
68319 rows created.
SQL> /
68319 rows created.
SQL> /
68319 rows created.
SQL> select count(*) from a;
COUNT(*)
----------
68319
SQL> /
PNUM SVRNAME PATH CH_ID SVR_ID SENDS RECVS PINGS
---------- -------------------- -------------------- ---------- ---------- ---------- ---------- ----------
9 192.168.203.200 192.168.203.200 0 1 17 475 0
9 192.168.203.200 192.168.203.200 1 1 0 0 0
26 192.168.203.200 192.168.203.200 0 1 0 0 0
30 192.168.203.200 192.168.203.200 0 1 0 0 0
31 192.168.203.200 192.168.203.200 0 1 160 478 0
32 192.168.203.200 192.168.203.200 0 1 0 0 0
37 192.168.203.200 192.168.203.200 0 1 0 0 0
7 rows selected.
---两个channel同时工作.
SQL> select /*+ parallel (a 4) */ count(*) from a;
COUNT(*)
----------
68319
SQL> /
PNUM SVRNAME PATH CH_ID SVR_ID SENDS RECVS PINGS
---------- -------------------- -------------------- ---------- ---------- ---------- ---------- ----------
9 192.168.203.200 192.168.203.200 0 1 0 0 0
9 192.168.203.200 192.168.203.200 1 1 0 0 0
11 192.168.203.200 192.168.203.200 0 1 0 0 0
11 192.168.203.200 192.168.203.200 1 1 0 0 0
26 192.168.203.200 192.168.203.200 0 1 0 0 0
30 192.168.203.200 192.168.203.200 0 1 0 0 0
31 192.168.203.200 192.168.203.200 0 1 0 0 0
32 192.168.203.200 192.168.203.200 0 1 293 870 0
33 192.168.203.200 192.168.203.200 0 1 293 871 0
34 192.168.203.200 192.168.203.200 0 1 293 875 0
35 192.168.203.200 192.168.203.200 0 1 293 870 0
PNUM SVRNAME PATH CH_ID SVR_ID SENDS RECVS PINGS
---------- -------------------- -------------------- ---------- ---------- ---------- ---------- ----------
37 192.168.203.200 192.168.203.200 0 1 0 0 0
12 rows selected.
--四个channel同时工作,这里说明了Direct NFS的并行操作.
通过NFS SERVER也可以观察到多个连接同时建立
# netstat -an | awk '/2049/'
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58018 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58017 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58009 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58008 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58007 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.58006 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.57992 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.57842 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.56195 ESTABLISHED
tcp4 0 0 192.168.203.200.2049 192.168.200.64.56194 ESTABLISHED
tcp6 0 0 *.2049 *.* LISTEN
tcp4 0 0 *.2049 *.* LISTEN
udp6 0 0 *.2049 *.*
udp4 0 0 *.2049 *.*
CopyRight Validba'S Home 版权所有.本站为个人学习之用,评论时请遵守本国法律
