溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這篇文章給大家介紹如何搭建BeeGFS開發環境，內容非常詳細，感興趣的小伙伴們可以參考借鑒，希望對大家能有所幫助。
我的測試環境使用CentOS 7， 安裝BeeGFS的V6版本。
使用下載好的RPM包進行安裝：
$ rpm -ivh beegfs_rpm/*.rpm $ rpm -qa | grep bee beegfs-meta-6.18-el7.x86_64 beegfs-admon-6.18-el7.x86_64 beegfs-utils-6.18-el7.x86_64 beegfs-helperd-6.18-el7.x86_64 beegfs-storage-6.18-el7.x86_64 beegfs-opentk-lib-6.18-el7.x86_64 beegfs-client-6.18-el7.noarch beegfs-common-6.18-el7.noarch beegfs-mgmtd-6.18-el7.x86_64
使用Yum安裝：
$ wget https://www.beegfs.io/release/beegfs_6/dists/beegfs-rhel7.repo -O /etc/yum.repos.d/beegfs-rhel7.repo $ yum install -y beegfs-mgmtd beegfs-meta libbeegfs-ib beegfs-storage libbeegfs-ib beegfs-client beegfs-helperd beegfs-utils beegfs-admon $ yum install -y mdadm
sacd01 $ rm -rfv /beegfs_mgmtd sacd01 $ /opt/beegfs/sbin/beegfs-setup-mgmtd -p /beegfs_mgmtd Preparing storage directory: /beegfs_mgmtd * Creating format.conf file... Updating config file: /etc/beegfs/beegfs-mgmtd.conf * Setting storage directory in config file... * Disabling usage of uninitialized storage directory in config file... All done.
兩個元數據節點的ID分別選擇2和3：
sacd01 $ rm /beegfs_meta/meta* -rf sacd01 $ /opt/beegfs/sbin/beegfs-setup-meta -p /beegfs_meta/meta1 -s 2 -m 172.29.39.125 Preparing storage directory: /beegfs_meta/meta1 * Creating format.conf file... * Creating server numeric ID file: /beegfs_meta/meta1/nodeNumID Updating config file: /etc/beegfs/beegfs-meta.conf * Setting management host: 172.29.39.125 * Setting storage directory in config file... * Disabling usage of uninitialized storage directory in config file... * Setting usage of extended attributes to: true All done. sacd02 $ rm /beegfs_meta/meta* -rf sacd02 $ /opt/beegfs/sbin/beegfs-setup-meta -p /beegfs_meta/meta2 -s 3 -m 172.29.39.125 Preparing storage directory: /beegfs_meta/meta2 * Creating format.conf file... * Creating server numeric ID file: /beegfs_meta/meta2/nodeNumID Updating config file: /etc/beegfs/beegfs-meta.conf * Setting management host: 172.29.39.125 * Setting storage directory in config file... * Disabling usage of uninitialized storage directory in config file... * Setting usage of extended attributes to: true All done.
sacd01 $ rm /beegfs_data/data* -rf
sacd01 $ /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs_data/data1 -s 4 -i 401 -m 172.29.39.125
Preparing storage target directory: /beegfs_data/data1
* Creating format.conf file...
* Creating chunks directory...
* Creating buddymir directory...
* Creating target numeric ID file: /beegfs_data/data1/targetNumID
* Creating server numeric ID file: /beegfs_data/data1/nodeNumID
Updating config file: /etc/beegfs/beegfs-storage.conf
* Setting management host: 172.29.39.125
* Appending to target directory list in config file...
* WARNING: Skipping append. Target directory seems to be included in current targets list already. ('-f' disables this check.)
* Disabling usage of uninitialized storage targets in config file...
All done.
sacd01 $ /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs_data/data2 -s 4 -i 402 -m 172.29.39.125
Preparing storage target directory: /beegfs_data/data2
* Creating format.conf file...
* Creating chunks directory...
* Creating buddymir directory...
* Creating target numeric ID file: /beegfs_data/data2/targetNumID
* Creating server numeric ID file: /beegfs_data/data2/nodeNumID
Updating config file: /etc/beegfs/beegfs-storage.conf
* Setting management host: 172.29.39.125
* Appending to target directory list in config file...
* WARNING: Skipping append. Target directory seems to be included in current targets list already. ('-f' disables this check.)
* Disabling usage of uninitialized storage targets in config file...
All done.
sacd02 $ rm /beegfs_data/data* -rf
sacd02 $ /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs_data/data3 -s 5 -i 501 -m 172.29.39.125
Preparing storage target directory: /beegfs_data/data3
* Creating format.conf file...
* Creating chunks directory...
* Creating buddymir directory...
* Creating target numeric ID file: /beegfs_data/data3/targetNumID
* Creating server numeric ID file: /beegfs_data/data3/nodeNumID
Updating config file: /etc/beegfs/beegfs-storage.conf
* Setting management host: 172.29.39.125
* Appending to target directory list in config file...
* WARNING: Skipping append. Target directory seems to be included in current targets list already. ('-f' disables this check.)
* Disabling usage of uninitialized storage targets in config file...
All done.
sacd02 $ /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs_data/data4 -s 5 -i 502 -m 172.29.39.125
Preparing storage target directory: /beegfs_data/data4
* Creating format.conf file...
* Creating chunks directory...
* Creating buddymir directory...
* Creating target numeric ID file: /beegfs_data/data4/targetNumID
* Creating server numeric ID file: /beegfs_data/data4/nodeNumID
Updating config file: /etc/beegfs/beegfs-storage.conf
* Setting management host: 172.29.39.125
* Appending to target directory list in config file...
* WARNING: Skipping append. Target directory seems to be included in current targets list already. ('-f' disables this check.)
* Disabling usage of uninitialized storage targets in config file...
All done.sacd03 $ /opt/beegfs/sbin/beegfs-setup-client -m 172.29.39.125 Updating config file: /etc/beegfs/beegfs-client.conf * Setting management host: 172.29.39.125 All done.
讓讓整個存儲集群使用指定的網口進行通信：
sacd01 $ echo bond1.1039 > /etc/beegfs/network/beegfs-network-interface.conf sacd02 $ echo bond1.1039 > /etc/beegfs/network/beegfs-network-interface.conf sacd03 $ echo bond1.1039 > /etc/beegfs/network/beegfs-network-interface.conf # Management sacd01 $ vi /etc/beegfs/beegfs-mgmtd.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf # Metadata sacd01 $ vi /etc/beegfs/beegfs-meta.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf sacd02 $ vi /etc/beegfs/beegfs-meta.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf # Storage sacd01 $ vi /etc/beegfs/beegfs-storage.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf sacd02 $ vi vi /etc/beegfs/beegfs-storage.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf # Client sacd03 $ vi /etc/beegfs/beegfs-client.conf connInterfacesFile = /etc/beegfs/network/beegfs-network-interface.conf
sacd01 $ systemctl start beegfs-mgmtd sacd01 $ systemctl start beegfs-meta sacd02 $ systemctl start beegfs-meta sacd01 $ systemctl start beegfs-storage sacd02 $ systemctl start beegfs-storage sacd03 $ beegfs-ctl --listtargets --nodetype=metadata --state TargetID Reachability Consistency NodeID ======== ============ =========== ====== 2 Online Good 2 3 Online Good 3 sacd03 $ beegfs-ctl --listtargets --nodetype=storage --state TargetID Reachability Consistency NodeID ======== ============ =========== ====== 401 Online Good 4 402 Online Good 4 501 Online Good 5 502 Online Good 5
sacd03 $ beegfs-ctl --addmirrorgroup --automatic --nodetype=meta New mirror groups: BuddyGroupID Node type Node ============ ========= ==== 1 primary 2 @ beegfs-meta sacd01 [ID: 2] secondary 3 @ beegfs-meta sacd02 [ID: 3] Mirror buddy group successfully set: groupID 1 -> target IDs 2, 3 sacd03 $ beegfs-ctl --listmirrorgroups --nodetype=meta BuddyGroupID PrimaryNodeID SecondaryNodeID ============ ============= =============== 1 2 3 sacd03 $ beegfs-ctl --mirrormd sacd01 $ systemctl restart beegfs-meta sacd02 $ systemctl restart beegfs-meta
sacd03 $ vi /etc/beegfs/beegfs-client.conf logClientID = true logLevel = 5 #sysMountSanityCheckMS = 11000 sysMountSanityCheckMS = 0 sacd03 $ systemctl start beegfs-helperd && systemctl start beegfs-client sacd03 $ systemctl status beegfs-helperd && systemctl status beegfs-client sacd01 $ systemctl enable beegfs-mgmtd beegfs-meta beegfs-storage sacd02 $ systemctl enable beegfs-meta beegfs-storage sacd03 $ systemctl enable beegfs-client
$ beegfs-ctl --listnodes --nodetype=management --nicdetails sacd01 [ID: 1] Ports: UDP: 8008; TCP: 8008 Interfaces: + bond1.1039[ip addr: 172.29.39.125; type: TCP] Number of nodes: 1 $ beegfs-ctl --listnodes --nodetype=metadata --nicdetails sacd01 [ID: 2] Ports: UDP: 8005; TCP: 8005 Interfaces: + bond1.1039[ip addr: 172.29.39.125; type: TCP] sacd02 [ID: 3] Ports: UDP: 8005; TCP: 8005 Interfaces: + bond1.1039[ip addr: 172.29.39.126; type: TCP] Number of nodes: 2 Root: 2 $ beegfs-ctl --listnodes --nodetype=storage --nicdetails sacd01 [ID: 4] Ports: UDP: 8003; TCP: 8003 Interfaces: + bond1.1039[ip addr: 172.29.39.125; type: TCP] sacd02 [ID: 5] Ports: UDP: 8003; TCP: 8003 Interfaces: + bond1.1039[ip addr: 172.29.39.126; type: TCP] Number of nodes: 2 $ beegfs-ctl --listnodes --nodetype=client --nicdetails C26A-5CD18762-sacd03 [ID: 5] Ports: UDP: 8004; TCP: 0 Interfaces: + bond1.1039[ip addr: 172.29.39.133; type: TCP] Number of nodes: 1
啟動客戶端服務時，可能出現如下錯誤：
$ systemctl start beegfs-client Job for beegfs-client.service failed because the control process exited with error code. See "systemctl status beegfs-client.service" and "journalctl -xe" for details. $ systemctl status beegfs-client.service -l ● beegfs-client.service - Start BeeGFS Client Loaded: loaded (/usr/lib/systemd/system/beegfs-client.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2019-05-07 16:37:01 CST; 17s ago Process: 134148 ExecStart=/etc/init.d/beegfs-client start (code=exited, status=1/FAILURE) Main PID: 134148 (code=exited, status=1/FAILURE) May 07 16:37:00 sacd03 systemd[1]: Starting Start BeeGFS Client... May 07 16:37:00 sacd03 beegfs-client[134148]: Starting BeeGFS Client: May 07 16:37:00 sacd03 beegfs-client[134148]: - Loading BeeGFS modules May 07 16:37:00 sacd03 beegfs-client[134148]: - BeeGFS module autobuild May 07 16:37:01 sacd03 beegfs-client[134148]: Makefile:117: *** Linux kernel build directory not found. Please check if the kernel module development packages are installed for the current kernel version. (RHEL: kernel-devel; SLES: linux-kernel-headers, kernel-source; Debian: linux-headers). Stop. May 07 16:37:01 sacd03 beegfs-client[134148]: make: *** [auto_rebuild] Error 2 May 07 16:37:01 sacd03 systemd[1]: beegfs-client.service: main process exited, code=exited, status=1/FAILURE May 07 16:37:01 sacd03 systemd[1]: Failed to start Start BeeGFS Client. May 07 16:37:01 sacd03 systemd[1]: Unit beegfs-client.service entered failed state. May 07 16:37:01 sacd03 systemd[1]: beegfs-client.service failed.
這是由于沒有安裝kernel-devel，沒有內核頭文件引起的。
或者是下面這個錯誤：
$ vi /var/log/beegfs-client.log Message from syslogd@sacd03 at May 8 09:44:12 ... dracut:dracut: creation of /boot/initramfs-3.10.0-957.12.1.el7.x86_64.tmp failed (5) May08 09:44:33 *df(63614) [C206-5CD22EA8-sacd03: FhgfsOps_getattr] >> called. Path: /; EntryID: root (0) May08 09:44:33 *df(63614) [C206-5CD22EA8-sacd03: Messaging (RPC node)] >> Invalid mirror buddy group ID: 0 (5) May08 09:44:33 *df(63614) [C206-5CD22EA8-sacd03: FhgfsOps_statfs] >> called. Path: n/a (no dentry); EntryID: n/a (no inode) (1) May08 09:44:33 *df(63614) [C206-5CD22EA8-sacd03: Remoting (stat storage targets)] >> No storage targets known. (5) May08 09:45:04 *df(65354) [C206-5CD22EA8-sacd03: FhgfsOps_getattr] >> called. Path: /; EntryID: root (0) May08 09:45:04 *df(65354) [C206-5CD22EA8-sacd03: Messaging (RPC node)] >> Invalid mirror buddy group ID: 0 (5) May08 09:45:04 *df(65354) [C206-5CD22EA8-sacd03: FhgfsOps_statfs] >> called. Path: n/a (no dentry); EntryID: n/a (no inode) (1) May08 09:45:04 *df(65354) [C206-5CD22EA8-sacd03: Remoting (stat storage targets)] >> No storage targets known. $ ls /mnt/beegfs/ ls: reading directory /mnt/beegfs/: Not a directory
這是BeeGFS 6.18在RHEL 3.10.0-862版本內核上的一個Bug引起的：
在所有節點上安裝kernel-3.10.0-693版本的內核和開發包即可解決問題：
$ yum list kernel
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: ap.stykers.moe
* epel: mirrors.tuna.tsinghua.edu.cn
* extras: ap.stykers.moe
* updates: mirrors.tuna.tsinghua.edu.cn
Installed Packages
kernel.x86_64 3.10.0-957.5.1.el7 @updates
Available Packages
kernel.x86_64 3.10.0-957.12.1.el7 updates
$ wget ftp://ftp.pbone.net/mirror/ftp.scientificlinux.org/linux/scientific/7.0/x86_64/updates/security/kernel-3.10.0-693.el7.x86_64.rpm
$ wget ftp://ftp.pbone.net/mirror/ftp.scientificlinux.org/linux/scientific/7.0/x86_64/updates/security/kernel-devel-3.10.0-693.el7.x86_64.rpm
# wget ftp://ftp.pbone.net/mirror/ftp.redhat.com/pub/redhat/linux/enterprise/7Server/en/RHEV/SRPMS/kernel-3.10.0-693.el7.src.rpm
$ rpm -ivh kernel* --force
$ grep "menuentry " /boot/grub2/grub.cfg | awk -F"'" '{print $2}'
CentOS Linux (3.10.0-693.el7.x86_64) 7 (Core)
CentOS Linux (3.10.0-957.12.1.el7.x86_64) 7 (Core)
CentOS Linux (0-rescue-e3a40c9c3add41e3b2eb72e341b52ec1) 7 (Core)
$ vi /etc/default/grub
GRUB_DEFAULT="CentOS Linux (3.10.0-693.el7.x86_64) 7 (Core)"
$ grub2-mkconfig -o /boot/grub2/grub.cfg
$ reboot
$ rpm -evh kernel-3.10.0-957.12.1.el7.x86_64
$ grep "menuentry " /boot/grub2/grub.cfg | awk -F"'" '{print $2}'
CentOS Linux (3.10.0-693.el7.x86_64) 7 (Core)
CentOS Linux (0-rescue-e3a40c9c3add41e3b2eb72e341b52ec1) 7 (Core)關于如何搭建BeeGFS開發環境就分享到這里了，希望以上內容可以對大家有一定的幫助，可以學到更多知識。如果覺得文章不錯，可以把它分享出去讓更多的人看到。
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
