如何在線安裝CDH5.4.1

這篇文章主要介紹如何在線安裝CDH5.4.1，文中介紹的非常詳細，具有一定的參考價值，感興趣的小伙伴們一定要看完！

下載CENTOS6.4
http://118.122.119.70/CentOS-6.4-x86_64-bin-DVD1.iso?fid=KBiGdobo4XpP2az01*useRyNKDoASHsDAQAAAPMPsesUnrepQ15p8WWIN7Ynh4Ft&mid=666&threshold=150&tid=4990D3C32C91D7AF85B132EF51EE881D&srcid=119&verno=1
配置主機名
網絡配置
安裝Apache 服務器
關閉防火墻和selinux 及配置信任關系
設置NTP時間同步
增加普通用戶hadoop 并設置sudo權限
安裝包準備
執行文件 http://archive.cloudera.com/cm5/installer
rpm包： http://archive.cloudra.com/cm5/repo-as-tarball/
給予權限 chmod 777 cloudra-manager-installer.bin
[root@localhost ~]# service iptables stop
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:  
root@localhost ~]# chkconfig iptables off
                             [  OK  ]
[root@localhost ~]# vi /etc/sysconfig/selinux
修改selinu                            
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

[root@localhost ~]#reboot
[root@localhost ~]#reboot
[root@localhost ~]# setup
[root@localhost ~]#  service network restart
Shutting down interface eth2:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth2:                                [  OK  ]
[root@localhost ~]# ifconfig
eth2      Link encap:Ethernet  HWaddr 00:0C:29:30:FE:A3  
          inet addr:192.168.141.11  Bcast:192.168.141.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe30:fea3/64 Scope:Link
          
          

修改IP和主機名映射關系vi /etc/hosts
192.168.141.11            CDH01
[root@localhost ~]# hostname CDH01
[root@localhost ~]# hostname
CDH01
[root@localhost ~]# vi /etc/sysconfig/network修改主機名
NETWORKING=yes
HOSTNAME=CDH01
查看是否安裝java（安裝了就卸載掉）
[root@CDH01 ~]# rpm -qa|grep jdk
[root@CDH01 ~]# rpm -qa|grep java
添加hadoop用戶
[root@CDH01 ~]# adduser hadoop
[root@CDH01 ~]# passwd hadoop
root@CDH01 ~]# su hadoop
[hadoop@CDH01 root]$ whoami
hadoop

hadoop@CDH01 root]$ cd /etc/
[hadoop@CDH01 etc]$ ls -l|grep sudo
-rw-r-----.  1 root root   1786 Sep 25  2012 sudo.conf
-r--r-----.  1 root root   4002 Mar  1  2012 sudoers
drwxr-x---.  2 root root   4096 Feb 21  2013 sudoers.d
-rw-r-----.  1 root root   3181 Jan 23  2013 sudo-ldap.conf
[hadoop@CDH01 etc]$ su
Password:
[root@CDH01 etc]# vi sudoers
在root    ALL=(ALL)       ALL后添加
hadoop  ALL=(root)NOPASSWD:ALL
[root@CDH01 etc]# sudo ls /root/就不會報錯了
[root@CDH01 etc]# ssh-keygen -t rsa
[root@CDH01 etc]# ssh-copy-id CDH01

[root@CDH01 etc]# ssh CDH01登錄不用密碼
[root@CDH01 ~]# rpm -qa|grep httpd
httpd-tools-2.2.15-26.el6.centos.x86_64
httpd-2.2.15-26.el6.centos.x86_64
root@CDH01 ~]# service httpd status
httpd is stopped
[root@CDH01 ~]# sudo service httpd start
[root@CDH01 ~]# sudo chkconfig httpd on
[root@CDH01 ~]#reboot
下載文件http://archive.cloudera.com/cm5/installer/5.4.1/cloudera-manager-installer.bin
下載文件http://archive.cloudera.com/cm5/repo-as-tarball/5.4.1/cm5.4.1-centos6.tar.gz（6代表是centos6.x）
[root@CDH01 /]# cd /opt
[root@CDH01 opt]# pwd
/opt
[hadoop@CDH01 opt]$ sudo rm -rf *
[hadoop@CDH01 opt]$ sudo mkdir softwares
[hadoop@CDH01 opt]$ ls
softwares
[hadoop@CDH01 opt]$ sudo yum install -y lrzsz

[root@CDH01 softwares]# sudo ls -la /opt
total 12
drwxr-xr-x.  3 root root 4096 Dec 18 09:15 .
dr-xr-xr-x. 22 root root 4096 Dec 18 08:06 ..
drwxr-xr-x   2 root root 4096 Dec 18 09:25 softwares
[root@CDH01 softwares]# sudo chown -R hadoop:hadoop /opt
[root@CDH01 softwares]# sudo ls -la /opt
total 12
drwxr-xr-x.  3 hadoop hadoop 4096 Dec 18 09:15 .
dr-xr-xr-x. 22 root   root   4096 Dec 18 08:06 ..
drwxr-xr-x   2 hadoop hadoop 4096 Dec 18 09:25 softwares
[root@CDH01 softwares]# tar -zxf cm5.4.1-centos6.tar.gz -C /var/www/html
[root@CDH02 softwares]# cd  /var/www/html
[root@CDH01 html]# sudo mkdir -p cm5/redhat/6/x86_64/
[root@CDH01 html]# sudo mv cm cm5/redhat/6/x86_64/
修改yum源
[root@CDH01 html]# cd /etc/yum.repos.d/
[root@CDH01 yum.repos.d]# ls
CentOS-Base.repo  CentOS-Debuginfo.repo  CentOS-Media.repo  CentOS-Vault.repo
[root@CDH01 yum.repos.d]# sudo vi cloudera-manager.repo
[cloudera-manager]
name=cloudera-manager
baseurl=http://CDH01/cm5/redhat/6/x86_64/cm/5/
enabled=1
gpgcheck=0

[root@CDH02 yum.repos.d]# yum repolist 【生成了yum源】
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'"
cloudera-manager                                         |  951 B     00:00
cloudera-manager/primary                                 | 4.3 kB     00:00
cloudera-manager                                                            7/7
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=extras&infra=stock error was
14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'"
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=updates&infra=stock error was
14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'"
repo id                             repo name                             status
base                                CentOS-6 - Base                       0
cloudera-manager                    cloudera-manager                      7
extras                              CentOS-6 - Extras                     0
updates                             CentOS-6 - Updates                    0
repolist: 7

[root@CDH02 softwares]# wget http://archive.cloudera.com/cm5/installer/5.9.0/cloudera-manager-installer.bin
[root@CDH01 softwares]# ./cloudera-manager-installer.bin[root用戶,并且要聯網]
[root@CDH01 cloudera-manager-installer]# ls
0.check-selinux.log     2.install-oracle-j2sdk1.7.log
1.install-repo-pkg.log  3.install-cloudera-manager-server.log
[root@CDH01 ~]# cd /opt/softwares/
[root@CDH01 softwares]# ls
cloudera-manager-installer.bin  cm5.4.1-centos6.tar.gz

next->next->yes->next->yes
訪問192.168.141.11:7180
[root@CDH01 softwares]# cd /var/www/html
[root@CDH01 html]# ls
cm5
[root@CDH01 html]# mkdir parcels 【http://archive.cloudera.com/cdh6/parcels/5.9.0/】
[root@CDH01 html]# ls
cm5  parcels
[root@CDH01 html]# cd parcels
[root@CDH01 parcels]# ls
CDH-5.4.1-1.cdh6.4.1.p0.6-el6.parcel       manifest.json
CDH-5.4.1-1.cdh6.4.1.p0.6-el6.parcel.sha1
[root@CDH01 parcels]# mv CDH-5.4.1-1.cdh6.4.1.p0.6-el6.parcel.sha1 CDH-5.4.1-1.cdh6.4.1.p0.6-el6.parcel.sha
訪問http://192.168.141.11/parcels/有三個文件

遠程 Parcel 存儲庫 URL http://192.168.141.11/parcels/
選擇 CDH 的版本
CDH-5.4.1-1.cdh6.4.1.p0.6

自定義存儲庫
http://192.168.141.11/cm5/redhat/6/x86_64/cm/5/

安裝 Oracle Java SE 開發工具包 (JDK) 打鉤
單用用戶打鉤
登錄到所有主機，作為hadoop

root用戶
     Cloudera 建議將 /proc/sys/vm/swappiness 設置為 0。當前設置為 60。使用 sysctl 命令在運行時更改該設置并編輯 /etc/sysctl.conf 以在重啟后保存該設置。您可以繼續進行安裝，但可能會遇到問題，Cloudera Manager 報告您的主機由于交換運行狀況不佳。以下主機受到影響：
[root@CDH01 parcels]# sudo sysctl vm.swappiness=0
vm.swappiness = 0
[root@CDH01 parcels]# vi /etc/sysctl.conf
vm.swappiness=0

已啟用“透明大頁面”，它可能會導致重大的性能問題。版本為“CentOS release 6.4 (Final)”且發行版為“2.6.32-358.el6.x86_64”的 Kernel 已將 enabled 設置為“[always] never”，
并將 defrag 設置為“[always] never”。請運行“echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag”以禁用此設置，然后將同一命令添加到一個 init 腳本中，如 /etc/rc.local，這樣當系統重啟時就會予以設置。或者，升級到 RHEL 6.5 或更新版本，它們不存在此錯誤。將會影響到以下主機：
root@CDH01 parcels]# sudo echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
[root@CDH01 parcels]# sudo vi /etc/rc.local
增加echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
添加Cloudera Management Service【先不安裝組件】
使用嵌入式數據庫-》繼續直到完成
添加服務zookeper

    [root@CDH01 cloudera-scm-agent]# service cloudera-scm-agent restart[可能權限有問題， chmod 777  /var/log/cloudera-scm-*]

    http://192.168.141.11:7180/cmf/login
[root@CDH01 cloudera-scm-server]# netstat -ano |grep 7180
tcp        0      0 0.0.0.0:7180                0.0.0.0:*                   LISTEN      off (0.00/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51316         ESTABLISHED off (0.00/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51315         ESTABLISHED off (0.00/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51314         ESTABLISHED off (0.00/0/0)
tcp        0      0 192.168.141.11:54184        192.168.141.11:7180         TIME_WAIT   timewait (34.65/0/0)
tcp        0      0 192.168.141.11:54204        192.168.141.11:7180         TIME_WAIT   timewait (54.24/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51312         ESTABLISHED off (0.00/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51313         ESTABLISHED off (0.00/0/0)
tcp        0      0 192.168.141.11:7180         192.168.141.1:51310         ESTABLISHED off (0.00/0/0)
[root@CDH02 bin]# cd /opt/cloudera/parcels/CDH/bin
配置->hdfs權限-》保存更改-》HDFS重啟
root@CDH02 bin]# /usr/java/jdk1.7.0_67-cloudera/bin/jps
18694 EventCatcherService
3614 NameNode
3639 DataNode
18733 Jps
2655 Main
4807 Main
3598 Bootstrap
5149 NodeManager
4828 Main
4762 AlertPublisher
3589 SecondaryNameNode
5372 ResourceManager
4787 Main
3587 QuorumPeerMain
[root@CDH02 CDH]# bin/yarn jar share/doc/hadoop-0.20-mapreduce/examples/hadoop-examples-2.6.0-mr1-cdh6.4.1.jar
bin/yarn jar share/doc/hadoop-0.20-mapreduce/examples/hadoop-examples-2.6.0-mr1-cdh6.4.1.jar wordcount /a /output01
配置-》操作-》啟用ha

以上是“如何在線安裝CDH5.4.1”這篇文章的所有內容，感謝各位的閱讀！希望分享的內容對大家有幫助，更多相關知識，歡迎關注億速云行業資訊頻道！