基于keystone認證的Swift的安裝與配置

一、概要

OpenStackObject Storage（Swift）是OpenStack開源云計算項目的子項目之一。Swift使用普通的服務器來構建冗余的、可擴展的分布式對象存儲集群，存儲容量可達PB級。Swift的是用Python開發，前身是Rackspace Cloud Files項目，隨著Rackspace加入到OpenStack社區，Racksapce也將Cloud Files的代碼貢獻給了社區，并逐漸形成現在Swift。Swift最新的發型版本為essex 1.4.6。

二、術語

node- a host machine running one or more Swift services

Proxynode - node that runs Proxy services; also runs keystoneAuth

Storagenode - node that runs Account, Container, and Object services

ring- a set of mappings of Swift data to physical devices

三、系統架構

四、測試環境

本次測試將Auth和Proxy安裝在1臺服務器，3個Storage節點做測試。單機安裝請參照官方SAIO安裝方式。Auth使用keystone。

版本說明：ubuntu-12.04    swift1.9.1

五、初始化設置

1）General Installation Steps for All Nodes

添加cloud archive gpg key:

apt-getinstall ubuntu-cloud-keyring

添加Ubuntu Cloud Archive repository

#vi/etc/apt/sources.list.d/grizzly.list添加如下內容

debhttp://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main

#apt-get update && apt-get upgrade

2）GeneralInstallation Steps for All Swift Nodes（include proxy）

安裝swift及依賴包

#apt-getinstall -y swift openssh-server rsync memcached python-netifaces python-xattrpython-memcache

創建Swift配置目錄

# mkdir-p /etc/swift

創建Swift配置文件，并同步到all Swift nodes

#vi/etc/swift/swift.conf    內容如下：

[swift-hash]
swift_hash_path_suffix = Gdr8ny7YyWqy2

修改Swift目錄權限

#chown-R swift:swift /etc/swift/

3）時間同步

Swiftproxy Node做ntp服務器，Swift Storage Node與它做同步

安裝ntp

#apt-getinstall -y ntp

配置ntp

# sed -i's/server0.ubuntu.pool.ntp.org/#server 0.ubuntu.pool.ntp.org/'/etc/ntp.conf

# sed -i's/server1.ubuntu.pool.ntp.org/#server 1.ubuntu.pool.ntp.org/'/etc/ntp.conf

# sed -i's/server2.ubuntu.pool.ntp.org/#server 2.ubuntu.pool.ntp.org/'/etc/ntp.conf

# sed -i's/server3.ubuntu.pool.ntp.org/#server 3.ubuntu.pool.ntp.org/'/etc/ntp.conf

# sed -i 's/server ntp.ubuntu.com/#server ntp.ubuntu.com/'/etc/ntp.conf

# sed -i '/ntp.ubuntu.com/a\serverswift-proxy' /etc/ntp.conf

4）修改hosts文件

#vi/etc/hosts添加如下行：

172.26.188.226  swift-proxy
172.26.188.227  swift1
172.26.188.228  swift2
172.26.188.229  swift3

六、Keystone 安裝（安裝在proxy節點）

官方文檔

http://docs.openstack.org/trunk/openstack-compute/install/apt/content/keystone-concepts.html

安裝Keystone

#apt-getinstall -y keystone python-keyring

配置keystone

#/etc/keystone/default_catalog.templates添加如下內容

catalog.RegionOne.object_store.name = Swift Service
catalog.RegionOne.object_store.publicURL = http://172.26.188.226:8080/v1/AUTH_$(tenant_id)s
catalog.RegionOne.object_store.adminURL = http://172.26.188.226:8080/
catalog.RegionOne.object_store.internalURL = http://172.26.188.226:8080/v1/AUTH_$(tenant_id)s

生成隨機token：

#opensslrand -hex 10

修改keystone配置如下

#vi/etc/keystone/keystone.conf

admin_token =8a1438899a78df19bb3f

verbose = True

log_config =/etc/keystone/logging.conf

connection =mysql://keystone:keystone@172.26.188.226/keystone

# idle_timeout =200

增加如下行：

idle_timeout =200

min_pool_size =5

max_pool_size =10

pool_timeout =200

[ssl]

enable = False

       ####token_format = UUID

安裝mysql python-mysqldb

apt-get install mysql python-mysqldb

修改/etc/mysql/my.cnf

bind-address= 0.0.0.0

#mysql

mysql>CREATEDATABASE keystone;

mysql>GRANTALL ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'keystone';

mysql>GRANTALL ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'keystone';

mysql>quit;

keystone默認使用PKI tokens

#keystone-manage pki_setup

#sudo chown -R keystone:keystone/etc/keystone/* /var/log/keystone/keystone.log

# servicekeystone restart

#keystone-manage db_sync

創建初始tenants、users、roles：

下載腳本:

#wget https://raw.github.com/EmilienM/openstack-folsom-guide/master/scripts/keystone-data.sh

修改admin密碼（你將要設置的admin密碼）及token（與keystone.conf中一樣）后執行導入數據

創建services、endpoints

wget https://raw.github.com/EmilienM/openstack-folsom-guide/master/scripts/keystone-endpoints.sh

修改腳本中以下內容：

           # MySQL definitions

MYSQL_USER=keystone

MYSQL_DATABASE=keystone

MYSQL_HOST=172.26.188.226

MYSQL_PASSWORD=keystone

# Keystone definitions

KEYSTONE_REGION=RegionOne

SERVICE_TOKEN= 8a1438899a78df19bb3f

SERVICE_ENDPOINT="http://172.26.188.226:35319/v2.0"

# other definitions

MASTER="172.26.188.226"

SWIFT_MASTER="172.26.188.226"

為便于運行客戶端命令創建以下文件并運行：（創建認證文件并加載該文件，便于運行commands命令）

#vi /etc/profile

     export SERVICE_TOKEN= 8a1438899a78df19bb3f

export SERVICE_ENDPOINT=http://172.26.188.226:35319/v2.0

export OS_USERNAME=swift

export OS_PASSWORD=swift

export OS_TENANT_NAME=admin

export OS_AUTH_STRATEGY=keystone

export OS_AUTH_URL=http:// 172.26.188.226:5000/v2.0

查看keystone設置:

           # keystone user-list

# keystone user-role-list

# keystone role-list

# keystone service-list

# keystone tenant-list

# keystone endpoint-list

           # keystone user-role-list --user swift--tenant services

新建test1、test2用戶權限為Member

         #keystone user-create --name test1 --passadmin --email test1@test.com

#keystone user-create --name test2 --pass admin --email test1@test.com

# keystone user-role-add --user test1 --tenant services --roleMember

# keystone user-role-add --user test2 --tenant services --role Member

七、SwiftProxy Node安裝

安裝Storage proxy node packages

      # apt-getinstall -y swift-proxy memcached python-keystoneclient python-swiftclientswift-plugin-s3

修改memcached配置

     #sed -i '/-l/s/127.0.0.1/172.26.188.226/g' /etc/memcached.conf

# servicememcached restart

創建swift配置目錄（如果不存在）

#mkdir/etc/swift/

#chown -Rswift:swift /etc/swift/

#chown -Rswift:swift /var/cache/swift/

創建證書        //否則不支持https（這里我沒用https，所以可以不做）

#cd /etc/swift

#openssl req -new -x509 -nodes -out cert.crt -keyoutcert.key

創建/etc/swift/proxy-server.conf配置文件

#vi/etc/swift/proxy-server.conf

[DEFAULT]
#cert_file = /etc/swift/cert.crt
#key_file = /etc/swift/cert.key
bind_port = 8080
workers = 8
user = swift
[pipeline:main]
pipeline = catch_errors healthcheck cache authtoken keystoneauth proxy-logging proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
[filter:proxy-logging]
use = egg:swift#proxy_logging
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = Member,admin, swiftoperator
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
signing_dir = /var/cache/swift
auth_host = 172.26.188.226
auth_port = 35319
auth_protocol = http
auth_uri = http://172.26.188.226:5000
admin_tenant_name = services
admin_user = swift
admin_password = 111111
delay_auth_decision = 10
cache = swift.cache
 [filter:cache]
use = egg:swift#memcache
memcache_servers = 172.26.188.226:11211,172.26.188.227:11211,172.26.188.228:11211
                                                                                                                                                                                                                                                                                                                                                    
[filter:catch_errors]
use = egg:swift#catch_errors
[filter:healthcheck]
use = egg:swift#healthcheck

創建ring

#cd /etc/swift

swift-ring-builder account.builder create 18 3 1

swift-ring-builder container.builder create 18 3 1

swift-ring-builder object.builder create 18 3 1

參數詳細信息請參見官方ring部分，以下供參考。

18:用于指定分區數。分區數目為2的冪次，如18，則分區數是2的18次方。

3：文件副本數目

1：分區移動的最小時間間隔，單位是小時。

給每個node添加ring

swift-ring-builder account.builder add z1-172.26.188.227:6002/loop100

swift-ring-builder container.builder add z1-172.26.188.227:6001/loop100

swift-ring-builder object.builder add z1-172.26.188.227:6000/loop100

swift-ring-builder account.builder add z2-172.26.188.228:6002/loop100

swift-ring-builder container.builder add z2-172.26.188.228:6001/loop100

swift-ring-builder object.builder add z2-172.26.188.228:6000/loop100

swift-ring-builder account.builder add z3-172.26.188.229:6002/loop100

swift-ring-builder container.builder add z3-172.26.188.229:6001/loop100

swift-ring-builder object.builder add z3-172.26.188.229:6000/loop100

確認ring內容

swift-ring-builder /etc/swift/account.builder

swift-ring-builder /etc/swift/container.builder

swift-ring-builder /etc/swift/object.builder

Rebalancethe rings

swift-ring-builder account.builder rebalance

swift-ring-builder container.builder rebalance

swift-ring-builder object.builder rebalance

復制account.ring.gz, container.ring.gz, andobject.ring.gz到其他proxy Node及Storage Node

# scp *.ring.gz swift1:/etc/swift

# scp *.ring.gz swift2:/etc/swift

# scp *.ring.gz swift3:/etc/swift

所有node檢查/etc/swift權限，owner修改為swift

#chown -R swift:swift /etc/swift/

啟動proxy

#swift-init proxy start

測試認證

驗證整個存儲架構是否成功

#swift -V 2.0 -A http://172.26.188.226:5000/v2.0 -U admin-K admin stat

#swift -V 2.0 -A http://172.26.188.226:5000/v2.0-U swift:service -K admin stat

測試上傳文件到container

#swift -V 2.0 -A http://172.26.188.226:5000/v2.0-U admin -K admin upload myfiles cert.key

curl測試

curl-d '{"auth": {"tenantName": "admin","passwordCredentials":{"username": "admin","password": "admin"}}}' -H "Content-type:application/json" http://172.26.188.226:35319/v2.0/tokens | python-mjson.tool

curl-s -d"{\"auth\":{\"passwordCredentials\":{\"username\": \"swift\", \"password\":\"admin\"}, \"tenantName\": \"services\"}}"-H "Content-type: application/json" http://172.26.188.226:35319/v2.0/tokens

八、SwiftStorage Node安裝步驟

安裝Storage node packages

#apt-get install-y swift-account swift-container swift-object xfsprogs parted

準備磁盤

選取某一個磁盤分區做存儲，本例使用loop.硬盤小于2T可以使用fdisk

#parted /dev/sdb mklabel gpt

#parted /dev/sdb mkpart primary 0% 100%

#mkfs.xfs -isize=1024 /loop

#mkdir –p /srv/node/loop

#echo "/loop/srv/node/loop xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >>/etc/fstab

#mount /srv/node/loop

#chown -Rswift:swift /srv/node

以上若有多塊硬盤，需重復執行，可以使用一下循環。

for i in b cd;do

parted /dev/sd${i}mklabel gpt

parted/dev/sd${i} mkpart primary 0% 100%

      mkfs.xfs -i size=1024 /dev/sd${i}1

mkdir -p/srv/node/sd${i}1

echo"/dev/sd${i}1 /srv/node/sd${i}1 xfs noatime,nodiratime,nobarrier,logbufs=80 0" >> /etc/fstab

      mount /srv/node/sd${i}1

chown -R swift:swift/srv/node/sd${i}

     done

配置rsync

#vi /etc/rsyncd.conf

uid = swift

gid = swift

log file = /var/log/rsyncd.log

pid file = /var/run/rsyncd.pid

address = [STORAGE_NET_IP]                //修改為你的ip

[account]

max connections = 2

path = /srv/node/

read only = false

lock file = /var/lock/account.lock

[container]

max connections = 2

path = /srv/node/

read only = false

lock file = /var/lock/container.lock

[object]

max connections = 2

path = /srv/node/

read only = false

lock file = /var/lock/object.lock

#sed -i '/RSYNC_ENABLE/ s/false/true/g' /etc/default/rsync

#service rsync start

配置memcached

#sed-i '/-l/ s/127.0.0.1/[STORAGE_NET_IP]/g'/etc/memcached.conf//STORAGE_NET_IP修改為你的ip

# service memcached restart

修改Swift Storage Node配置

#vi /etc/swift/account-server.conf

[DEFAULT]

bind_ip = [STORAGE_NET_IP]                //修改為你的ip

workers = 2

[pipeline:main]

pipeline = account-server

[app:account-server]

use = egg:swift#account

[account-replicator]

[account-auditor]

[account-reaper]

#vi /etc/swift/container-server.conf

[DEFAULT]

bind_ip = [STORAGE_NET_IP]

workers = 2

[pipeline:main]

pipeline = container-server

[app:container-server]

use = egg:swift#container

[container-replicator]

[container-updater]

[container-auditor]

#vi /etc/swift/object-server.conf

[DEFAULT]

bind_ip = [STORAGE_NET_IP]

workers = 2

[pipeline:main]

pipeline = object-server

[app:object-server]

use = egg:swift#object

[object-replicator]

[object-updater]

[object-auditor]

[object-expirer]

啟動storage services        //沒有ring files，啟動會有錯誤。

      swift-init all start