仅供参考 这里要注意python的版本 软件源的匹配https://blog.csdn.net/cojn52/article/details/85793902服务器端1 准备镜像镜像centos7.5 18042 关闭防火墙 selinuxsystemctl status firewalldystemctl stop firewalldgetenforce #查看selinux 状态setenforce 0 #临时关闭seLinux也可以永久关闭selinux 但是需要重启3 设置主机名4 安装 Ansible git vim5 从git 上下载ceph-ansible6 进入ceph-ansible 目录git taggit checkout v3.2.0git branch7 在ceph-ansible 目录vim hosts8 给免密操作ssh-keygen8在ceph-ansible 目录 进入group_vars 编辑 all.yml9 添加网络源在/etc/yum.repo.d/在原有基础上添加网络源9.1、 这个是ceph的yum源 前提是必须要有epel源解决依赖问题[rootceph-1 yum.repos.d]#vim ceph_stable.repo[ceph_stable]baseurl http://mirrors.163.com/ceph/rpm-luminous/el7/$basearchgpgcheck 1gpgkey http://mirrors.163.com/ceph/keys/release.ascname Ceph Stable repo9.2、[rootceph-1 yum.repos.d]# vim elrepo.repo### Name: ELRepo.org Community Enterprise Linux Repository for el7### URL: http://elrepo.org/[elrepo]nameELRepo.org Community Enterprise Linux Repository - el7baseurlhttp://elrepo.org/linux/elrepo/el7/$basearch/http://mirrors.coreix.net/elrepo/elrepo/el7/$basearch/http://mirror.rackspace.com/elrepo/elrepo/el7/$basearch/http://repos.lax-noc.com/elrepo/elrepo/el7/$basearch/http://mirror.ventraip.net.au/elrepo/elrepo/el7/$basearch/mirrorlisthttp://mirrors.elrepo.org/mirrors-elrepo.el7enabled1gpgcheck1gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.orgprotect0[elrepo-testing]nameELRepo.org Community Enterprise Linux Testing Repository - el7baseurlhttp://elrepo.org/linux/testing/el7/$basearch/http://mirrors.coreix.net/elrepo/testing/el7/$basearch/http://mirror.rackspace.com/elrepo/testing/el7/$basearch/http://repos.lax-noc.com/elrepo/testing/el7/$basearch/http://mirror.ventraip.net.au/elrepo/testing/el7/$basearch/mirrorlisthttp://mirrors.elrepo.org/mirrors-elrepo-testing.el7enabled0gpgcheck1gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.orgprotect0[elrepo-kernel]nameELRepo.org Community Enterprise Linux Kernel Repository - el7baseurlhttp://elrepo.org/linux/kernel/el7/$basearch/http://mirrors.coreix.net/elrepo/kernel/el7/$basearch/http://mirror.rackspace.com/elrepo/kernel/el7/$basearch/http://repos.lax-noc.com/elrepo/kernel/el7/$basearch/http://mirror.ventraip.net.au/elrepo/kernel/el7/$basearch/mirrorlisthttp://mirrors.elrepo.org/mirrors-elrepo-kernel.el7enabled0gpgcheck1gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.orgprotect0[elrepo-extras]nameELRepo.org Community Enterprise Linux Extras Repository - el7baseurlhttp://elrepo.org/linux/extras/el7/$basearch/http://mirrors.coreix.net/elrepo/extras/el7/$basearch/http://mirror.rackspace.com/elrepo/extras/el7/$basearch/http://repos.lax-noc.com/elrepo/extras/el7/$basearch/http://mirror.ventraip.net.au/elrepo/extras/el7/$basearch/mirrorlisthttp://mirrors.elrepo.org/mirrors-elrepo-extras.el7enabled0gpgcheck1gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.orgprotect09.3、[rootceph-1 yum.repos.d]# vim epel.repo[epel]nameExtra Packages for Enterprise Linux 7 - $basearch#baseurlhttp://download.fedoraproject.org/pub/epel/7/$basearchmetalinkhttps://mirrors.fedoraproject.org/metalink?repoepel-7arch$basearchfailovermethodpriorityenabled1gpgcheck0gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7[epel-debuginfo]nameExtra Packages for Enterprise Linux 7 - $basearch - Debug#baseurlhttp://download.fedoraproject.org/pub/epel/7/$basearch/debugmetalinkhttps://mirrors.fedoraproject.org/metalink?repoepel-debug-7arch$basearchfailovermethodpriorityenabled0gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7gpgcheck1[epel-source]nameExtra Packages for Enterprise Linux 7 - $basearch - Source#baseurlhttp://download.fedoraproject.org/pub/epel/7/SRPMSmetalinkhttps://mirrors.fedoraproject.org/metalink?repoepel-source-7arch$basearchfailovermethodpriorityenabled0gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7gpgcheck19.4、 [rootceph-1 yum.repos.d]# vim epel-testing.repo[epel-testing]nameExtra Packages for Enterprise Linux 7 - Testing - $basearch#baseurlhttp://download.fedoraproject.org/pub/epel/testing/7/$basearchmetalinkhttps://mirrors.fedoraproject.org/metalink?repotesting-epel7arch$basearchfailovermethodpriorityenabled0gpgcheck1gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7[epel-testing-debuginfo]nameExtra Packages for Enterprise Linux 7 - Testing - $basearch - Debug#baseurlhttp://download.fedoraproject.org/pub/epel/testing/7/$basearch/debugmetalinkhttps://mirrors.fedoraproject.org/metalink?repotesting-debug-epel7arch$basearchfailovermethodpriorityenabled0gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7gpgcheck1[epel-testing-source]nameExtra Packages for Enterprise Linux 7 - Testing - $basearch - Source#baseurlhttp://download.fedoraproject.org/pub/epel/testing/7/SRPMSmetalinkhttps://mirrors.fedoraproject.org/metalink?repotesting-source-epel7arch$basearchfailovermethodpriorityenabled0gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7gpgcheck110安装依赖包 不安装会报错丢失notarioyum -y install epel-releaseyum -y install python-pippip install notario11 部署ceph在ceph-ansible目录下执行ansible-playbook site.yml.sample -i hosts跑完之后12 部署osd12.1 安装ceph-osd安装包yum install ceph-osd –y12.2 查看磁盘12.3 格式化磁盘ceph-disk prepare --bluestore /dev/sdbceph-disk prepare --bluestore /dev/sdcceph-disk prepare --bluestore /dev/sdd12.4 给ceph添加osd盘ceph-disk prepare --bluestore /dev/sdbceph-disk prepare --bluestore /dev/sdcceph-disk prepare --bluestore /dev/sd13 ceph部署完成14 创建rbd 块14.1 ceph osd pool create rbd 512 创建rbd14.2 rbd create volume -p rbd --size 1T 指定大小15 TCMU实现ISCSI接口注大部分对接失败的原因就是没有通过tcmu 实现iscsi的借口15.1 在tmp目录下载tcmp代码cd /tmp/git clone https://github.com/open-iscsi/tcmu-runner.git15.2 安装tcmu(1) cd tcmu-runner/(2) ./extra/install_dep.sh(3) cmake -Dwith-glfsfalse -Dwith-qcowfalse -DSUPPORT_SYSTEMDON -DCMAKE_INSTALL_PREFIX/usr .(4) make(5) make install(6) 启动服务tcmusystemctl status tcmu-runnersystemctl start tcmu-runnersystemctl status tcmu-runnersystemctl enable tcmu-runner16 安装ISCSI targetcliyum install targetcli –y17 配置iscsi 服务器端[rootceph-1 ~]# targetclitargetcli shell version 2.1.fb46Copyright 2011-2013 by Datera, Inc and others.For help on commands, type help./ cd backstores/user:rbd//backstores/user:rbd create cfgstringrbd/volume namedisk size1T/backstores/user:rbd cd // cd iscsi/iscsi create iqn.2019-01.redhat.ceph/iscsi cd iqn.2019-01.redhat.ceph/tpg1/luns/iscsi/iqn.20...eph/tpg1/luns create /backstores/user:rbd/disk/iscsi/iqn.20...eph/tpg1/luns cd // cd iscsi/iqn.2019-01.redhat.ceph/tpg1//iscsi/iqn.20...hat.ceph/tpg1 set attribute authentication0 demo_mode_write_protect0/iscsi/iqn.20...hat.ceph/tpg1 cd // cd iscsi/iqn.2019-01.redhat.ceph/tpg1/portals/ # 保持不变/iscsi/iqn.20.../tpg1/portals exit18、安装iscsi 软件包yum install iscsi-initiator-utils19、查看是否共享成功iscsiadm -m discovery -t sendtargets -p 10.22.20.25客户端通过ssh或者界面console进入平台检查是否安装iscsi安装包如果没有进行安装yum install iscsi-initiator-utils -y3、查看是否能查看到共享成功iscsiadm -m discovery -t sendtargets -p 10.22.20.25修改vim /etc/iscsi/initiatorname.iscsi配置文件并重启服务iscsiadm -m discovery -t sendtargets -p 10.22.20.25的输出InitiatorName10.22.20.25:3260,1 iqn.2019-01.redhat.cephInitiatorAliasxenserversystemctl restart iscsid通过xencenter界面挂载iscsi至此客户端挂载成功下面测试是否可用