【大数据之数据仓库】安装部署GreenPlum集群

骑猪看日落 2021-09-29 11:10 388阅读 0赞

**本篇将向大家介绍如何快捷的安装部署GreenPlum测试集群，大家可以跟着我一块儿实践一把^\_^**

**1.主机资源**

申请2台网易云主机，操作系统必须是RedHat或者CentOS，配置尽量高一点。如果是sa统一初始化的物理机，这里不再描述，因为中间有很多操作（比如关闭puppet服务等等）

**2.下载安装包**

下载链接： [https://network.pivotal.io/products/pivotal-gpdb][https_network.pivotal.io_products_pivotal-gpdb]，需要先注册才能下载，建议用非163邮箱注册；（下载困难可以泡泡找我）

**3.开始安装**

以下操作如果没有特殊说明，则需要在2台主机上同时操作！

**3.1修改系统参数：**

为简化操作，部分参数不再这里描述。

[root@hzayq-helf-xxx ~]# chkconfig iptables off             关闭防火墙
    [root@hzayq-helf-xxx ~]# chkconfig --list iptables          查看防火墙
    
    [root@hzayq-helf-xxx ~]# echo "MaxStartups 300:30:1000" >> /etc/ssh/sshd_config  提高ssh并发
    [root@hzayq-helf-xxx ~]# service sshd restart
    
    [root@hzayq-helf-xxx ~]# vi /etc/sysctl.conf                 增加（不是覆盖）以下参数
    kernel.sysrq = 1
    xfs_mount_options = rw,noatime,inode64,allocsize=16m
    kernel.shmmax = 500000000
    kernel.shmmni = 4096
    kernel.shmall = 4000000000
    kernel.sem = 250 512000 100 2048
    kernel.sysrq = 1
    kernel.core_uses_pid = 1
    kernel.msgmnb = 65536
    kernel.msgmax = 65536
    kernel.msgmni = 2048
    net.ipv4.tcp_syncookies = 1
    net.ipv4.ip_forward = 0
    net.ipv4.conf.default.accept_source_route = 0
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_max_syn_backlog = 4096
    net.ipv4.conf.all.arp_filter = 1
    net.ipv4.ip_local_port_range = 1025 65535
    net.core.netdev_max_backlog = 10000
    net.core.rmem_max = 2097152
    net.core.wmem_max = 2097152
    vm.overcommit_memory = 2
    [root@hzayq-helf-xxx ~]# sysctl -p                           生效
    
    [root@hzayq-helf-xxx ~]# vi /etc/security/limits.conf        增加以下参数
    * soft nofile 65536
    * hard nofile 65536
    * soft nproc 131072
    * hard nproc 131072

**3.2安装unzip：**

[root@hzayq-helf-xxx ~]# yum install unzip

**3.3安装GreenPlum：**

[root@hzayq-helf-xxx ~]# ./greenplum-db-4.3.11.1-build-1-rhel5-x86_64.bin

**3.4创建普通用户：**

[root@hzayq-helf-xxx ~]# useradd -d /home/gpadmin gpadmin

**3.5生成普通用户的公私钥：**

[root@hzayq-helf-xxx ~]# su - gpadmin
    [gpadmin@hzayq-helf-xxx ~]# ssh-keygen -t rsa -P ''

**3.6配置/etc/hosts：**

[gpadmin@hzayq-helf-xxx ~]# hostname                        获取主机名
    [gpadmin@hzayq-helf-xxx ~]# exit                            切换回root
    [root@hzayq-helf-xxx ~]# vi /etc/hosts                      添加ip和主机名的映射
    比如：
    10.171.160.142  hzayq-helf-xxx
    10.171.160.143  hzayq-helf-yyy

**3.7设置免密登陆：**

[gpadmin@hzayq-helf-xxx ~]# cat .ssh/id_rsa.pub             查看公钥
    ... ...                                                     切换到其他的主机
    [gpadmin@hzayq-helf-yyy ~]# vi .ssh/authorized_keys         创建认证文件，并把所有主机的公钥粘贴进去，记住把自己的公钥也贴进去哦
    [gpadmin@hzayq-helf-yyy ~]# chmod 600 .ssh/authorized_keys  修改认证文件权限属性
    ... ...                                                     切换回原来的机器
    [gpadmin@hzayq-helf-xxx ~]# ssh hzayq-helf-yyy              验证免密登陆是否OK

**3.8创建数据目录：**

[gpadmin@hzayq-helf-xxx ~]# mkdir -p gp/data/master          创建master目录
    [gpadmin@hzayq-helf-xxx ~]# mkdir -p gp/data/primary1        创建第1个primary目录
    [gpadmin@hzayq-helf-xxx ~]# mkdir -p gp/data/primary2        创建第2个primary目录
    [gpadmin@hzayq-helf-xxx ~]# mkdir -p gp/data/mirror1         创建第1个mirror目录
    [gpadmin@hzayq-helf-xxx ~]# mkdir -p gp/data/mirror2         创建第2个mirror目录

**3.9编辑环境变量：**

[gpadmin@hzayq-helf-xxx ~]# echo "source /usr/local/greenplum-db/greenplum_path.sh" >> .bashrc
    [gpadmin@hzayq-helf-xxx ~]# source .bashrc

**3.10初始化GreenPlum系统：**

任选1台主机做master，1台做standby master，使用 **附件中的2个配置文件**， **需要修改下gpinitsystem\_config中MASTER\_HOSTNAME字段为master节点**

[gpadmin@hzayq-helf-xxx ~]# gpinitsystem -c gpinitsystem_config -h hostfile_gpinitsystem -s hzayq-helf-yyy

一路Y即可安装成功！

**3.11最后，再编辑下环境变量：**

[gpadmin@hzayq-helf-xxx ~]# vi ~/.bashrc    增加以下两个环境变量
    export MASTER_DATA_DIRECTORY=/home/gpadmin/gp/data/master/gpseg-1
    export PGPORT=5432
    [gpadmin@hzayq-helf-xxx ~]# source ~/.bashrc

**3.12使用psql接入：**

[gpadmin@hzayq-helf-xxx ~]# psql -d template1

附件

[test.zip][]

**欢迎来到GreenPlum的世界！^\_^**

看这里：

《 [【大数据之数据仓库】选型流水记][Link 1]》

本文来自网易云社区，经作者何李夫授权发布。

原文地址：[【大数据之数据仓库】安装部署GreenPlum集群][GreenPlum]

更多网易研发、产品、运营经验分享请访问[网易云社区][Link 2]。

转载于:https://www.cnblogs.com/163yun/p/9289586.html

[https_network.pivotal.io_products_pivotal-gpdb]: https://network.pivotal.io/products/pivotal-gpdb
[test.zip]: http://nos.netease.com/knowledge/7473ebcf-5eed-4b26-8fa5-7c9c4344b28a?download=test.zip
[Link 1]: https://sq.163yun.com/blog/article/174995941069086720
[GreenPlum]: https://sq.163yun.com/blog/article/174996403499274240
[Link 2]: https://link.zhihu.com/?target=https%3A//sq.163yun.com/blog