hbase入门（一）

╰半夏微凉° 2021-12-13 14:11 462阅读 0赞

**目录**

HBase介绍

概念

Column Family（简称CF）

Qualifier

Table

RowName

Value

安装

注意

启动

监控

网上教程

Java Api

HBaseAdmin

HTableDescriptor

SpringBoot整合HBase

HbaseTemplate

pom.xml

config

或者spring配置

HbaseTemplate方法

demo地址

HbaseTemplate改写

更加详细的资料

--------------------

# HBase介绍 #

HBase基于HDSF文件的NoSQL数据库，单表数据量达到百万，十亿的数量，面向列的数据库，虽然说查出来是行，但是物理上是按列去储存。为null的化不进行储存，节省空间。

# 概念 #

## Column Family（简称CF） ##

为啥有这么一个设计？有点类似我们平时的查询都不会\*把所有都查出来，只查询部分字段的感jio

## Qualifier ##

在CF下的列，类似二级列

## Table ##

顾名思义：表

## RowName ##

行名

## Value ##

值

![20190707222311939.JPG][]

上面的图会直观的展示上面这些概念

# 安装 #

个人使用docker安装

需要安装jdk，hbase，docker具体可以参照网上

hbase依赖内置的zookeeper，也可以使用外面的zookeeper，需要配置，以及可以配置白名单，比如ip

## 注意 ##

docker域名解析不了的问题，就是docker启动容器的时候命名host，导致应用解析不了host名称。

相应的解决方案：[https://blog.csdn.net/weixin\_38336658/article/details/94733138][https_blog.csdn.net_weixin_38336658_article_details_94733138]

# 启动 #

docker run -d -h myhbase -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9095:9095 -p 16000:16000 -p 16010:16010 -p 16201:16201 -p 16301:16301 --name hbase1.3 harisekhon/hbase:1.3

# 监控 #

浏览器访问http://ip:16010/master-status

# 网上教程 #

[https://www.yiibai.com/hbase][https_www.yiibai.com_hbase]

# Java Api #

## HBaseAdmin ##

主要是创建表，修改表的有效状态等等

## HTableDescriptor ##

表描述，可以添加表CF

HTableDescriptor desc = hBaseAdmin.getTableDescriptor(tableName);
    desc.addFamily(column);

# SpringBoot整合HBase #

可以说以HBaseTemplate去调用的话很粗糙，比较复杂的查询都得自己重新包装

## HbaseTemplate ##

### pom.xml ###

<dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-client</artifactId>
                <version>1.4.4</version>
                <exclusions>
                    <exclusion>
                        <groupId>javax.servlet</groupId>
                        <artifactId>servlet-api</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>

### config ###

application.yml

hbase:
      config:
        hbase.zookeeper.quorum: xx
        hbase.zookeeper.port: 2181

xx为hbase注册zookeeper的ip地址

/**
     * 用来读取hbase配置文件
     */
    @ConfigurationProperties(prefix = "hbase")
    public class HBaseProperties {
    
    
        private Map<String,String> config;
    
    
        public Map<String, String> getConfig() {
            return config;
        }
    
        public void setConfig(Map<String, String> config) {
            this.config = config;
        }
    }

import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.client.HBaseAdmin;
    import org.springframework.boot.context.properties.EnableConfigurationProperties;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.hadoop.hbase.HbaseTemplate;
    
    import java.io.IOException;
    import java.util.Map;
    import java.util.Set;
    
    /**
     * 把配置加入到hbaseTemplate中，交给spring容器
     * 再把hbaseAdmin交给spring容器
     */
    @Configuration
    @EnableConfigurationProperties(HBaseProperties.class)
    public class HBaseConfig {
    
    
        private final HBaseProperties properties;
    
        public HBaseConfig(HBaseProperties properties){
            this.properties =properties;
        }
    
        @Bean
        public HbaseTemplate hbaseTemplate(){
            HbaseTemplate hbaseTemplate =new HbaseTemplate();
            hbaseTemplate.setConfiguration(configuration());
            hbaseTemplate.setAutoFlush(true);
            return hbaseTemplate;
        }
    
        @Bean
        public HBaseAdmin hbaseAdmin() throws IOException {
            return  new HBaseAdmin(hbaseTemplate().getConfiguration());
        }
    
        private org.apache.hadoop.conf.Configuration configuration() {
            org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();
    
            Map<String, String> config = properties.getConfig();
    
            Set<String> keySet = config.keySet();
            for (String key : keySet) {
                configuration.set(key,config.get(key));
            }
            return configuration;
        }
    }

### 或者spring配置 ###

Configuration config = HBaseConfiguration.create();
            config.set("hbase.zookeeper.quorum", zookeeper);
            try {
                conn = ConnectionFactory.createConnection(config);
            } catch (IOException e) {
                e.printStackTrace();
            }

zookeeper配置为hbase注册到ookeeper的地址，如果没有修改则为内置的，即hbase ip地址

### HbaseTemplate方法 ###

get，find，get等等

**创建表**

@Autowired
        private HBaseAdmin hBaseAdmin;
    
        /**
         * 创建表
         * @throws IOException
         */
        @Test
        public void createTable() throws IOException {
            HTableDescriptor desc=new HTableDescriptor(TableName.valueOf("Student"));
            desc.addFamily(new HColumnDescriptor("info".getBytes()));
            if(hBaseAdmin.tableExists("Student")){
                System.err.println("===========================table====exists====================================");
            }
            hBaseAdmin.createTable(desc);
        }

**插入数据**

/**
         * 添加数据
         */
        @Test
        public void insert()  {
            hbaseTemplate.put("Student","row5","info","dajitui","111".getBytes());
            hbaseTemplate.put("Student","row2","info","score","99".getBytes());
            hbaseTemplate.put("Student","row3","info","name","和凯凯".getBytes());
        }

**查询所有数据**

/**
         * 查询所有数据
         */
        @Test
        public void findAll(){
            Object obj= hbaseTemplate.find("Student",new Scan(),(Result result, int i)->{
                Cell[] cells = result.rawCells();
                List<HBaseBean> list=new ArrayList<>();
                    for (Cell cell : cells) {
                       String columnFamily= new String(CellUtil.cloneFamily(cell));
                        String rowName = new String(CellUtil.cloneRow(cell));
                        String key = new String(CellUtil.cloneQualifier(cell ));
                        String value = new String(CellUtil.cloneValue(cell));
                        list.add(new HBaseBean(columnFamily,rowName,key,value));
                    }
                    return list;
            });
            String json = JSONArray.toJSONString(obj);
            System.out.println(json);
        }

**复杂查询**

e.g.where a= a and b= b,这个要修改上面Scan的查询规则具体百度哦

**查询一行的数据**

@Test
        public void findOntColumn(){
             Object result=hbaseTemplate.get("Student", "row1",new RowMapper<Map<String,Object>>(){
                @Override
                public Map<String, Object> mapRow(Result result, int i) throws Exception {
                    List<Cell> ceList =   result.listCells();
                    Map<String,Object> map = new HashMap<String, Object>();
                    if(ceList!=null&&ceList.size()>0){
                        for(Cell cell:ceList){
                            String key = new String(CellUtil.cloneQualifier(cell ));
                            String value = new String(CellUtil.cloneValue(cell));
                            map.put(key,value);
                        }
                    }
                    return map;
                }
            });
            System.out.println(JSON.toJSONString(result));
        }

**数据转Object**

Object object = hbaseTemplate.find("Student", new Scan(), (Result result, int i) -> {
                UserInfo userInfo=new UserInfo();
                BeanWrapper beanWrapper = PropertyAccessorFactory.forBeanPropertyAccess(userInfo);
                List<Cell> ceList = result.listCells();
                for (Cell cellItem : ceList) {
                    String cellName = new String(CellUtil.cloneQualifier(cellItem));
                    beanWrapper.setPropertyValue(cellName, new String(CellUtil.cloneValue(cellItem)));
                }
                return userInfo;
            });
            System.out.println(object);

**添加列**

/**
         * 添加列,族(family)
         * @throws IOException
         */
        @Test
        public void addFamily() throws IOException {
            TableName tableName = TableName.valueOf("Student");
            hBaseAdmin.disableTable(tableName);
            HTableDescriptor desc = hBaseAdmin.getTableDescriptor(tableName);
            HColumnDescriptor column=new HColumnDescriptor("abc");
            desc.addFamily(column);
            hBaseAdmin.modifyTable("Student",desc);
            hBaseAdmin.enableTable(tableName);
            hBaseAdmin.close();
        }

**批量插入**

@Override
        public void insertList(String tableName, List objectList) throws IOException {
            if (!CollectionUtils.isEmpty(objectList)) {
    
                Table table = conn.getTable(TableName.valueOf(tableName));
                Iterator iterator = objectList.iterator();
                while (iterator.hasNext()) {
                    Object object = iterator.next();
                    try {
                        Class<?> class1 = Class.forName(object.getClass().getName());
                        Field[] fields = class1.getDeclaredFields();
                        List<Put> puts = new ArrayList<>(fields.length);
                        for (Field field : fields) {
                            //打开私有访问
                            field.setAccessible(true);
                            //获取属性
                            String name = field.getName();
                            //获取属性值
                            Object value = field.get(object);
                            if (value != null) {
                                Put put1 = new Put(Bytes.toBytes(Long.toBinaryString(snowflakeIdWorker.nextId())));
                                put1.addColumn(Bytes.toBytes("info"), Bytes.toBytes("yy"), JSON.toJSONString(name).getBytes());
                                put1.addColumn(Bytes.toBytes("info"), Bytes.toBytes("yy"), JSON.toJSONString(value).getBytes());
                                puts.add(put1);
                            }
                        }
                        if (!CollectionUtils.isEmpty(puts)) {
                            table.put(puts);
                        }
                    } catch (ClassNotFoundException e) {
                        e.printStackTrace();
                    } catch (IllegalAccessException e) {
                        e.printStackTrace();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }
    
        }

private static Connection conn;
    
        //zookeeper集群
        private static String zookeeper = "xxx";
    
        public static void init() {
            Configuration config = HBaseConfiguration.create();
            config.set("hbase.zookeeper.quorum", zookeeper);
            try {
                conn = ConnectionFactory.createConnection(config);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

都放在同一个列，snowflakeIdWorker.nextId()数据随意设置即可

批量插入必须使用htable才能实现所以比较麻烦

# demo地址 #

copy网上的，进行修改，重写hbasetemplate

[下载地址][Link 1]

# HbaseTemplate改写 #

参照[https://blog.csdn.net/github\_34560747/article/details/52708374][https_blog.csdn.net_github_34560747_article_details_52708374]

# 更加详细的资料 #

[https://yq.aliyun.com/articles/609458][https_yq.aliyun.com_articles_609458]

[20190707222311939.JPG]: /images/20211213/945610d772f14ba4bc1a2ba7b351be94.png
[https_blog.csdn.net_weixin_38336658_article_details_94733138]: https://blog.csdn.net/weixin_38336658/article/details/94733138
[https_www.yiibai.com_hbase]: https://www.yiibai.com/hbase
[Link 1]: https://github.com/dajitui/hbaseTest
[https_blog.csdn.net_github_34560747_article_details_52708374]: https://blog.csdn.net/github_34560747/article/details/52708374
[https_yq.aliyun.com_articles_609458]: https://yq.aliyun.com/articles/609458