doris初步使用

电玩女神 2023-03-02 13:23 3阅读 0赞

官网地址：[http://doris.incubator.apache.org/master/zh-CN/installing/compilation.html\#%E4%BD%BF%E7%94%A8-docker-%E5%BC%80%E5%8F%91%E9%95%9C%E5%83%8F%E7%BC%96%E8%AF%91%EF%BC%88%E6%8E%A8%E8%8D%90%EF%BC%89][http_doris.incubator.apache.org_master_zh-CN_installing_compilation.html_E4_BD_BF_E7_94_A8-docker-_E5_BC_80_E5_8F_91_E9_95_9C_E5_83_8F_E7_BC_96_E8_AF_91_EF_BC_88_E6_8E_A8_E8_8D_90_EF_BC_89]

1，编译安装过程就不说了，我没弄，让同事弄的。

2，与zeppelin的整合，实际很简单，就是zeppelin与mysql的整合，下面都是图，都是实操验证过的：

参考zeppelin的文档：

[https://github.com/apache/zeppelin/blob/master/docs/interpreter/flink.md\#paragraph-local-properties][https_github.com_apache_zeppelin_blob_master_docs_interpreter_flink.md_paragraph-local-properties]

[http://zeppelin.apache.org/docs/latest/interpreter/jdbc.html\#mysql][http_zeppelin.apache.org_docs_latest_interpreter_jdbc.html_mysql]

![20200729173919135.png][]

![20200729173835638.png][]

![20200729173954761.png][]

![20200729174007197.png][]

3，这个时候我们要在doris先创建表，在mysql客户端可以看的很清楚，当然在zeppelin也可以看到。

在zeppelin执行sql（我这里是2个模型）：

先创建的是doris的要写入的表，后面的表是对应的读取kafka的表。一个sink源，一个是source源.

>    
> ‘AGGREGATE模型’测试语句：
> 
> %mysql  
>   CREATE TABLE example\_db.testing\_table  
> (  
>     siteid INT DEFAULT '10',  
>     username VARCHAR(32) DEFAULT '',  
>     pv BIGINT SUM DEFAULT '0'  
> )  
> AGGREGATE KEY(siteid, username)  
> DISTRIBUTED BY HASH(siteid) BUCKETS 10  
> PROPERTIES("replication\_num" = "1");
> 
> CREATE ROUTINE LOAD example\_db.testing\_topic ON testing\_table  
> COLUMNS TERMINATED BY "|",  
> COLUMNS(siteid,username,pv)  
> PROPERTIES(       
>     "desired\_concurrent\_number"="1",  
>     "max\_batch\_interval" = "20",  
>     "max\_batch\_rows" = "300000",  
>     "max\_batch\_size" = "209715200",  
>     "strict\_mode" = "false"  
> )  
> FROM KAFKA  
>         (  
>             "kafka\_broker\_list" = "192.168.12.186:9092",  
>             "kafka\_topic" = "test\_topic",  
>             "kafka\_partitions" = "0",  
>             "kafka\_offsets" = "OFFSET\_BEGINNING"  
>         );  
>           
> ================  
>    
> ‘Uniq模型’测试语句：
> 
> %mysql  
>   CREATE TABLE example\_db.testing\_table  
> (  
>     siteid INT DEFAULT '10',  
>     username VARCHAR(32) DEFAULT '',  
>     pv BIGINT SUM DEFAULT '0'  
> )  
> AGGREGATE KEY(siteid, username)  
> DISTRIBUTED BY HASH(siteid) BUCKETS 10  
> PROPERTIES("replication\_num" = "1");
> 
> CREATE TABLE IF NOT EXISTS example\_db.testing\_table2  
> (  
>     \`siteid\` INT NOT NULL COMMENT "用户id",  
>     \`username\` VARCHAR(50) NOT NULL COMMENT "用户昵称",  
>     \`city\` VARCHAR(20) COMMENT "用户所在城市",  
>     \`pv\` BIGINT COMMENT "用户年龄",  
>     \`register\_time\` VARCHAR(50) COMMENT "用户注册时间"  
> )  
> UNIQUE KEY(\`siteid\`, \`username\`)  
> DISTRIBUTED BY HASH(siteid) BUCKETS 10  
> PROPERTIES("replication\_num" = "1");
> 
> CREATE ROUTINE LOAD example\_db.testing\_topic2 ON testing\_table2  
> COLUMNS TERMINATED BY "|",  
> COLUMNS(\`siteid\`,\`username\`,\`pv\`,\`register\_time\`)  
> PROPERTIES(       
>     "desired\_concurrent\_number"="1",  
>     "max\_batch\_interval" = "20",  
>     "max\_batch\_rows" = "5000",  
>     "max\_batch\_size" = "209715200",  
>     "strict\_mode" = "false"  
> )  
> FROM KAFKA  
>         (  
>             "kafka\_broker\_list" = "192.168.12.186:9092",  
>             "kafka\_topic" = "test\_topic2",  
>             "kafka\_partitions" = "0",  
>             "kafka\_offsets" = "OFFSET\_BEGINNING"  
>         );  
>

4，在执行sql后我们在zeppelin或者mysql客户端执行命令，查看相关的任务信息：

SHOW ROUTINE LOAD;

下图箭头，很标识的地方看一下就知道是啥意思了。红框很重要，如果这个任务不是running，那么看日志是什么错误，也要看‘errorRows’这个显示是不是为0，不为0代表数据消费error了，要看数据了。

![202007291744247.png][]

5，我这里的doris是远古版本了，要安装最新的版本，需要加入顶顶群或者微信群。

![20200729174649553.png][]

6，写的很简陋，最后是广告时间：

先跟鸡哥打个广告 ，博客地址: [https://me.csdn.net/weixin\_47482194][https_me.csdn.net_weixin_47482194]

写的博客很有水平的，上了几次官网推荐了。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70][]

[http_doris.incubator.apache.org_master_zh-CN_installing_compilation.html_E4_BD_BF_E7_94_A8-docker-_E5_BC_80_E5_8F_91_E9_95_9C_E5_83_8F_E7_BC_96_E8_AF_91_EF_BC_88_E6_8E_A8_E8_8D_90_EF_BC_89]: http://doris.incubator.apache.org/master/zh-CN/installing/compilation.html#%E4%BD%BF%E7%94%A8-docker-%E5%BC%80%E5%8F%91%E9%95%9C%E5%83%8F%E7%BC%96%E8%AF%91%EF%BC%88%E6%8E%A8%E8%8D%90%EF%BC%89
[https_github.com_apache_zeppelin_blob_master_docs_interpreter_flink.md_paragraph-local-properties]: https://github.com/apache/zeppelin/blob/master/docs/interpreter/flink.md#paragraph-local-properties
[http_zeppelin.apache.org_docs_latest_interpreter_jdbc.html_mysql]: http://zeppelin.apache.org/docs/latest/interpreter/jdbc.html#mysql
[20200729173919135.png]: /images/20230208/c9752cb6f563442c93cc2829e4f8f4bf.png
[20200729173835638.png]: /images/20230208/6989adfb81ea48d9b0e3727c6529e443.png
[20200729173954761.png]: /images/20230208/a5a9a83859014ea0afe8fd810e165f68.png
[20200729174007197.png]: /images/20230208/dde4c791323448d9a6d25c7cae726f9f.png
[202007291744247.png]: /images/20230208/a917cf0d3a2c43dab724e3c1185eeede.png
[20200729174649553.png]: /images/20230208/b6b33a1d5c3548e1a652b2f5308d178f.png
[https_me.csdn.net_weixin_47482194]: https://me.csdn.net/weixin_47482194
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70]: /images/20230208/849954c9180149e5b7501f7340e79b08.png