presto单机单节点安装搭建与案例

爱被打了一巴掌 2022-02-16 01:06 1045阅读 0赞

**一、什么是presto**  
Presto是Facebook开发的数据查询引擎，可对250PB以上的数据进行快速地交互式分析。  
**二、搭建presto**  
首先Centos 7.X 系统，JDK采用的Oracle的1.8.X的版本。presto的版本是采用最新版0.218，Python为版本Python 2.4+  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 1]  
presto下载地址：  
[http://prestodb.github.io/resources.html][http_prestodb.github.io_resources.html]  
[https://repo1.maven.org/maven2/com/facebook/presto/presto-server/][https_repo1.maven.org_maven2_com_facebook_presto_presto-server]  
jdk下载地址：  
[https://www.oracle.com/technetwork/cn/java/javase/downloads/jdk8-downloads-2133151-zhs.html][https_www.oracle.com_technetwork_cn_java_javase_downloads_jdk8-downloads-2133151-zhs.html]  
**三、安装与解压**  
1、首先安装jdk，安装步骤：  
[https://jingyan.baidu.com/article/9f7e7ec0f8c26b6f28155433.html][https_jingyan.baidu.com_article_9f7e7ec0f8c26b6f28155433.html]  
2、解压配置presto

tar zxvf presto-server-0.218.tar.gz

![在这里插入图片描述][20190421111251496.png]3、创建preston的日志存放目录

mkdir prestoData

![在这里插入图片描述][20190421111352138.png]  
4、创建配置文件

mkdir etc
    cd etc
    touch node.properties jvm.config config.properties log.properties
    mkdir catalog
    cd catalog
    touch jmx.properties

![在这里插入图片描述][2019042111190058.png]  
![在这里插入图片描述][20190421112305169.png]  
5、编写配置文件config.properties、jvm.config、log.properties、node.properties

vi config.properties
     
    coordinator=true
    node-scheduler.include-coordinator=false     #单机版的此处为true
    query.max-memory=8GB
    query.max-memory-per-node=1GB
    http-server.http.port=9999    #与discovery.uri的端口一样
    discovery-server.enabled=true
    discovery.uri=http://192.168.252.152:9999    #ip为本机外网地址

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 2]

vi jvm.config
    -server
    -Xmx8G
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=32M
    -XX:+UseGCOverheadLimit
    -XX:+ExplicitGCInvokesConcurrent
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:+ExitOnOutOfMemoryError

![在这里插入图片描述][2019042111301029.png]

vi log.properties
    book.presto=INFO

![在这里插入图片描述][20190421113040704.png]

vi node.properties
    node.environment=presto
    node.id=presto_f1 #名字随便 但是不要重复
    node.data-dir=/home/presto/prestoData  #日志路径

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 3]

vi launcher
    export JAVA_HOME=/usr/local/java/jdk1.8.0_171
    export PATH=$JAVA_HOME:$PATH

![在这里插入图片描述][20190421113505441.png]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 4]  
**四、运行/查看状态/停止**

启动

./launcher start

![在这里插入图片描述][20190421113756599.png]  
查看状态

./launcher status

![在这里插入图片描述][20190421113920270.png]  
停止

./launcher stop

访问

http://192.168.252.152:9999/ui/

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 5]五、创建字节点  
1、与创建主节点一样，但是需要改一下配置如下：

vi config.properties
    coordinator=false
    http-server.http.port=9999
    query.max-memory=1GB
    discovery.uri=http://192.168.252.152:9999

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 6]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 7]**六、springboot+presto**  
pom.xml

<?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.1.4.RELEASE</version>
            <relativePath/> 
        </parent>
        <groupId>com.citydo</groupId>
        <artifactId>springboot_presto</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>springboot_presto</name>
        <description>Demo project for Spring Boot</description>
    
        <properties>
            <java.version>1.8</java.version>
        </properties>
    
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-jdbc</artifactId>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>
    
            <dependency>
                <groupId>org.postgresql</groupId>
                <artifactId>postgresql</artifactId>
                <version>9.4.1212</version>
            </dependency>
    
            <dependency>
                <groupId>mysql</groupId>
                <artifactId>mysql-connector-java</artifactId>
                <version>6.0.5</version>
            </dependency>
    
            <dependency>
                <groupId>com.facebook.presto</groupId>
                <artifactId>presto-jdbc</artifactId>
                <version>0.217</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-autoconfigure</artifactId>
                <version>1.5.14.RELEASE</version>
            </dependency>
    
            <dependency>
                <groupId>com.github.pagehelper</groupId>
                <artifactId>pagehelper</artifactId>
                <version>4.2.0</version>
            </dependency>
            <dependency>
                <groupId>org.mybatis</groupId>
                <artifactId>mybatis-spring</artifactId>
                <version>1.3.2</version>
            </dependency>
            <dependency>
                <groupId>org.mybatis</groupId>
                <artifactId>mybatis</artifactId>
                <version>3.4.6</version>
            </dependency>
    
    
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    
    </project>

application.yml

postgresql:
      spring:
        datasource:
          pa:
            driver-class-name: org.postgresql.Driver
            url: jdbc:postgresql://localhost:5432/postgres
            username: root
            password: 123456
    
    presto:
      spring:
        datasource:
          marketing:
            driver-class-name: com.facebook.presto.jdbc.PrestoDriver
            url: jdbc:presto://localhost:8080/hive/default?user=databus
            #url: jdbc:presto://localhost:8080/hive/default?root=databus
    
    mybatis:
      type-aliases-package: com.citydo.springboot_presto.domain.domain
      #mapper-location: classpath:mapper/*.xml

package com.citydo.springboot_presto.config;
    
    
    
    import org.apache.ibatis.session.SqlSessionFactory;
    import org.mybatis.spring.SqlSessionFactoryBean;
    import org.mybatis.spring.SqlSessionTemplate;
    import org.mybatis.spring.annotation.MapperScan;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.beans.factory.annotation.Qualifier;
    import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
    import org.springframework.boot.context.properties.ConfigurationProperties;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
    import org.springframework.jdbc.datasource.DataSourceTransactionManager;
    
    import javax.sql.DataSource;
    /**
     * @ClassName DataSourcePrestoConfig
     * @Description 配置presto数据源
     * @Author qixin
     * @Date 2018/7/6 10:14
     * Version 1.0
     **/
    /*marketing数据库*/
    @Configuration
    @MapperScan(basePackages = "com.citydo.springboot_presto.dao.test", sqlSessionTemplateRef  = "marketingSqlSessionTemplate")
    public class DataSourcePrestoConfig {
        private final static Logger logger = LoggerFactory.getLogger(DataSourcePrestoConfig.class);
        @Bean(name = "marketingDataSource")
        @ConfigurationProperties(prefix = "presto.spring.datasource.marketing")
        public DataSource testDataSource() {
            return DataSourceBuilder.create().build();
        }
    
        @Bean(name = "marketingSqlSessionFactory")
        public SqlSessionFactory testSqlSessionFactory(@Qualifier("marketingDataSource") DataSource dataSource) throws Exception {
            SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
            bean.setDataSource(dataSource);
            /*加载mybatis全局配置文件*/
            //bean.setConfigLocation(new PathMatchingResourcePatternResolver().getResource("classpath:mybatis/mybatis-config.xml"));
            /*加载所有的mapper.xml映射文件*/
    
            bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath:mapper/test/*.xml"));
            logger.info("加载presto数据库连接......");
            return bean.getObject();
        }
    
        @Bean(name = "marketingTransactionManager")
        public DataSourceTransactionManager testTransactionManager(@Qualifier("marketingDataSource") DataSource dataSource) {
            return new DataSourceTransactionManager(dataSource);
        }
    
        @Bean(name = "marketingSqlSessionTemplate")
        public SqlSessionTemplate testSqlSessionTemplate(@Qualifier("marketingSqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception {
            return new SqlSessionTemplate(sqlSessionFactory);
        }
    
    
    }

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 8]  
参考:[https://blog.csdn.net/Alice\_qixin/article/details/80976245][https_blog.csdn.net_Alice_qixin_article_details_80976245]  
参考:[https://blog.csdn.net/weixin\_43704393/article/details/87275671][https_blog.csdn.net_weixin_43704393_article_details_87275671]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20190421105411242.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 1]: /images/20220216/c385d3e93bc0473c872079578a7364e9.png
[http_prestodb.github.io_resources.html]: http://prestodb.github.io/resources.html
[https_repo1.maven.org_maven2_com_facebook_presto_presto-server]: https://repo1.maven.org/maven2/com/facebook/presto/presto-server/
[https_www.oracle.com_technetwork_cn_java_javase_downloads_jdk8-downloads-2133151-zhs.html]: https://www.oracle.com/technetwork/cn/java/javase/downloads/jdk8-downloads-2133151-zhs.html
[https_jingyan.baidu.com_article_9f7e7ec0f8c26b6f28155433.html]: https://jingyan.baidu.com/article/9f7e7ec0f8c26b6f28155433.html
[20190421111251496.png]: /images/20220216/95c317cfb58e42e09c9840c306d26a53.png
[20190421111352138.png]: /images/20220216/f70a5c5f19fc4790a24a4427d07347c0.png
[2019042111190058.png]: /images/20220216/d69f6672881845eeaf7c6205c88cf70d.png
[20190421112305169.png]: /images/20220216/a5b992cc2ec342c9a91ac39d2776a635.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 2]: /images/20220216/67337eaf13414a8383d87f79c840de89.png
[2019042111301029.png]: /images/20220216/ed78a0a1f68e4aff976f79ea1d5f3050.png
[20190421113040704.png]: /images/20220216/cb1bca9971494a1f87a6bc59cbb3204b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 3]: /images/20220216/5231a352857745fcbfe3c27f233c6ac2.png
[20190421113505441.png]: https://img-blog.csdnimg.cn/20190421113505441.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 4]: /images/20220216/b4a23eb79aba434ca0b5f792393d08f5.png
[20190421113756599.png]: /images/20220216/c297fe17c14448ff89e81a26791573c7.png
[20190421113920270.png]: /images/20220216/0d4f6f22d42c4f91bdd48e648c14b87c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 5]: /images/20220216/dfad30fe1b6149ab9ab31aa733ece9ca.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20190421115418936.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 7]: /images/20220216/ba95656e1e164eb8b53824e412629159.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 8]: /images/20220216/9bd9cb8a0ba8448f9186f76d596cbeaa.png
[https_blog.csdn.net_Alice_qixin_article_details_80976245]: https://blog.csdn.net/Alice_qixin/article/details/80976245
[https_blog.csdn.net_weixin_43704393_article_details_87275671]: https://blog.csdn.net/weixin_43704393/article/details/87275671