【SonarQube】CentOS7安装SonarQube并集成GitLab-CI实现代码提交后自动扫描

àì夳堔傛蜴生んèń 2022-10-10 12:48 848阅读 0赞

# 1. 背景描述 #

## 1.1 需求 ##

实现功能：开发人员每提交一次代码到gitlab仓库即触发一次SonarQube代码扫描，扫描结果通过SonarQube Web UI界面可以查看。

## 1.2 实现方法 ##

gitlab-ci集成SonarQube完成代码检测。

流程：开发人员提交代码到gitlab仓库，gitlab通过gitlab-runner实现触发，通过.gitlab-ci.yml控制触发后流程，通过脚本实现sonnar-scanner对代码的检测，并将检测结果传到SonarQube网页。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70][]

*  gitlab-ci：GitLab-CI是一套配合GitLab使用的持续集成系统（GitLab-8.0+自带GitLab-CI且默认启用；另有其他解决方案：Jenkins）。
 *  gitlab-runner：GitLab-Runner配合GitLab-CI使用。GitLab里面的每一个工程都会定义一个属于这个工程的软件集成脚本，用来自动化地完成一些软件集成工作。当这个工程的仓库代码发生变动时，比如有人push了代码，GitLab就会将这个变动通知GitLab-CI。这时GitLab-CI会找出与这个工程相关联的Runner，并通知这些Runner把代码更新到本地并执行预定义好的执行脚本。
 *  gitlab-runner：代替GitLab CI来执行构建任务的工具（减少GitLab CI对GitLab的资源消耗）。
 *  SonarQube：Sonar是一款静态代码质量分析工具，支持Java、Python、PHP、JavaScript、CSS等25种以上的语言，而且能够集成在IDE、Jenkins、Git等服务中，方便随时查看代码质量分析报告。
 *  sonar-scanner：执行代码规范扫描的工具，配合SonarQube使用，扫描结果推送给SonarQube Web UI。

## 1.3 效果达成 ##

开发人员push代码到GitLab仓库，稍后（约1分钟）即可在SonarQube网页上查看代码扫描结果。

# 2. 环境资源 #

## 2.1 系统架构 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 1][]

*  已有运维GitLab服务器，自主构建Gitlab Runner + Sonar Scanner + SonarQube 服务器（这几个服务可以分开在不同的服务器部署，也可以放在一起）即可。

## 2.2 软件版本 ##

*  操作系统：CentOS Linux release 7.9.2009 (Core)
 *  Java版本：java-11-openjdk-11.0.11.0.9-1.el7\_9.x86\_64
 *  PostgreSQL版本：postgresql-13.3
 *  SonarQube版本：[SonarQube Community EditionVersion 8.9.1][]
 *  sonar-scanner版本：sonnar-scanner-4.6.2.2472
 *  sonar-gitlab-plugin插件：sonar-gitlab-plugin-3.0.1
 *  gitlab-runner版本：gitlab-runner-9.5.0

具体实现过程：

# 3. 安装SonarQube #

## 3.1 安装前准备 ##

### 3.1.1 JDK ###

最新的SonarQube-8.9版本运行需要Java11+，CentOS7自带的Java8版本不够时，需要先升级。

# 先升级yum源
    $ yum -y update
    
    # 查看是不是有可以在线安装的java11
    $ yum list | grep java-11-openjdk
    
    # 查看并卸载旧版本的java
    $ rpm -qa | grep java
    $ rpm -qa | grep jdk
    $ rpm -e --nodeps jdk1.8-1.8.0_201-fcs.x86_64
    $ rpm -e --nodeps ...
    
    # 安装Java11
    $ yum -y install java-11-openjdk
    $ java -version
    openjdk version "11.0.11" 2021-04-20 LTS
    OpenJDK Runtime Environment 18.9 (build 11.0.11+9-LTS)
    OpenJDK 64-Bit Server VM 18.9 (build 11.0.11+9-LTS, mixed mode, sharing)

### 3.1.2 PostgreSQL数据库 ###

SonarQube的安装包里自带有H2数据库，SonarQube安装完成后，网页会提示”Embedded database should be used for evaluation purposes only”，我们生产中，须使用SonarQube支持的其他数据库类型：Oracle, PostgreSQL and Microsoft SQLServer。这里我们选择使用PostgreSQL数据库。

**安装**

# 安装RPM仓库
    $ sudo yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
    
    # 安装PostgreSQL
    $ sudo yum install -y postgresql13-server
    
    # 初始化数据库
    $ sudo /usr/pgsql-13/bin/postgresql-13-setup initdb
    
    # 设置开机自启动
    $ sudo systemctl enable postgresql-13
    
    # 启动数据库
    $ sudo systemctl start postgresql-13

**修改数据库文件存放路径（可选）**

$ sudo find / -name postgresql-13.service
    /usr/lib/systemd/system/postgresql-13.service
    ...
    $ sudo vi /usr/lib/systemd/system/postgresql-13.service
    # 修改下面的路径
    [Service]
    Environment=PGDATA=/data/pgsql/13/data/
    
    # 保存退出
    # 把当前的数据库文件移到要修改的目录下
    $ mv /var/lib/pgsql/ /data/
    $ cd /data/
    $ ls -ltr
    drwx------ 3 postgres  postgres     37 Jun 21 14:49 pgsql
    
    # 重启数据库
    $ sudo systemctl stop postgresql-13.service
    $ sudo systemctl start postgresql-13.service
    
    # 检查
    # su - postgres
    -bash-4.2$ psql
    psql (13.3)
    Type "help" for help.
    postgres=# show data_directory;
       data_directory
    ---------------------
     /data/pgsql/13/data
    (1 row)
    postgres=# \q
    -bash-4.2$ exit
    logout

**修改超级管理员账号密码（可选）**

#修改登录PostgreSQL的密码
    $ sudo -u postgres psql
    postgres=# ALTER USER postgres WITH PASSWORD 'yourpassword';
    postgres=# \q
    
    #修改linux系统postgres用户的密码
    PostgreSQL数据库安装时系统会创建一个默认的linux用户postgres，修改密码步骤如下
    $ sudo passwd -d postgres
    Removing password for user postgres.
    passwd: Success
    
    $ sudo -u postgres passwd
    Changing password for user postgres.
    New password: ******
    Retype new password: ******
    passwd: all authentication tokens updated successfully.

**为SonarQube创建数据库和账号**

数据库安装完成以后会自动创建linux用户postgres，用su - postgres切换进去，再用psql进入命令行模式，为SonarQube创建数据库和用户

$ su - postgres
    Password: Pg18sql2021##
    -bash-4.2$ psql
    # 创建sonar数据库
    postgres=# create database sonar;
    CREATE DATABASE
    # 创建sonar用户
    postgres=# create user sonar;
    CREATE ROLE
    # 设置密码
    postgres=# alter user sonar with password 'postgres';
    ALTER ROLE
    # 给soner用户授权
    postgres=# alter role sonar createdb;alter role sonar superuser;alter role sonar createrole;
    ALTER ROLE
    ALTER ROLE
    ALTER ROLE
    # 更改sonar数据库属主
    postgres=# alter database sonar owner to sonar;
    ALTER DATABASE
    postgres=#

查看数据库\\l![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 2][]

查看用户\\du

![20210628155420938.png][]

至此，数据库准备完毕。

## 3.2 安装SonarQube Web服务 ##

单机版SonarQube服务安装（下载.zip压缩包解压安装）。

**下载解压SonarQube**

# 下载
    $ wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-8.9.1.44547.zip
    
    # 解压到指定路径下
    $ unzip -d /home/developer/apps/ sonarqube-8.9.1.44547.zip
    $ cd apps
    $ ls -ltr
    drwxr-xr-x 11 developer developer 4096 Jun  8 09:18 sonarqube-8.9.1.44547
    $

**配置数据库**

配置SonarQube使用PostgreSQL数据库。

$ vi /home/developer/apps/sonarqube-8.9.1.44547/conf/sonar.properties
    # 修改如下的配置
    # DATABASE
    ...
    # User credentials.
    sonar.jdbc.username=sonar
    sonar.jdbc.password=postgres
    #----- PostgreSQL 9.3 or greater
    sonar.jdbc.url=jdbc:postgresql://localhost/sonar

**配置Elasticsearch存储路径**

Elasticsearch的默认存储路径在 $SONARQUBE-HOME/data，修改的话配置在这里

$ vi sonarqube-8.9.1.44547/conf/sonar.properties
    修改下面的配置
    sonar.path.data=/data/sonarqubedata/data
    sonar.path.temp=/data/sonarqubedata/temp

**修改web服务配置**

Web服务默认的端口号是9000，我们服务器上9000端口被占用，通过配置文件修改成其他的端口。

$ vi sonarqube-8.9.1.44547/conf/sonar.properties
    # 修改下面的配置
    sonar.web.port=9003
    sonar.web.context=/data/sonarqubedata/context

**Linux服务器配置修改**

使用root用户，修改下面的配置

# vi /etc/sysctl.conf
    # 在末尾添加两行
    vm.max_map_count=262144
    fs.file-max=65536

修改/etc/profile文件

#vi /etc/profile
    # 在末尾添加一行保存
    ulimit -n 65536
    # 使配置生效
    source /etc/profile

**启动SonarQube服务**

# 进到SonarQube家目录下，启动服务
    $ pwd
    /home/developer/apps/sonarqube-8.9.1.44547
    $ bin/linux-x86-64/sonar.sh start
    Starting SonarQube...
    Started SonarQube.
    $

服务成功启动，通过浏览器可以直接访问SonarQube了，初始账号：admin/admin。首次登入后修改密码。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 3][]

# 4. 安装sonar-scanner客户端 #

sonar-scanner提供本地的代码扫描，客户端可以安装在与服务端相同的服务器，也可以装在不同的服务器（也可以安装在开发人员本地电脑）。这里我们安装在与web服务端相同的服务器。扫描时，需要将项目代码上传到服务器上完成扫描。

**下载解压**

使用浏览器打开刚刚装好的SonarQube网页，新建project时根据指引，可以找到对应OS的sonar-scanner客户端下载地址以及使用命令。![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 4][]

# 下载
    $ wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.6.2.2472-linux.zip
    # 解压
    $ unzip -d /data/ sonar-scanner-cli-4.6.2.2472-linux.zip

**修改配置文件**

# 修改配置
    $ vi /data/sonar-scanner-4.6.2.2472-linux/conf/sonar-scanner.properties
    # 打开下面两项配置
    sonar.host.url=http://localhost:9003 # 因为是在本地，不用修改了
    sonar.sourceEncoding=UTF-8
    # 并添加两行
    sonar.login=admin
    sonar.password=yourpassword

**配置环境变量**

使用root账号，修改操作系统环境变量

# 修改环境变量（下面的步骤使用root账号操作）
    # vi /etc/profile
    # 在最下面添加一行并保存
    export PATH="$PATH:/data/sonar-scanner-4.6.2.2472-linux/bin"
    
    # 使配置生效
    # source /etc/profile
    
    # 环境变量设置成功，可以在服务器的任何地方执行sonar-scanner命令了
    # sonar-scanner -v
    INFO: Scanner configuration file: /data/sonar-scanner-4.6.2.2472-linux/conf/sonar-scanner.properties
    INFO: Project root configuration file: NONE
    INFO: SonarScanner 4.6.2.2472
    INFO: Java 11.0.11 AdoptOpenJDK (64-bit)
    INFO: Linux 3.10.0-693.2.2.el7.x86_64 amd64
    #

到这一步，Sonar Scanner + SonarQube Web服务部署完成，我们可以手动从仓库拉取代码到Sonnar Scanner所在的服务器上，手动执行sonnar-scanner命令完成代码扫描，结果会自动传到SonarQube的网页上。手工的方法参照附录（\# 7.代码手动扫描）。

而我们的目的是开发者push代码到GitLab仓库时，自动触发完成代码扫描，得到结果。所以继续下一步，SonarQube与GitLab的集成。

# 5. [SonarQube与GitLab集成][SonarQube_GitLab] #

## 5.1 安装Sonar Gitlab Plugin ##

**下载解压**

下载[sonar-gitlab-plugin][] 插件，放到 $SONARQUBE-HOME/extensions/plugins/ 下

$ pwd
    /home/developer/apps/sonarqube-8.9.1.44547/extensions/plugins
    $ ls -ltr
    -rw-r--r-- 1 developer developer 7859547 Jun 23 14:23 sonar-gitlab-plugin-3.0.1.jar

**配置GitLab通用账户**

为了方便统一管理，我们在gitlab上注册一个通用账户：sonarqube，作为对接SonarQube的统一账户，同时还需要将SonarQube加到项目成员或项目所属项目组成员里面去，这样才能有权限comment和添加注释行。

获取该账户的Token：登录GitLab - Eddit Profile - Access Token - Personal Access Tokens，获取的Personal Access Tokens为下边插件配置使用。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 5][]

**配置sonar-gitlab-plugin插件**

使用admin账户登录SonarQube，点击 配置 - 通用设置 - GitLab - 新建配置，填入GitLab url和GitLab User Token

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 6][]

## 5.2 安装配置gitlab-runner ##

gitlab-runner是配合gitlab-ci进行使用的。一般地，gitlab里面的每一个工程都会定义一个属于这个工程的软件集成脚本，用来自动化地完成一些软件集成工作。 当这个工程的仓库代码发生变动时，比如有人push了代码或者分支合并，gitlab就会将这个变动通知gitlab-ci。这时gitlab-ci会找出与这个工程相关联的runner，并通知这些runner把代码更新到本地并执行预定义好的执行脚本。

**[GitLab CI Runner][] 安装**

回到gitlab - 项目主页 - settings - CI/CD - Runners - Specific runners下面，按照网页提示完成gitlab-runner的安装

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 7][]

**下载安装**

# wget -O /usr/local/bin/gitlab-runner https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-linux-amd64 
    # 如果需要代理的话在命令行后面跟上 -e use_proxy=yes -e https_proxy=http://username:password@yourhost:port/ --no-check-certificate
    
    # 授权sudo
    $ sudo chmod +x /usr/local/bin/gitlab-runner
    
    # 创建Linux用户"gitlab-runner"
    $ sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

**注册Runner**

向GitLab-CI注册一个Runner需要两样东西：GitLab-CI的url和注册token。打开GitLab中的项目页面，在项目的Settings设置页面，选择CI/CD Pipelines选项卡，找到GitLab-CI的url和注册token。

$ sudo /usr/local/bin/gitlab-runner register（或者 $ sudo /usr/local/bin/gitlab-runner register --url http://***.***.***.***/ --registration-token your_token）
    Running in system-mode.
    Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
    http://***.***.***.***/    # gitlab网页 - 项目主页 - settings - CI/CD - Runners - Specific runners
    Please enter the gitlab-ci token for this runner:
    your_token    # gitlab网页 - 项目主页 - settings - CI/CD - Runners - Specific runners
    Please enter the gitlab-ci description for this runner:
    [node03]: devops    # 写个描述
    Please enter the gitlab-ci tags for this runner (comma separated):
    dev    # 取个tag名称
    Whether to run untagged builds [true/false]:
    [false]: true
    Whether to lock Runner to current project [true/false]:
    [false]:    # 直接回车
    Registering runner... succeeded                     runner=VYGze_zT
    Please enter the executor: docker-ssh, parallels, shell, ssh, virtualbox, docker, docker-ssh+machine, kubernetes, docker+machine:
    shell    # 选择shell
    Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

注册成功后在gitlab网页 - 项目主页 - settings - CI/CD - Runners - Specific runners下面可以看到添加成功的Available specific runners

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 8][]

备注：其他命令

# 查看已注册的Runners
    $ sudo /usr/local/bin/gitlab-runner list
    
    #注销Runner
    Delete runner from /etc/gitlab-runner/config.toml file and start the runner.

**安装并启动gitlab-runner**

$ sudo /usr/local/bin/gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
    $ sudo /usr/local/bin/gitlab-runner start

备注：[https://docs.gitlab.com/runner/][https_docs.gitlab.com_runner]

*  Shared runners are for use by all projects — 需要Git管理员（不是仓库管理员）的权限
 *  Group runners are for all projects and subgroups in a group — 需要Group的master以上的权限
 *  Specific runners are for individual projects

## 5.3 配置.gitlab-ci.yml ##

GitLab Runner负责项目的构建，但构建流程是由项目根目录的[.gitlab-ci.yml][]文件控制的。因此我们要在项目的根目录下面创建.gitlab-ci.yml文件。

新建.gitlab-ci.yml文件，提交到项目的根目录下面去。[文件模板示例][Link 1]

$ more .gitlab-ci.yml
    stages:
      - test
    
    job1:
      stage: test
      only:
        - master
      script:
        - /data/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner -Dsonar.host.url="http://***.***.***.***:9003" -Dsonar.login=admin -Dsonar.projectKey=TEST_PROJECT1 -Dsonar.sources=.
      tags:
        - dev

将.gitlab-ci.yml文件提交到gitlab上项目的根目录下，这时候我们从该项目的Pipelines选项卡可以看到，有正在运行的刚新建的boss-tag这个runner的pipelines了。点进去可以看到控制台实时输出日志。Pipelines最上面一行记录Status显示绿色的Pass，则代表整个流程跑通了。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 9][]

至此，整个代码提交-检测流程集成完毕，开发者向GitLab仓库push代码，待job执行完毕，就可以通过SonnarQube的Web页面看到扫描结果了。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 10][]

#  #

# 6. GitLab新项目加入代码自动检测流程 #

GitLab新的Project -> 启用已有的Specific Runner -> 给新Project新增.gitlab-ci.yml配置文件 -> 完成。

以TEST\_PROJECT2仓库为例：

**启用已有的Specific Runner**

因为我们第一个项目TEST\_PROJECT1已经注册了Specific Runner（devops），这里同在一个GitLab Group下的TEST\_PROJECT2可以直接启用，启用后Runne与TEST\_PROJECT1项目共用一个Runner。

GitLab Project下面 - Settings - CI/CD - Runners - Specific runners - Other available runners - 找到devops runner，点击Enable for this project

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 11][]

**新增.gitlab-ci.yml**

新建一个.gitlab-ci.yml配置文件，push到EST\_PROJECT2仓库的根目录下

$ more .gitlab-ci.yml
    stages:
      - test
    job1:
      stage: test
      only:
        - master
      script:
        - /data/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner -Dsonar.host.url="http://***.***.***.***:9003" -Dsonar.login=admin -Dsonar.projectKey=TEST_PROJECT2 -Dsonar.sources=.
      tags:
        - dev

特别注意：配置文件内Script下面code里的projectKey改成对应仓库的名称，这样在SonarQube网页上更容易找到对应项目的测试结果。

查看GitLab项目下面的Pipelines，执行成功

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 12][]

再去SonarQube Web页面查看，测试结果已经成功传上去了

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 13][]

至此，新的代码仓库加入代码自动化检测流程完成。

# 7. 代码手动扫描 #

如果不想通过GitLab push -> SonarQube 的自动化代码扫描，也可以手动在命令行使用命令来完成代码扫描。

**拉取项目代码**

从gitlab仓库拉取代码到服务器（实际使用中，按测试要求拉取对应tag的代码）

# 登录到sonar-scanner服务器上使用root账号操作
    # cd /data/sonar_projects/
    # git clone http://***.***.***.***/XXX/test_project2.git

**添加配置文件sonar-project.properties**

进入项目代码的根目录，新建一个配置文件sonar-project.properties

# cd test_project2/
    # touch sonar-project.properties
    # 写入如下的内容
    # must be unique in a given SonarQube instance
    sonar.projectKey=test_project2
    # --- optional properties ---
    # defaults to project key
    sonar.projectName=test_project2
    # defaults to 'not provided'
    sonar.projectVersion=1.0    # 修改成代码实际的版本号
    # Path is relative to the sonar-project.properties file. Defaults to .
    sonar.sources=.
    # Encoding of the source code. Default is default system encoding
    sonar.sourceEncoding=UTF-8

**执行扫描**

在当前目录执行sonnar-scanner

# sonar-scanner
    INFO: Scanner configuration file: /data/sonar-scanner-4.6.2.2472-linux/conf/sonar-scanner.properties
    INFO: Project root configuration file: /data/sonar_projects/test_project2/sonar-project.properties
    ......
    ......
    http://localhost:9003/dashboard?id=test_project2
    INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
    INFO: More about the report processing at http://localhost:9003/api/ce/task?id=AXoyfOjArSHJczayS5bx
    INFO: Analysis total time: 39.385 s
    INFO: ------------------------------------------------------------------------
    INFO: EXECUTION SUCCESS
    INFO: ------------------------------------------------------------------------
    INFO: Total time: 41.029s
    INFO: Final Memory: 8M/30M
    INFO: ------------------------------------------------------------------------

扫描结束后，就可以去SonarQube网页上查看扫描结果了

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 14][]

# 8. 常见故障处理 #

## 8.1 unknown failure 处理 ##

GitLab CI/CD页面提示job执行fail，错误提示”There is an unknown failure, please try again”.

去gitlab-runner服务器检查gitlab-runner状态正常

手动执行sonnar-scanner扫描，报错：

# sonar-scanner
    INFO: Scanner configuration file: /data/sonar-scanner-4.6.2.2472-linux/conf/sonar-scanner.properties
    INFO: Project root configuration file: /data/sonar_projects/boss3/sonar-project.properties
    INFO: SonarScanner 4.6.2.2472
    INFO: Java 11.0.11 AdoptOpenJDK (64-bit)
    INFO: Linux 3.10.0-693.2.2.el7.x86_64 amd64
    INFO: User cache: /root/.sonar/cache
    INFO: Scanner configuration file: /data/sonar-scanner-4.6.2.2472-linux/conf/sonar-scanner.properties
    INFO: Project root configuration file: /data/sonar_projects/test_project2/sonar-project.properties
    INFO: Analyzing on SonarQube server 8.9.1
    INFO: Default locale: "en_US", source code encoding: "UTF-8"
    INFO: Load global settings
    INFO: Load global settings (done) | time=333ms
    INFO: Server id: A7EE8CF2-AXotwldcrSHJczayS0qf
    INFO: User cache: /root/.sonar/cache
    INFO: Load/download plugins
    INFO: Load plugins index
    INFO: Load plugins index (done) | time=281ms
    INFO: Plugin [l10nzh] defines 'l10nen' as base plugin. This metadata can be removed from manifest of l10n plugins since version 5.2.
    INFO: Load/download plugins (done) | time=349ms
    INFO: ------------------------------------------------------------------------
    INFO: EXECUTION FAILURE
    INFO: ------------------------------------------------------------------------
    INFO: Total time: 1.967s
    INFO: Final Memory: 5M/24M
    INFO: ------------------------------------------------------------------------
    ERROR: Error during SonarScanner execution
    java.lang.IllegalStateException: Unable to load component class org.sonar.scanner.scan.ProjectLock
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:678)
        at org.sonar.core.platform.ComponentContainer.getComponentByType(ComponentContainer.java:282)
        at org.sonar.scanner.scan.ProjectScanContainer.doBeforeStart(ProjectScanContainer.java:156)
        at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
        at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
        at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:150)
        at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
        at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
        at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
        at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
        at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
        at com.sun.proxy.$Proxy0.execute(Unknown Source)
        at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
        at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
        at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
        at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
        at org.sonarsource.scanner.cli.Main.main(Main.java:61)
    Caused by: java.lang.IllegalStateException: Unable to load component class org.sonar.api.batch.fs.internal.DefaultInputProject
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
        at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
        at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
        at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
        at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
        at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
        at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
        at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
        at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
        at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
        at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
        at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
        ... 22 more
    Caused by: java.lang.IllegalStateException: Unable to load component class org.sonar.scanner.scan.ProjectBuildersExecutor
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
        at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
        at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
        at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
        at org.picocontainer.injectors.SingleMemberInjector.getMemberArguments(SingleMemberInjector.java:61)
        at org.picocontainer.injectors.MethodInjector.getMemberArguments(MethodInjector.java:100)
        at org.picocontainer.injectors.MethodInjector$2.run(MethodInjector.java:112)
        at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
        at org.picocontainer.injectors.MethodInjector.decorateComponentInstance(MethodInjector.java:120)
        at org.picocontainer.injectors.CompositeInjector.decorateComponentInstance(CompositeInjector.java:58)
        at org.picocontainer.injectors.Reinjector.reinject(Reinjector.java:142)
        at org.picocontainer.injectors.ProviderAdapter.getComponentInstance(ProviderAdapter.java:96)
        at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
        ... 36 more
    Caused by: java.lang.IllegalStateException: Unable to load component class com.talanlabs.sonar.plugins.gitlab.CommitProjectBuilder
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:621)
        at org.picocontainer.parameters.CollectionComponentParameter.getArrayInstance(CollectionComponentParameter.java:334)
        at org.picocontainer.parameters.CollectionComponentParameter.access$100(CollectionComponentParameter.java:49)
        at org.picocontainer.parameters.CollectionComponentParameter$1.resolveInstance(CollectionComponentParameter.java:139)
        at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:141)
        at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
        at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
        at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
        at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
        at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
        at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
        at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
        at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
        at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
        ... 51 more
    Caused by: org.picocontainer.injectors.AbstractInjector$UnsatisfiableDependenciesException: com.talanlabs.sonar.plugins.gitlab.CommitProjectBuilder hgitlab.CommitFacade' for constructor 'public com.talanlabs.sonar.plugins.gitlab.CommitProjectBuilder(com.talanlabs.sonar.plugins.gitlab.GitLabPluginCom.talanlabs.sonar.plugins.gitlab.CommitFacade)' from org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer@4e4efc1b:399<[ImmutableicoContainer@3bb8aabc:49<|
        at org.picocontainer.injectors.ConstructorInjector.getGreediestSatisfiableConstructor(ConstructorInjector.java:191)
        at org.picocontainer.injectors.ConstructorInjector.getGreediestSatisfiableConstructor(ConstructorInjector.java:110)
        at org.picocontainer.injectors.ConstructorInjector.access$100(ConstructorInjector.java:51)
        at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:331)
        at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
        at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
        at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
        at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
        at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
        at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
        at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
        at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
        ... 67 more
    ERROR:
    ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

这是因为我们引入了第三方的插件（比如中文语言插件）冲突，把第三方插件删除，再在本地执行扫描，就不报错了。第三方插件路径：$SONARQUBE\_HOME/extensions/plugins/

遇到手动扫描成功，但是GitLab Pipelines还报错的情况。去服务器gitlab-runner用户builds目录下，把对应仓库的数据清理掉，再去GitLab网页点retry就成功了。

# 路径
    $ pwd
    /home/gitlab-runner/builds/3PrAnCmk/0/XXXGroup
    
    $ ls -ltr
    total 16
    drwxrwxr-x  3 gitlab-runner gitlab-runner 4096 Jun 25 14:04 test_project1.tmp
    drwxrwxr-x  6 gitlab-runner gitlab-runner 4096 Jun 25 16:56 test_project1
    drwxrwxr-x  3 gitlab-runner gitlab-runner 4096 Jun 25 16:58 test_project2.tmp
    drwxrwxr-x 15 gitlab-runner gitlab-runner 4096 Jun 25 16:58 test_project2

还有一种类似的报错，没有记录日志，原因是sonnar-scanner每次都会重新计算所有的ES索引。解决方法是删除SonarQube的数据库文件目录下的es文件夹，重启SonarQube服务。下次遇到了再补充报错日志。

## 8.2 SonarQube 语言插件版本Warnings ##

**故障现象**

GitLab pipelines 执行成功，但是job日志里面有Warnings告警

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 15][]

同时SonarQube页面有语言插件版本不匹配的告警：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 16][]

这是因为服务器本地环境nodejs版本过低引起的。

**解决方法**

方法1：升级服务器上的nodejs版本；  
方法2：在服务器上另外安装高版本的nodejs用于SonarQube测试。

# 我们用方法1
    # 查看服务器当前nodejs版本
    [root@node03 ~]# node -v
    
    v6.17.1
    
    # 安装nodejs管理工具n（n是nodejs管理工具，是TJ写的，Github: https://github.com/tj/n）
    [root@node03 ~]# npm config set proxy http_proxy=http://yourproxyaddress:proxyport
    [root@node03 ~]# npm config set https-proxy http_proxy=http://yourproxyaddress:proxyport
    [root@node03 ~]# npm install -g n
    
    # 安装最新版本的nodejs
    [root@node03 ~]# export http_proxy=http://yourproxyaddress:proxyport
    [root@node03 ~]# export https_proxy=http://yourproxyaddress:proxyport
    [root@node03 ~]# n stable
      installing : node-v14.17.1
           mkdir : /usr/local/n/versions/node/14.17.1
           fetch : https://nodejs.org/dist/v14.17.1/node-v14.17.1-linux-x64.tar.xz
       installed : v14.17.1 (with npm 6.14.13)
    Note: the node command changed location and the old location may be remembered in your current shell.
             old : /usr/bin/node
             new : /usr/local/bin/node
    To reset the command location hash either start a new shell, or execute PATH="$PATH"
    
    # 用命令行通过上下键选择环境使用的n版本，敲回车确定
    [root@node03 ~]# n
       installed : v14.17.1 (with npm 6.14.13)
    
    # 再次查看，还是旧版本
    [root@node03 ~]# node -v
    v6.17.1
    
    # 设置环境变量，让新版本的nodejs生效
    [root@node03 ~]# vi ~/.bash_profile
    # 加入两行
    export N_PREFIX=/usr/local
    export PATH=$N_PREFIX/bin:$PATH
    
    [root@ac3-node03 ~]# source ~/.bash_profile
    # 再次查看，新版本已经生效
    [root@node03 ~]# node -v
    v14.17.1

再次触发GitLab Pipelines的任务，没有错误出现，问题解决。

## 8.3 Marketplace 插件自动更新失败 ##

SonarQube的Marketplace会自动搜索插件更新，因为我们在内网，插件会更新失败。两个解决方案：

**方案一：关闭自动更新**

$ vi conf/sonar.properties
    sonar.updatecenter.activate=false
    
    # 重启SonarQube服务
    $ cd /home/developer/apps/sonarqube-8.9.1.44547
    $  bin/linux-x86-64/sonar.sh stop
    $ bin/linux-x86-64/sonar.sh start

方案二：给MarketPlace设置[代理][Link 2]，让其可以联网更新

$ vi conf/sonar.properties
    http.proxyHost=<your.proxy.host>
    http.proxyPort=<yout.proxy.port>
    #If proxy authentication is required
    http.proxyUser=<your.proxy.user>
    http.proxyPassword=<your.proxy.password>
    
    # 重启SonarQube服务
    $ cd /home/developer/apps/sonarqube-8.9.1.44547
    $  bin/linux-x86-64/sonar.sh stop
    $ bin/linux-x86-64/sonar.sh start

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70]: /images/20221005/ee970cad2ebd465387a34825a6599ec5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 1]: /images/20221005/4edb5238988847f79fbe12b954d0722b.png
[SonarQube Community EditionVersion 8.9.1]: https://docs.sonarqube.org/latest/setup/install-server/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 2]: /images/20221005/c167aeb6687d4516be53d76805da9de1.png
[20210628155420938.png]: /images/20221005/bd2aa483425442b4b85b94524ab50c08.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 3]: /images/20221005/e5e1c9d8ea0342a7bb3f46cee5395888.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 4]: /images/20221005/7bdbf009692941c38d453cf699062a77.png
[SonarQube_GitLab]: https://docs.sonarqube.org/latest/analysis/gitlab-integration/
[sonar-gitlab-plugin]: https://gitlab.talanlabs.com/gabriel-allaigre/sonar-gitlab-plugin
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 5]: /images/20221005/cc3a115d11564307adefb2930c44da2e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 6]: /images/20221005/afe1a2356dee413bb44e963d26e6cc4a.png
[GitLab CI Runner]: https://docs.gitlab.com/runner/install/linux-repository.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 7]: /images/20221005/190a194398ec4b37877af7d7ec7d7e1e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 8]: /images/20221005/54c1b48fa93b41369445f2c57143e0ca.png
[https_docs.gitlab.com_runner]: https://docs.gitlab.com/runner/
[.gitlab-ci.yml]: https://docs.gitlab.com/ee/ci/yaml/README.html
[Link 1]: https://docs.gitlab.com/ee/ci/examples/README.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 9]: /images/20221005/3a27dc1f4c3c436f9a69dfe3ecefd27d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 10]: /images/20221005/f7281a9273454869acd4fbffe05d216b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 11]: /images/20221005/143524d5a99945ffb125cfab32df46f1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 12]: /images/20221005/61f654ccc7c94d75beeedcfea87baf9d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 13]: /images/20221005/2e373fdf330a4349a0ba05afd22299aa.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 14]: /images/20221005/c958d3e0a2ac4ed094d83508da4234a2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 15]: /images/20221005/40a82cd4aaa5495ab16e6b5f2e1ce6ec.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhd2E4ODk5_size_16_color_FFFFFF_t_70 16]: /images/20221005/9dbaf7d495b447a281dbaf6544b84e9c.png
[Link 2]: https://docs.sonarqube.org/latest/instance-administration/marketplace/