DEVOPS技術實踐_05：sonar靜態代碼掃描

本文轉載自查看原文 2019-04-22 08:06 980 linux/ jenkins/ Devops

一、SonarQube靜態代碼掃描平台

1.1 安裝

https://www.sonarqube.org/官網

1.2 下載軟件包

https://www.sonarqube.org/downloads/

[root@sonar-server ~]# mkdir /usr/local/sonarc

[root@sonar-server ~]# cd /usr/local/sonarc

[root@sonar-server sonarc]# wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-6.7.7.zip

[root@sonar-server sonarc]# wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip

[root@sonar-server sonarc]# ll

-rw-r--r--. 1 root root 159921852 Apr 17  2019 sonarqube-6.7.7.zip
-rw-r--r--. 1 root root  73866903 Jan  8 22:27 sonar-scanner-cli-3.3.0.1492-linux.zip

sonar是一個用於代碼質量管理的開放平台。通過插件機制，sonar可以集成不同的測試工具，代碼分析工具，以及持續集成工具。比如pmd-cpd，checkstyle，findbugs。Jkens。通過不同的插件對這些結果進行再加工處理。通過量化的方式度量代碼質量的變化，從而可以方便的對不同規模和種類的工程進行代碼質量管理。同時sonar還對大量的集成工具提供了接口支持，可以很方便的在持續集成中使用Sonar。

[root@sonar-server sonarc]# yum -y install java-1.8.0-openjdk-devel.x86_64

官方文檔

環境要求

https://docs.sonarqube.org/latest/requirements/requirements/

JDK JRE：1.8

[root@sonar-server sonarc]# java -version

openjdk version "1.8.0_201"
OpenJDK Runtime Environment (build 1.8.0_201-b09)
OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)

1.3 安裝mysql 5.6 +

[root@sonar-server sonarc]# mkdir /usr/local/mysql

[root@sonar-server sonarc]# cd /usr/local/mysql

[root@sonar-server mysql]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm

[root@sonar-server mysql]# rpm -ivh mysql-community-release-el7-5.noarch.rpm

Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-release-el7-5    ################################# [100%]

[root@sonar-server mysql]# yum -y install mysql mysql-devel mysql-server mysql-utilities

Installed:
  mysql-community-client.x86_64 0:5.6.43-2.el7                                        
  mysql-community-devel.x86_64 0:5.6.43-2.el7                                         
  mysql-community-libs.x86_64 0:5.6.43-2.el7                                          
  mysql-community-server.x86_64 0:5.6.43-2.el7                                        
  mysql-utilities.noarch 0:1.6.5-1.el7

啟動mysql

[root@sonar-server mysql]# systemctl start mysqld

[root@sonar-server mysql]# netstat -ntlp

tcp6       0      0 :::3306                 :::*                    LISTEN      53978/mysqld

1.4 創建數據庫

[root@sonar-server mysql]# mysql -u root

mysql> show databases;

+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+

mysql> create database sonar default CHARSET utf8;

mysql> show databases;

+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sonar              |
+--------------------+

mysql> grant all on sonar.* to sonar@'%' IDENTIFIED by 'meiyoumima';

mysql> flush PRIVILEGES;

1.5 安裝sonar

If you're running on Linux, you must ensure that:
vm.max_map_count is greater or equals to 262144
fs.file-max is greater or equals to 65536
the user running SonarQube can open at least 65536 file descriptors
the user running SonarQube can open at least 2048 threads
You can see the values with the following commands:
sysctl vm.max_map_count
sysctl fs.file-max
ulimit -n
ulimit -u

根據文檔執行命令

[root@sonar-server ~]# sysctl vm.max_map_count
vm.max_map_count = 65530
[root@sonar-server ~]# sysctl fs.file-max
fs.file-max = 379752
[root@sonar-server ~]# ulimit -n
1024
[root@sonar-server ~]# ulimit -u
15012
修改這些參數
[root@sonar-server ~]# sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
[root@sonar-server ~]# sysctl -w fs.file-max=65536
fs.file-max = 65536
[root@sonar-server ~]# ulimit -n 65536
[root@sonar-server ~]# ulimit -u 2048

添加sonar用戶（sonar明確規定不允許使用root運行）

[root@sonar-server ~]# useradd sonar

[root@sonar-server ~]# cd /usr/local/sonarc/

[root@sonar-server sonarc]# unzip sonarqube-6.7.7.zip -d /home/sonar/

[root@sonar-server sonarc]# cd /home/sonar/

[root@sonar-server sonar]# mv sonarqube-6.7.7 sonarqube

修改sonar的配置文件

[root@sonar-server sonar]# cd sonarqube

[root@sonar-server sonarqube]# ll

drwxr-xr-x. 8 root root  136 Apr 16  2019 bin
drwxr-xr-x. 2 root root   50 Apr 16  2019 conf
-rw-r--r--. 1 root root 7651 Apr 16  2019 COPYING
drwxr-xr-x. 2 root root   24 Apr 16  2019 data
drwxr-xr-x. 7 root root  150 Apr 16  2019 elasticsearch
drwxr-xr-x. 4 root root   40 Apr 16  2019 extensions
drwxr-xr-x. 9 root root  140 Apr 16  2019 lib
drwxr-xr-x. 2 root root    6 Apr 16  2019 logs
drwxr-xr-x. 2 root root   24 Apr 16  2019 temp
drwxr-xr-x. 9 root root 4096 Apr 16  2019 web

[root@sonar-server sonarqube]# ll ./bin/

drwxr-xr-x. 2 root root  25 Apr 16  2019 jsw-license
drwxr-xr-x. 3 root root  48 Apr 16  2019 linux-x86-32
drwxr-xr-x. 3 root root  48 Apr 16  2019 linux-x86-64
drwxr-xr-x. 3 root root  48 Apr 16  2019 macosx-universal-64
drwxr-xr-x. 3 root root 167 Apr 16  2019 windows-x86-32
drwxr-xr-x. 3 root root 167 Apr 16  2019 windows-x86-64

[root@sonar-server sonarqube]# ll ./bin/linux-x86-64/

drwxr-xr-x. 2 root root     27 Apr 16  2019 lib
-rwxr-xr-x. 1 root root  15522 Apr 16  2019 sonar.sh     #sonar的啟動腳本
-rwxr-xr-x. 1 root root 111027 Apr 16  2019 wrapper

[root@sonar-server sonarqube]# vim conf/sonar.properties

sonar.web.host=172.25.254.133
sonar.web.port=9000
sonar.jdbc.username=sonar
sonar.jdbc.password=meiyoumima
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false

修改文件的屬主屬組

[root@sonar-server sonar]# chown -R sonar:sonar /home/sonar/*

切換用戶設置環境變量

[root@sonar-server sonar]# su - sonar

[sonar@sonar-server ~]$ vim .bash_profile

export SONAR_HOME=/home/sonar/sonarqube
export PATH=$PATH:$SONAR_HOME/bin

[sonar@sonar-server ~]$ source .bash_profile

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh start

Starting SonarQube...
Started SonarQube.

1.7 啟動sonar

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh

Usage: ./sonarqube/bin/linux-x86-64/sonar.sh { console | start | stop | restart | status | dump }

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh console

Running SonarQube...     #輸出信息
wrapper  | --> Wrapper Started as Console
wrapper  | Launching a JVM...
jvm 1    | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1    |   Copyright 1999-2006 Tanuki Software, Inc.  All Rights Reserved.
jvm 1    | 
jvm 1    | 2019.04.13 15:17:58 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /home/sonar/sonarqube/temp
jvm 1    | 2019.04.13 15:17:58 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1    | 2019.04.13 15:17:58 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/home/sonar/sonarqube/elasticsearch]: /home/sonar/sonarqube/elasticsearch/bin/elasticsearch -Epath.conf=/home/sonar/sonarqube/temp/conf/es
jvm 1    | 2019.04.13 15:17:58 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1    | 2019.04.13 15:18:14 INFO  app[][o.e.p.PluginsService] no modules loaded
jvm 1    | 2019.04.13 15:18:14 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1    | 2019.04.13 15:18:52 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
jvm 1    | 2019.04.13 15:18:52 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/home/sonar/sonarqube]: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/sonar/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -cp ./lib/common/*:./lib/server/*:/home/sonar/sonarqube/lib/jdbc/mysql/mysql-connector-java-5.1.42.jar org.sonar.server.app.WebServer /home/sonar/sonarqube/temp/sq-process7782197132334949329properties

看日志

[sonar@sonar-server ~]$ cd sonarqube/logs/
[sonar@sonar-server logs]$ tail -f es.log
2019.04.13 15:19:22 INFO  es[][o.e.n.Node] closing ...
2019.04.13 15:19:22 INFO  es[][o.e.n.Node] closed
[sonar@sonar-server logs]$ tail -f sonar.log 
2019.04.13 15:19:22 INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped
2019.04.13 15:19:22 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
<-- Wrapper Stopped
[sonar@sonar-server logs]$ tail -n 100 web.log 
    at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:106)    #數據庫問題，不能使用sonar@localhost
    ... 29 common frames omitted
Caused by: java.sql.SQLException: Access denied for user 'sonar'@'localhost' (using password: YES)

嘗試使用sonar連接數據庫

[sonar@sonar-server logs]$ mysql -hlocalhost -usonar -pmeiyoumima
Warning: Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'sonar'@'localhost' (using password: YES)
[sonar@sonar-server logs]$ mysql -h127.0.0.1 -usonar -pmeiyoumima
Warning: Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'sonar'@'localhost' (using password: YES)
[sonar@sonar-server logs]$ mysql -h172.25.254.133 -usonar -pmeiyoumima    #成功
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| sonar |
+--------------------+

修改配置文件連接數據庫

[sonar@sonar-server logs]$ vim /home/sonar/sonarqube/conf/sonar.properties

sonar.web.host=172.25.254.133
sonar.web.port=9000
sonar.jdbc.username=sonar
sonar.jdbc.password=meiyoumima
sonar.jdbc.url=jdbc:mysql://172.25.254.133:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false

[sonar@sonar-server logs]$ /home/sonar/sonarqube/bin/linux-x86-64/sonar.sh start

[sonar@sonar-server logs]$ /home/sonar/sonarqube/bin/linux-x86-64/sonar.sh console

Running SonarQube...
SonarQube is already running.

成功

1.8 瀏覽器訪問

初始密碼：admin admin登錄

1.9 安裝一個maketpalce的漢化插件

重啟

二、配置snoar-scanner

2.1 安裝

[sonar@sonar-server logs]$ unzip /usr/local/sonarc/sonar-scanner-cli-3.3.0.1492-linux.zip -d /home/sonar/

[sonar@sonar-server logs]$ cd

[sonar@sonar-server ~]$ ll

drwxr-xr-x. 11 sonar sonar 141 Apr 16  2019 sonarqube
drwxr-xr-x.  6 sonar sonar  51 Jan  8 12:19 sonar-scanner-3.3.0.1492-linux

[sonar@sonar-server ~]$ mv sonar-scanner-3.3.0.1492-linux sonar-scanner

[sonar@sonar-server ~]$ cd sonar-scanner/

[sonar@sonar-server sonar-scanner]$ ll

drwxr-xr-x. 2 sonar sonar  54 Jan  8 12:19 bin
drwxr-xr-x. 2 sonar sonar  38 Jan  8 12:17 conf
drwxr-xr-x. 4 sonar sonar 186 Jan  8 12:19 jre
drwxr-xr-x. 2 sonar sonar  46 Jan  8 12:19 lib

[sonar@sonar-server sonar-scanner]$ vim conf/sonar-scanner.properties

sonar.host.url=http://172.25.254.133:9000
sonar.sourceEncoding=UTF-8

[root@sonar-server ~]# vim .bash_profile

export SCAN_HOME=/home/sonar/sonar-scanner/
export PATH=$PATH:$SCAN_HOME/bin

[root@sonar-server ~]# source .bash_profile

2.2 測試

[root@sonar-server ~]# sonar-scanner -X

[root@sonar-server ~]# mkdir code

[root@sonar-server ~]# cd code/

[root@sonar-server code]# vim sonar-project.properties

sonar.projectKey=test-project1
sonar.projectName=cloud
sonar.projectVersion=1.0
sonar.source=src
sonar.language=python
sonar.sourceEncoding=UTF-8

[root@sonar-server code]mkdir src

[root@sonar-server code]# vim ./src/test.py

print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")

執行

[root@sonar-server code]# python ./src/test.py
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld

[root@sonar-server ~]# sonar-scanner

ERROR: You must define the following mandatory properties for 'Unknown': sonar.projectKey, sonar.sources
ERROR: 
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

2.3 修改權限

[root@sonar-server code]# chown sonar:sonar /root/code/* -R

[root@sonar-server code]# ll

-rw-r--r--. 1 sonar sonar 485 Apr 21 18:23 sonar-project.properties

[root@sonar-server ~]# sonar-scanner

ERROR: Error during SonarQube Scanner execution
ERROR: No quality profiles have been found, you probably don't have any language plugin installed.
ERROR: 
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

2.4 安裝plugin

[root@sonar-server code]# sonar-scanner

ERROR: Error during SonarQube Scanner execution
ERROR: You must install a plugin that supports the language 'python'
ERROR: #依然報錯
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

把python修改為py

[root@sonar-server code]# vim sonar-project.properties

sonar.projectKey=test-project1
sonar.projectName=cloud
sonar.projectVersion=1.0
sonar.language=py     #查閱修改此參數   https://www.cnblogs.com/ckat/p/3638887.html
sonar.sources=src
sonar.sourceEncoding=UTF-8

[root@sonar-server code]# sonar-scanner

INFO: Scanner configuration file: /home/sonar/sonar-scanner-3.3.0.1492-linux/conf/sonar-scanner.properties
INFO: Project root configuration file: /root/code/sonar-project.properties
INFO: SonarQube Scanner 3.3.0.1492
INFO: Java 1.8.0_121 Oracle Corporation (64-bit)
INFO: Linux 3.10.0-693.el7.x86_64 amd64
INFO: User cache: /root/.sonar/cache
INFO: SonarQube server 6.7.7
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Publish mode
INFO: Load global settings
INFO: Load global settings (done) | time=62ms
INFO: Server id: A623D34D-AWoVn6_8P1KovjAYWYot
INFO: User cache: /root/.sonar/cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=112ms
INFO: Plugin [l10nzh] defines 'l10nen' as base plugin. This metadata can be removed from manifest of l10n plugins since version 5.2.
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=35ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=29ms
INFO: Load active rules
INFO: Load active rules (done) | time=344ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=26ms
INFO: Project key: test-project1
INFO: -------------  Scan cloud
INFO: Load server rules
INFO: Load server rules (done) | time=34ms
INFO: Base dir: /root/code
INFO: Working dir: /root/code/.scannerwork
INFO: Source paths: src
INFO: Source encoding: UTF-8, default locale: en_US
INFO: Language is forced to py
INFO: Index files
INFO: 1 file indexed
INFO: Quality profile for py: Sonar way
INFO: Sensor Python Squid Sensor [python]
INFO: Sensor Python Squid Sensor [python] (done) | time=101ms
INFO: Sensor Cobertura Sensor for Python coverage [python]
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=11ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=0ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=28ms
INFO: Sensor CPD Block Indexer
INFO: Sensor CPD Block Indexer (done) | time=0ms
INFO: SCM Publisher is disabled
INFO: 1 file had no CPD blocks
INFO: Calculating CPD for 0 files
INFO: CPD calculation finished
INFO: Analysis report generated in 274ms, dir size=6 KB
INFO: Analysis reports compressed in 15ms, zip size=3 KB
INFO: Analysis report uploaded in 410ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://172.25.254.133:9000/dashboard/index/test-project1
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://172.25.254.133:9000/api/ce/task?id=AWo_l71TD8zrSa_Nq7yJ
INFO: Task total time: 2.315 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 3.566s
INFO: Final Memory: 9M/183M
INFO: ------------------------------------------------------------------------