SQL審計平台之Archery安裝

一、簡介

Archery是一個開源的SQL審核查詢平台，基於python開發。支持多數據庫的SQL上線和查詢，同時支持豐富的MySQL運維功能，底層基於goinception和inception兩個開源審計組件。Archery功能導圖和部署文檔。

二、安裝

1.安裝pyenv和virtualenv

具體步驟請參考：Centos7安裝pyenv和virtualenv

2.准備虛擬環境

創建虛擬環境並激活

pyenv virtualenv 3.6.4 venv4archery
pyenv activate venv4archery

3.下載解壓

wget https://github.com/hhyo/archery/archive/v1.8.0.tar.gz
tar xvf v1.8.0.tar.gz
cd Archery-1.8.0/

4.安裝依賴

yum依賴

yum -y install gcc gcc-c++ python-devel mysql-devel openldap-devel unixODBC-devel gettext python36 python36-pip

pip依賴

 pip install -r requirements.txt

報錯1

ModuleNotFoundError: No module named 'setuptools_rust'

解決辦法

pip install --upgrade pip

報錯2

ModuleNotFoundError: No module named 'Crypto'

解決辦法

pip install pycrypto

5.安裝redis

安裝並啟動

tar xf redis-5.0.3.tar.gz
cd redis-5.0.3
make PREFIX=/usr/local/redis install
mkdir  /usr/local/redis/etc
cp redis.conf  /usr/local/redis/etc/
/usr/local/redis/bin/redis-server  /usr/local/redis/etc/redis.conf

6.安裝Inception

下載

wget https://github.com/hhyo/inception/releases/download/v2.1.52.2/Inception_2.1.52.2
chmod +x Inception_2.1.52.2

配置

cat inc.cnf
[inception]
general_log=1
general_log_file=inception.log
port=6669
socket=/tmp/inc.socket
character-set-client-handshake=0
character-set-server=utf8
inception_language_code=zh-CN
inception_remote_system_password=root
inception_remote_system_user=root
inception_remote_backup_port=3306
inception_remote_backup_host=127.0.0.1
inception_support_charset=utf8,utf8mb4
inception_enable_nullable=0
inception_check_primary_key=1
inception_check_column_comment=1
inception_check_table_comment=1
inception_osc_on=OFF
inception_osc_bin_dir=/usr/bin
inception_osc_min_table_size=1
inception_osc_chunk_time=0.1
inception_enable_blob_type=1
inception_check_column_default_value=1

啟動

./Inception_2.1.52.2 --defaults-file=inc.cnf

7.安裝goinception

wget https://github.com/hanchuanchuan/goInception/releases/download/v1.2.4/goInception-linux-v1.2.4.tar.gz
tar xvf goInception-linux-v1.2.4.tar.gz
mv config/config.toml.default config/config.toml
./goInception -config=config/config.toml

8.修改配置

vi archery/settings.py

# 關閉debug模式
DEBUG = False
# 該項目本身的mysql數據庫地址
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'archery',
        'USER': 'root',
        'PASSWORD': '123456',
        'HOST': '127.0.0.1',
        'PORT': '3357',
        'OPTIONS': {
            'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
            'charset': 'utf8mb4'
        },
        'TEST': {
            'NAME': 'test_archery',
            'CHARSET': 'utf8mb4',
        },
    }
}
# 緩存配置
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/0",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": ""
        }
    },
    "dingding": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": ""
        }
    }
}

建議MySQL版本5.6以上否則下面步驟會報錯。

9.啟動准備

# 數據庫初始化
python3 manage.py makemigrations sql
python3 manage.py migrate 
# 數據初始化
python3 manage.py dbshell<sql/fixtures/auth_group.sql
python3 manage.py dbshell<src/init_sql/mysql_slow_query_review.sql
# 創建管理用戶
python3 manage.py createsuperuser

MySQL版本5.6會報錯，推薦使用5.7及以上版本
MySQLdb._exceptions.OperationalError: (1071, 'Specified key was too long）

10.啟動

pyenv activate venv4archery
#啟動Django-Q，需保持后台運行
python3 manage.py qcluster
#啟動服務
python3 manage.py runserver 0.0.0.0:9123  --insecure

11.訪問登陸

http://10.0.0.51:9123/

12.安裝SQLAdvisor（可選）

安裝依賴

yum install install cmake libaio-devel libffi-devel glib2 glib2-devel -y
yum install https://mirrors.cnnic.cn/percona/ps-56/yum/release/6/RPMS/x86_64/Percona-Server-shared-56-5.6.50-rel90.0.1.el6.x86_64.rpm
cd /usr/lib64/
ln -s libperconaserverclient_r.so.18 libperconaserverclient_r.so

下載源碼包編譯安裝,先編譯依賴項sqlparser，再安裝SQLAdvisor源碼

wget https://github.com/Meituan-Dianping/SQLAdvisor/archive/refs/tags/v2.0.tar.gz
tar xvf v2.0.tar.gz
cd SQLAdvisor-2.0/
cmake -DBUILD_CONFIG=mysql_release -DCMAKE_BUILD_TYPE=debug -DCMAKE_INSTALL_PREFIX=/usr/local/sqlparser ./
make && make install
cd SQLAdvisor/sqladvisor/
cmake -DCMAKE_BUILD_TYPE=debug ./
make

查看幫助

./sqladvisor --help
Usage:
  sqladvisor [OPTION?] sqladvisor

SQL Advisor Summary

Help Options:
  -?, --help              Show help options

Application Options:
  -f, --defaults-file     sqls file
  -u, --username          username
  -p, --password          password
  -P, --port              port
  -h, --host              host
  -d, --dbname            database name
  -q, --sqls              sqls
  -v, --verbose           1:output logs 0:output nothing

安裝文檔

13.soar安裝（可選）

GitHub地址

14.gh-ost安裝

gh-ost
其它配置操作請參照官方文檔。