Mongo-BI（bi-connector）配置使用筆記

一、概述

MongoDB 官方提供的 BI Connector ，可以用來通過SQL(MySQL協議)方式直接訪問MongoDB。

僅可以查詢數據(基本上聚合函數、多表關聯查詢等都可支持)，但不能通過此方式編輯或刪除mongodb數據。

bi-connector官方文檔：https://docs.mongodb.com/bi-connector/master/reference

二、下載安裝

bi-connector支持不同平台安裝部署，這里針對Linux環境安裝部署配置進行記錄。

通過官網下載：https://www.mongodb.com/try/download/bi-connector

我這里下載的文件版本為mongodb-bi-linux-x86_64-rhel70-v2.14.0.tgz

下載后解壓到/usr/local/mongodb-bi/目錄

三、創建證書

當MongoDB啟用認證時，bi-connector必須要配置使用證書，才能通過bi-connector連接mongodb

這里先創建證書

cd /usr/local/mongodb-bi/
mkdir certs && cd ./certs/
openssl req -nodes -newkey rsa:2048 -keyout myapp.key -out myapp.crt -x509 -days 3650 -subj "/C=US/ST=myapp/L=myapp/O=myapp Security/OU=IT Department/CN=myapp.com"
cat myapp.crt myapp.key > myapp.pem

四、 生成Schema

bi-connector需要schema(sql查詢時對應的字段、類型與mongodb中對應的字段和類型的映射關系)來支持查詢訪問。可以有兩種方案，一是預生成，即根據現有mongodb中集合，生產對應的靜態schema文件；二是動態更新生成，實際生產環境使用時，建議使用預生成，動態方式對服務器存儲和壓力較大(需要有單獨的mongodb庫存儲動態生成的schema，時間長會比較大)。這里使用預生成方式(缺點是如果集合有字段編號，需要手動重新生成)。

預生成schema命令如下：

cd /usr/local/mongodb-bi/
./bin/mongodrdl --host 192.168.1.100:27017 --username myapp --password mypwd --db myapp --authenticationDatabase myapp --out schemas/schemas.drdl

也可以根據條件限制僅針對指定的庫和集合(或視圖)生成schema，而不是全部庫或集合

cd /usr/local/mongodb-bi/
./bin/mongodrdl --host 127.0.0.1:27017 --username myapp --password mypwd --db myapp --authenticationDatabase myapp sample --db myapp --collection view_* --sampleSize 10000 --out ./schemas/schemas.drdl

五、修改配置 mongosqld-config.yml

解壓目錄下默認有example-mongosqld-config.yml，拷貝一份改名為mongosqld-config.yml
根據實際修改配置：

## This is a example configuration file for mongosqld.

## The full documentation is available at:
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#configuration-file

## Network options - configure how mongosqld should accept connections.
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#network-options
net:
  bindIp: "0.0.0.0" # To bind to multiple IP addresses, enter a list of comma separated values.
  port: 3307
  # unixDomainSocket:
  #   enabled: false
  #   pathPrefix: "/var"
  #   filePermissions: "0600"
  ssl:
    mode: "allowSSL"
    allowInvalidCertificates: true
    PEMKeyFile: './certs/myapp.pem'
    # PEMKeyPassword: <string>
    # CAFile: <string>
    minimumTLSVersion: TLS1_0

## MongoDB options - configure how mongosqld should connect to your MongoDB cluster.
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#mongodb-host-options
mongodb:
  # versionCompatibility: <string>
  net:
    uri: "mongodb://192.168.1.100:27017" # https://docs.mongodb.com/manual/reference/connection-string/#mongodb-uri
    ssl:
      enabled: false
    ## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#mongodb-tls-ssl-options
    #   allowInvalidCertificates: false
    #   allowInvalidHostnames: false
    #   PEMKeyFile: <string>
    #   PEMKeyPassword: <string>
    #   CAFile: <string>
    #   CRLFile: <string>
    #   FIPSMode: false
    #   minimumTLSVersion: TLSv1_1
    auth:
      username: myapp
      password: mypwd
      source: myapp # This is the name of the database to authenticate against.
      mechanism: SCRAM-SHA-1
     #gssapiServiceName: mongodb

# Security options - configure mongosqld's authentication (disabled by default).
## Enable security options if your MongoDB cluster requires authentication.
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#security-options
security:
  enabled: true
  defaultMechanism: "SCRAM-SHA-1"
  defaultSource: "myapp"
#   gssapi:
#     hostname: <string>
#     serviceName: "mongosql"

## Logging options
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#logging-options
systemLog:
  ## The path to the file where log output will be written to.
  ## Defaults to stderr.
  path: ./mongosqld.log
  quiet: true
  ## 0|1|2 - Verbosity of the log output, this is overridden if `quiet` is true.
  verbosity: 1
  logAppend: true
  #logRotate: "rename" # "rename"|"reopen"

## Schema options
## These configuration options define how the mongosqld should sample your MongoDB
## data so that it can be used by the relational application.
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#data-sampling-options
schema:
  ## If you've generated a DRDL schema file using mongodrdl, you can supply the
  ## path for mongosqld to use that schema DRDL file.
  path: ./schemas
  maxVarcharLength: 65535
  ## Use the `refreshIntervalSecs` option to specify an interval in seconds for
  ## mongosqld to update its schema, either by resampling or by re-reading from
  ## the schema source. The default value for this option is 0, which means that
  ## mongosqld does not automatically refresh the schema after it is
  ## initialized.
  refreshIntervalSecs: 0
  #stored:
    #mode: "custom" # "auto"|"custom"
    #source: "mongosqld_data" # the database where schemas are stored in stored-schema modes
    #name: "schema" # the named schema to read/write to in stored-schema modes
  sample:
    size: 1000 # The amount of random documents we sample from each collection.
    namespaces: ["*.*"]
    #namespaces: ["myapp*.view_*"]
    prejoin: false
    # uuidSubtype3Encoding: "old" # <[old|csharp|java]>

## Process management options
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#process-management-options
processManagement:
  service:
    name: "mongosql"
    displayName: "MongoSQL Service"
    description: "MongoSQL accesses MongoDB data with SQL"

## Runtime options
## https://docs.mongodb.com/bi-connector/master/reference/mongosqld/#runtime-options
# runtime:
#   memory:
#     ## A value of `0` indicates there is no enforced maximum.
#     maxPerStage: 0
#     maxPerServer: 0
#     maxPerConnection: 0

這里需要注意的是，生產環境 systemLog.quiet需要配置為true,即無日志記錄，不然會生產比較大的日志文件

六、啟動mongodb-bi

/usr/local/mongodb-bi/bin/mongosqld --config mongosqld-config.yml

七 、通過數據庫工具和JDBC方式連接使用

通過MySQL工具（如Navicat等）連接時，需要勾選“使用SSL",可以不用實際指定證書。

通過jdbc訪問也必須要啟用ssl ,加上useSSL=true&verifyServerCertificate=false

例如：

jdbc:mysql://192.168.1.100:3307/myapp?useUnicode=true&zeroDateTimeBehavior=convertToNull&characterEncoding=UTF-8&serverTimeZone=Asia/Shanghai&useSSL=true&verifyServerCertificate=false