文章記錄在centos 7上安裝golang環境及go版TensorFlow的過程及中間遇到的問題及解決方法。
1 安裝golang環境
環境是宿主機安裝virtualbox加載的centos 7環境,采用的yum源安裝。此部分參考此文 CentOS7安裝golang
yum -y install golang
安裝完成后,查看
[root@localhost ~]# go version
go version go1.13.14 linux/amd64
[root@localhost ~]#
設置環境變量,在home目錄創建GOPATH目錄
[root@localhost home]# mkdir gopath^C
[root@localhost home]#
[root@localhost home]#
[root@localhost home]# ll
總用量 16548
drwxr-xr-x. 4 root root 99 8月 27 15:45 gopath
-rwxr-xr-x. 1 root root 8496 8月 26 18:00 hello_tf
-rw-r--r--. 1 root root 151 8月 26 18:00 hello_tf.c
drwxr-xr-x. 3 root root 22 8月 27 07:52 openblas
drwxr-xr-x. 19 501 501 4096 8月 27 11:31 Python-3.7.0
-rw-r--r--. 1 root root 16922100 8月 27 10:29 Python-3.7.0.tar.xz
[root@localhost home]#
編輯/root/.bashrc文件,添加如下信息
export GOPATH=/home/gopath
export GOROOT=/usr/lib/golang
export GOBIN=$GOROOT/bin
export PATH=$PATH:$GOBIN
重新加載/root/.bashrc文件
source /root/.bashrc
查看go的相關設置是否生效
[root@localhost home]# go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/usr/lib/golang/bin"
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gopath"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/lib/golang"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build389723181=/tmp/go-build -gno-record-gcc-switches"
[root@localhost home]#
環境設置已完成,進行測試,在/home/gopath下編寫測試程序
[root@localhost gopath]# vim test.go
package main
import "fmt"
func main() {
fmt.Println("Hello world!")
}
保存后,輸入命令
go build test.go
在輸入
[root@localhost gopath]# ./test
Hello world!
[root@localhost gopath]#
測試go已經配置完成。過程很順利的完成。
2 安裝C版TensorFlow
此部分是按照 安裝 Go 版 TensorFlow 中的說明,先設置TensorFlow C 庫,安裝過程參考 安裝 C 版 TensorFlow
2.1安裝壓縮包
下載:https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz
webget https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz
解壓縮下載的歸檔文件,其中包含要添加到 C 程序中的頭文件以及要與之關聯的共享庫。在 Linux 和 macOS 上,您可能需要解壓縮到 /usr/local/lib:
sudo tar -C /usr/local -xzf (downloaded file)
在 Linux/macOS 上,如果將 TensorFlow C 庫解壓縮到系統目錄(例如 /usr/local),請使用 ldconfig 配置鏈接器:
sudo ldconfig
2.2編譯
安裝 TensorFlow C 庫后,使用以下源代碼創建一個示例程序 (hello_tf.c):
#include <stdio.h>
#include <tensorflow/c/c_api.h>
int main() {
printf("Hello from TensorFlow C library version %s\n", TF_Version());
return 0;
}
編譯示例程序以創建可執行文件,然后運行以下命令:
gcc hello_tf.c -ltensorflow -o hello_tf
./hello_tf
執行上述命令后報錯如下:
[root@localhost local]# ./hello_tf
./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
百度相關后,找到如下解決方案:
$ gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow -o hello_tf
$ ./hello_tf
./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
$ ./hello_tf
Hello from TensorFlow C library version 1.14.0
$ unset LD_LIBRARY_PATH
$ ./hello_tf
./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
基於該方案,設置LD_LIBRARY_PATH,如下所示:
[root@localhost local]# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
[root@localhost local]# ./hello_tf
Hello from TensorFlow C library version 1.15.0
[root@localhost local]#
另一種解決方案(備注該方案為佳),打開/etc/ld.so.conf文件,增加/usr/local/lib,主要操作如下:
[root@localhost lib]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib
而后執行ldconfig
[root@localhost lib]# ldconfig
[root@localhost lib]#
此方法參考 Suppressed: java.lang.UnsatisfiedLinkError: libxxx.so: 無法打開共享對象文件: 沒有那個文件或目錄
3 安裝 Go 版 TensorFlow
此部分是按照 安裝 Go 版 TensorFlow 的說明逐步進行的,中間有很多問題,后基於網上一些建議調整安裝成功。
3.1安裝過程
在設置完C 版 TensorFlow相關配置后,進行go版TensorFlow的安裝,使用命令
go get github.com/tensorflow/tensorflow/tensorflow/go
我在安裝的過程中,經常出現如下的問題
[root@localhost gopath]# go get -d -v github.com/tensorflow/tensorflow/tensorflow/go
github.com/tensorflow/tensorflow (download)
# cd .; git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: 過早的文件結束符(EOF)
fatal: index-pack failed
package github.com/tensorflow/tensorflow/tensorflow/go: exit status 128
[root@localhost gopath]#
分析發現是先從GitHub上clone相關的文件到本機,所以在/home/gopath目錄嘗試了如下命令
git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow
就是使用git clone到本機,但該命令經常在達到10%左右就卡住,不在下載了。故采取從瀏覽器下載到宿主機在上傳到虛擬機的方式進行,打開GitHub的TensorFlow地址,前期在是喲用rasa時用的是TensorFlow的1.13.1版本,故這里也下載1.13.1的版本,下載得到tensorflow-1.13.1.zip壓縮包,修改名稱為tensorflow.zip,而后通過rz命令上傳到宿主機
[root@localhost gopath]# cd src/github.com/
[root@localhost github.com]# ll
總用量 40544
drwxr-xr-x. 3 root root 22 8月 27 15:30 golang
drwxr-xr-x. 3 root root 18 8月 26 18:22 gonum
drwxr-xr-x. 3 root root 26 8月 27 07:56 mattn
drwxr-xr-x. 3 root root 25 8月 27 07:56 olekukonko
drwxr-xr-x. 3 root root 19 8月 27 07:56 peterh
drwxr-xr-x. 3 root root 20 8月 27 07:55 pkg
drwxr-xr-x. 4 root root 32 8月 27 07:55 spf13
drwxr-xr-x. 3 root root 24 8月 27 15:31 tensorflow
-rw-r--r--. 1 root root 41515134 8月 27 15:22 tensorflow.zip
drwxr-xr-x. 3 root root 18 8月 27 07:53 ynqa
[root@localhost github.com]#
解壓tensorflow.zip,得到tensorflow目錄,回到/home/gopath目錄進行構建,命令如下
[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
can't load package: package github.com/tensorflow/tensorflow/tensorflow/go/op: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/op" in any of:
/usr/lib/golang/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOROOT)
/home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOPATH)
[root@localhost gopath]#
報錯,說找不到該目錄,下載的源碼怎么沒有該目錄內,直接cd查看
[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op: 沒有那個文件或目錄
[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/: 沒有那個文件或目錄
[root@localhost gopath]#
確實沒有該目錄,回到解壓處查看,發現源碼解壓后只有兩層tensorflow,而go get的有三層,則我們直接增加一層tensorflow,在進行構建
[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
package google.golang.org/protobuf/encoding/prototext: unrecognized import path "google.golang.org/protobuf/encoding/prototext" (https fetch: Get https://google.golang.org/protobuf/encoding/prototext?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/encoding/protowire: unrecognized import path "google.golang.org/protobuf/encoding/protowire" (https fetch: Get https://google.golang.org/protobuf/encoding/protowire?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/proto: unrecognized import path "google.golang.org/protobuf/proto" (https fetch: Get https://google.golang.org/protobuf/proto?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoreflect: unrecognized import path "google.golang.org/protobuf/reflect/protoreflect" (https fetch: Get https://google.golang.org/protobuf/reflect/protoreflect?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoregistry: unrecognized import path "google.golang.org/protobuf/reflect/protoregistry" (https fetch: Get https://google.golang.org/protobuf/reflect/protoregistry?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoiface: unrecognized import path "google.golang.org/protobuf/runtime/protoiface" (https fetch: Get https://google.golang.org/protobuf/runtime/protoiface?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoimpl: unrecognized import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
../genop/main.go:17: running "bash": exit status 1
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]#
報錯protobuf導入超時
import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout
解決方法參考 go get 找不到 google.golang.org/protobuf github.com/gin-gonic/gin 解決辦法 也就是從GitHub上clone代碼下來,如下操作
[root@localhost gopath]# mkdir -p $GOPATH/src/google.golang.org
[root@localhost gopath]# git clone https://e.coding.net/robinqiwei/googleprotobuf.git $GOPATH/src/google.golang.org/protobuf
正克隆到 '/home/gopath/src/google.golang.org/protobuf'...
remote: Enumerating objects: 13821, done.
remote: Counting objects: 100% (13821/13821), done.
remote: Compressing objects: 100% (5172/5172), done.
remote: Total 13821 (delta 8112), reused 13821 (delta 8112), pack-reused 0
接收對象中: 100% (13821/13821), 7.44 MiB | 932.00 KiB/s, done.
處理 delta 中: 100% (8112/8112), done.
[root@localhost gopath]#
完成后,在重新構建成功,如下
[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
# github.com/golang/protobuf/protoc-gen-go
../../../../../golang/protobuf/protoc-gen-go/main.go:71:6: gen.SupportedFeatures undefined (type *protogen.Plugin has no field or method SupportedFeatures)
../../../../../golang/protobuf/protoc-gen-go/main.go:71:27: undefined: internal_gengo.SupportedFeatures
../genop/main.go:17: running "bash": exit status 2
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]#
這里發現exit status 1,也不知是啥問題,沒有去研究了。然后驗證安裝結果
[root@localhost gopath]# go test github.com/tensorflow/tensorflow/tensorflow/go
ok github.com/tensorflow/tensorflow/tensorflow/go 1.848s
[root@localhost gopath]#
程序驗證,GO示例,創建tf-hello.go
[root@localhost gopath]# vim tf-hello.go
package main
import (
tf "github.com/tensorflow/tensorflow/tensorflow/go"
"github.com/tensorflow/tensorflow/tensorflow/go/op"
"fmt"
)
func main() {
// Construct a graph with an operation that produces a string constant.
s := op.NewScope()
c := op.Const(s, "Hello from TensorFlow version " + tf.Version())
graph, err := s.Finalize()
if err != nil {
panic(err)
}
// Execute the graph in a session.
sess, err := tf.NewSession(graph, nil)
if err != nil {
panic(err)
}
output, err := sess.Run(nil, []tf.Output{c}, nil)
if err != nil {
panic(err)
}
fmt.Println(output[0].Value())
}
執行
[root@localhost gopath]# go run tf-hello.go
2020-08-27 15:44:47.868756: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-08-27 15:44:47.882914: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1799995000 Hz
2020-08-27 15:44:47.884161: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x22d3190 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-27 15:44:47.884211: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
Hello from TensorFlow version 1.15.0
[root@localhost gopath]#
C示例,C和Go實際上共用了相同的動態鏈接庫和頭文件,安裝好Go版本后,C的API可以直接使用。創建tf.c
[root@localhost gopath]# vim tf.c
#include <stdio.h>
#include <tensorflow/c/c_api.h>
int main() {
printf("Hello from TensorFlow C library version %s\n", TF_Version());
return 0;
}
編譯,執行
[root@localhost gopath]# gcc -I/usr/local/include -L/usr/local/lib tf.c -ltensorflow
[root@localhost gopath]# ll
總用量 1988
-rwxr-xr-x. 1 root root 8496 8月 27 15:45 a.out
drwxr-xr-x. 3 root root 25 8月 27 15:40 pkg
drwxr-xr-x. 5 root root 65 8月 27 15:40 src
-rwxr-xr-x. 1 root root 2008751 8月 26 17:56 test
-rw-r--r--. 1 root root 74 8月 26 17:56 test.go
-rw-r--r--. 1 root root 157 8月 27 15:45 tf.c
-rw-r--r--. 1 root root 671 8月 27 15:44 tf-hello.go
[root@localhost gopath]# ./a.out
Hello from TensorFlow C library version 1.15.0
[root@localhost gopath]#
至此,整個環境已經整合完成,后續准備使用golang調用TensorFlow模型。