安裝 go 版 TensorFlow的問題記錄


文章記錄在centos 7上安裝golang環境及go版TensorFlow的過程及中間遇到的問題及解決方法。

1 安裝golang環境

環境是宿主機安裝virtualbox加載的centos 7環境,采用的yum源安裝。此部分參考此文 CentOS7安裝golang

yum -y install golang

安裝完成后,查看

[root@localhost ~]# go version
go version go1.13.14 linux/amd64
[root@localhost ~]# 

設置環境變量,在home目錄創建GOPATH目錄

[root@localhost home]# mkdir gopath^C
[root@localhost home]# 
[root@localhost home]# 
[root@localhost home]# ll
總用量 16548
drwxr-xr-x.  4 root root       99 8月  27 15:45 gopath
-rwxr-xr-x.  1 root root     8496 8月  26 18:00 hello_tf
-rw-r--r--.  1 root root      151 8月  26 18:00 hello_tf.c
drwxr-xr-x.  3 root root       22 8月  27 07:52 openblas
drwxr-xr-x. 19  501  501     4096 8月  27 11:31 Python-3.7.0
-rw-r--r--.  1 root root 16922100 8月  27 10:29 Python-3.7.0.tar.xz
[root@localhost home]# 

編輯/root/.bashrc文件,添加如下信息

export GOPATH=/home/gopath
export GOROOT=/usr/lib/golang
export GOBIN=$GOROOT/bin
export PATH=$PATH:$GOBIN

重新加載/root/.bashrc文件

source /root/.bashrc

查看go的相關設置是否生效

[root@localhost home]# go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/usr/lib/golang/bin"
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gopath"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/lib/golang"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build389723181=/tmp/go-build -gno-record-gcc-switches"
[root@localhost home]# 

環境設置已完成,進行測試,在/home/gopath下編寫測試程序

[root@localhost gopath]# vim test.go 
package main
import "fmt"
func main() {
fmt.Println("Hello world!")
}

保存后,輸入命令

go build test.go

在輸入

[root@localhost gopath]# ./test 
Hello world!
[root@localhost gopath]#

測試go已經配置完成。過程很順利的完成。

2 安裝C版TensorFlow

此部分是按照 安裝 Go 版 TensorFlow 中的說明,先設置TensorFlow C 庫,安裝過程參考 安裝 C 版 TensorFlow

2.1安裝壓縮包

下載:https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz

webget https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz

解壓縮下載的歸檔文件,其中包含要添加到 C 程序中的頭文件以及要與之關聯的共享庫。在 Linux 和 macOS 上,您可能需要解壓縮到 /usr/local/lib:

sudo tar -C /usr/local -xzf (downloaded file)

在 Linux/macOS 上,如果將 TensorFlow C 庫解壓縮到系統目錄(例如 /usr/local),請使用 ldconfig 配置鏈接器:

sudo ldconfig

2.2編譯

安裝 TensorFlow C 庫后,使用以下源代碼創建一個示例程序 (hello_tf.c):

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
  printf("Hello from TensorFlow C library version %s\n", TF_Version());
  return 0;
}

編譯示例程序以創建可執行文件,然后運行以下命令:

gcc hello_tf.c -ltensorflow -o hello_tf

./hello_tf

執行上述命令后報錯如下:

[root@localhost local]# ./hello_tf
./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

百度相關后,找到如下解決方案:

$ gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow -o hello_tf
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
 $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
 $ ./hello_tf 
 Hello from TensorFlow C library version 1.14.0
 $ unset LD_LIBRARY_PATH 
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

基於該方案,設置LD_LIBRARY_PATH,如下所示:

[root@localhost local]# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
[root@localhost local]# ./hello_tf
Hello from TensorFlow C library version 1.15.0
[root@localhost local]# 

另一種解決方案(備注該方案為佳),打開/etc/ld.so.conf文件,增加/usr/local/lib,主要操作如下:

[root@localhost lib]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib

而后執行ldconfig

[root@localhost lib]# ldconfig
[root@localhost lib]# 

此方法參考 Suppressed: java.lang.UnsatisfiedLinkError: libxxx.so: 無法打開共享對象文件: 沒有那個文件或目錄

3 安裝 Go 版 TensorFlow

此部分是按照 安裝 Go 版 TensorFlow 的說明逐步進行的,中間有很多問題,后基於網上一些建議調整安裝成功。

3.1安裝過程

在設置完C 版 TensorFlow相關配置后,進行go版TensorFlow的安裝,使用命令

go get github.com/tensorflow/tensorflow/tensorflow/go

我在安裝的過程中,經常出現如下的問題

[root@localhost gopath]# go get -d -v github.com/tensorflow/tensorflow/tensorflow/go
github.com/tensorflow/tensorflow (download)
# cd .; git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: 過早的文件結束符(EOF)
fatal: index-pack failed
package github.com/tensorflow/tensorflow/tensorflow/go: exit status 128
[root@localhost gopath]#

分析發現是先從GitHub上clone相關的文件到本機,所以在/home/gopath目錄嘗試了如下命令

git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow

就是使用git clone到本機,但該命令經常在達到10%左右就卡住,不在下載了。故采取從瀏覽器下載到宿主機在上傳到虛擬機的方式進行,打開GitHub的TensorFlow地址,前期在是喲用rasa時用的是TensorFlow的1.13.1版本,故這里也下載1.13.1的版本,下載得到tensorflow-1.13.1.zip壓縮包,修改名稱為tensorflow.zip,而后通過rz命令上傳到宿主機

[root@localhost gopath]# cd src/github.com/
[root@localhost github.com]# ll
總用量 40544
drwxr-xr-x. 3 root root       22 8月  27 15:30 golang
drwxr-xr-x. 3 root root       18 8月  26 18:22 gonum
drwxr-xr-x. 3 root root       26 8月  27 07:56 mattn
drwxr-xr-x. 3 root root       25 8月  27 07:56 olekukonko
drwxr-xr-x. 3 root root       19 8月  27 07:56 peterh
drwxr-xr-x. 3 root root       20 8月  27 07:55 pkg
drwxr-xr-x. 4 root root       32 8月  27 07:55 spf13
drwxr-xr-x. 3 root root       24 8月  27 15:31 tensorflow
-rw-r--r--. 1 root root 41515134 8月  27 15:22 tensorflow.zip
drwxr-xr-x. 3 root root       18 8月  27 07:53 ynqa
[root@localhost github.com]#

解壓tensorflow.zip,得到tensorflow目錄,回到/home/gopath目錄進行構建,命令如下

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
can't load package: package github.com/tensorflow/tensorflow/tensorflow/go/op: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/op" in any of:
	/usr/lib/golang/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOROOT)
	/home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOPATH)
[root@localhost gopath]#

報錯,說找不到該目錄,下載的源碼怎么沒有該目錄內,直接cd查看

[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op: 沒有那個文件或目錄
[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/: 沒有那個文件或目錄
[root@localhost gopath]#

確實沒有該目錄,回到解壓處查看,發現源碼解壓后只有兩層tensorflow,而go get的有三層,則我們直接增加一層tensorflow,在進行構建

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op

package google.golang.org/protobuf/encoding/prototext: unrecognized import path "google.golang.org/protobuf/encoding/prototext" (https fetch: Get https://google.golang.org/protobuf/encoding/prototext?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/encoding/protowire: unrecognized import path "google.golang.org/protobuf/encoding/protowire" (https fetch: Get https://google.golang.org/protobuf/encoding/protowire?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/proto: unrecognized import path "google.golang.org/protobuf/proto" (https fetch: Get https://google.golang.org/protobuf/proto?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoreflect: unrecognized import path "google.golang.org/protobuf/reflect/protoreflect" (https fetch: Get https://google.golang.org/protobuf/reflect/protoreflect?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoregistry: unrecognized import path "google.golang.org/protobuf/reflect/protoregistry" (https fetch: Get https://google.golang.org/protobuf/reflect/protoregistry?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoiface: unrecognized import path "google.golang.org/protobuf/runtime/protoiface" (https fetch: Get https://google.golang.org/protobuf/runtime/protoiface?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoimpl: unrecognized import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
../genop/main.go:17: running "bash": exit status 1
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]# 

報錯protobuf導入超時

import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout

解決方法參考 go get 找不到 google.golang.org/protobuf github.com/gin-gonic/gin 解決辦法 也就是從GitHub上clone代碼下來,如下操作

[root@localhost gopath]# mkdir -p $GOPATH/src/google.golang.org
[root@localhost gopath]# git clone https://e.coding.net/robinqiwei/googleprotobuf.git $GOPATH/src/google.golang.org/protobuf
正克隆到 '/home/gopath/src/google.golang.org/protobuf'...
remote: Enumerating objects: 13821, done.
remote: Counting objects: 100% (13821/13821), done.
remote: Compressing objects: 100% (5172/5172), done.
remote: Total 13821 (delta 8112), reused 13821 (delta 8112), pack-reused 0
接收對象中: 100% (13821/13821), 7.44 MiB | 932.00 KiB/s, done.
處理 delta 中: 100% (8112/8112), done.
[root@localhost gopath]# 

完成后,在重新構建成功,如下

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
# github.com/golang/protobuf/protoc-gen-go
../../../../../golang/protobuf/protoc-gen-go/main.go:71:6: gen.SupportedFeatures undefined (type *protogen.Plugin has no field or method SupportedFeatures)
../../../../../golang/protobuf/protoc-gen-go/main.go:71:27: undefined: internal_gengo.SupportedFeatures
../genop/main.go:17: running "bash": exit status 2
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]#

這里發現exit status 1,也不知是啥問題,沒有去研究了。然后驗證安裝結果

[root@localhost gopath]# go test github.com/tensorflow/tensorflow/tensorflow/go
ok  	github.com/tensorflow/tensorflow/tensorflow/go	1.848s
[root@localhost gopath]# 

程序驗證,GO示例,創建tf-hello.go

[root@localhost gopath]# vim tf-hello.go 

package main

import (
    tf "github.com/tensorflow/tensorflow/tensorflow/go"
    "github.com/tensorflow/tensorflow/tensorflow/go/op"
    "fmt"
)
func main() {
    // Construct a graph with an operation that produces a string constant.
    s := op.NewScope()
    c := op.Const(s, "Hello from TensorFlow version " + tf.Version())
    graph, err := s.Finalize()
    if err != nil {
        panic(err)
    }
    // Execute the graph in a session.
    sess, err := tf.NewSession(graph, nil)
    if err != nil {
        panic(err)
    }
    output, err := sess.Run(nil, []tf.Output{c}, nil)
    if err != nil {
        panic(err)
    }
    fmt.Println(output[0].Value())
}

執行

[root@localhost gopath]# go run tf-hello.go
2020-08-27 15:44:47.868756: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-08-27 15:44:47.882914: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1799995000 Hz
2020-08-27 15:44:47.884161: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x22d3190 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-27 15:44:47.884211: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Hello from TensorFlow version 1.15.0
[root@localhost gopath]#

C示例,C和Go實際上共用了相同的動態鏈接庫和頭文件,安裝好Go版本后,C的API可以直接使用。創建tf.c

[root@localhost gopath]# vim tf.c 

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
    printf("Hello from TensorFlow C library version %s\n", TF_Version());
    return 0;
}

編譯,執行

[root@localhost gopath]# gcc -I/usr/local/include -L/usr/local/lib tf.c -ltensorflow
[root@localhost gopath]# ll
總用量 1988
-rwxr-xr-x. 1 root root    8496 8月  27 15:45 a.out
drwxr-xr-x. 3 root root      25 8月  27 15:40 pkg
drwxr-xr-x. 5 root root      65 8月  27 15:40 src
-rwxr-xr-x. 1 root root 2008751 8月  26 17:56 test
-rw-r--r--. 1 root root      74 8月  26 17:56 test.go
-rw-r--r--. 1 root root     157 8月  27 15:45 tf.c
-rw-r--r--. 1 root root     671 8月  27 15:44 tf-hello.go
[root@localhost gopath]# ./a.out 
Hello from TensorFlow C library version 1.15.0
[root@localhost gopath]# 

至此,整個環境已經整合完成,后續准備使用golang調用TensorFlow模型。


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM