更正歷史
- 2020.05.22 完成初稿
- 2020.06.03 增加 jupyter notebook 配置說明
- 2020.08.28 修正了配置 jupyter notebook 時可能出現
KeyError: 'allow_remote_access'錯誤- 2021.10.24 修復圖片鏈接失效問題
在上一篇文章PyCharm 部署 Docker 鏡像解釋器中，我們詳細說明了如何將 PyCharm 的 python 解釋器指向 Docker 鏡像中，而與模型相關的代碼和數據依然保留在主機上。上述環境的配置不適用於需要對容器進行更改的場景，比如通過 PyCharm 安裝新的 python 包。本文將說明如何讓 PyCharm 指向 Docker 容器（而非鏡像），來實現需要經常改動容器的場景。
實驗環境：
- PyCharm：2020.1
- 服務器：Ubuntu 16.04.4 LTS
- Docker：19.03.4, build 9013bf583a
配置過程主要包括以下幾個步驟：
- 在服務器上，使用指定的鏡像新建容器，並指定端口映射；
- 在服務器上，進入容器，配置 ssh；
- 在自己電腦上，配置 PyCharm
1 新建容器並指定映射端口
# 將 Docker 容器的 22 端口映射到宿主機的 10022 端口上（注：假定服務器上的 share_data Volumes已經創建好）
~]# docker run -it --name tf2_gpu --gpus all --volumes-from share_data -p 10022:22 tensorflow/tensorflow:latest-gpu /bin/bash
________ _______________
___ __/__________________________________ ____/__ /________ __
__ / _ _ \_ __ \_ ___/ __ \_ ___/_ /_ __ /_ __ \_ | /| / /
_ / / __/ / / /(__ )/ /_/ / / _ __/ _ / / /_/ /_ |/ |/ /
/_/ \___//_/ /_//____/ \____//_/ /_/ /_/ \____/____/|__/
WARNING: You are running this container as root, which can cause new files in
mounted volumes to be created as the root user on your host machine.
To avoid this, run the container by specifying your user's userid:
$ docker run -u $(id -u):$(id -g) args...
root@ed13c547d9df:/#
2 配置 ssh 服務
root@ed13c547d9df:/# apt update
# 安裝 openssh-sever，讓本機開放SSH服務
root@ed13c547d9df:/# apt install openssh-server
# 安裝文本編輯器
root@ed13c547d9df:/# apt install vim
# 配置 ssh
root@ed13c547d9df:/# vim +/PermitRootLogin /etc/ssh/sshd_config
增加一行：PermitRootLogin yes
# 保存退出，重啟 ssh 服務
root@ed13c547d9df:/# /etc/init.d/ssh restart
* Restarting OpenBSD Secure Shell server sshd [ OK ]
#設置 root 密碼
root@ed13c547d9df:/# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
3 配置 jupyter
- 安裝: pip install jupyter
- 配置 jupyter
（1）生成 jupyter 配置文件
~]# jupyter notebook --generate-config
（2） 生成 jupyter 密碼
>>> from notebook.auth import passwd
>>> passwd() # 比如，密碼設置為 hello。操作步驟為：輸入 hello 兩次，得到 hello 的 hash 值
Enter password:
Verify password:
'sha1:e7a41b1987ae:b828d19c03a6e122da4348c2f143d855e980f59f'
（3）修改 jupyter 配置文件
vim ~/.jupyter/jupyter_notebook_config.py
#c.NotebookApp.allow_root = True
c.NotebookApp.ip = '*'
c.NotebookApp.password = u'sha1:e7a41b1987ae:b828d19c03a6e122da4348c2f143d855e980f59f'
c.NotebookApp.open_browser = False
c.NotebookApp.allow_remote_access = True
c.NotebookApp.port = 8889
測試 ssh 和 jupyter 配置是否成功
回到自己的電腦，測試是否能夠 ssh 服務器上的 Docker 容器
C:\Users\leaf>ssh root@192.168.7.165 -p 1122
___ __/__________________________________ ____/__ /________ __
__ / _ _ \_ __ \_ ___/ __ \_ ___/_ /_ __ /_ __ \_ | /| / /
_ / / __/ / / /(__ )/ /_/ / / _ __/ _ / / /_/ /_ |/ |/ /
/_/ \___//_/ /_//____/ \____//_/ /_/ /_/ \____/____/|__/
WARNING: You are running this container as root, which can cause new files in
mounted volumes to be created as the root user on your host machine.
To avoid this, run the container by specifying your user's userid:
$ docker run -u $(id -u):$(id -g) args...
root@ed13c547d9df:~# jupyter notebook --allow-root
[I 10:01:48.764 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[W 10:01:49.309 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 10:01:49.314 NotebookApp] Serving notebooks from local directory: /
[I 10:01:49.314 NotebookApp] The Jupyter Notebook is running at:
[I 10:01:49.314 NotebookApp] http://e1b8d553ab8c:8889/
[I 10:01:49.314 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
出現如上結果，則表示配置成功。
4 配置 PyCharm
接下來的配置工作與普通的服務器上的配置一樣。
(1) 新建一項目，名稱為「wander」
（2）選擇 「Existing Interpreter」右邊的 「...」，新建 Python 解釋器。界面如下所示
輸入密碼后，
選擇 python 解釋器，完成配置。
（3）新建「connectivity_test.py] python 文件，內容如下：
import sys
import tensorflow as tf
print(sys.executable)
print('\n', tf.test.is_gpu_available())
（4）設置 Run 配置信息：「Run」 -> [Run」->「Edit Configurations..」
（5）運行結果如下
ssh://root@192.168.7.165:1122/usr/bin/python3 -u /tmp/pycharm_project_917/connectivity_test.py
/usr/bin/python3
WARNING:tensorflow:From /tmp/pycharm_project_917/connectivity_test.py:5: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2020-05-22 09:48:58.264438: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-05-22 09:48:58.279597: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 1699975000 Hz
2020-05-22 09:48:58.280972: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ffa78000b20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-05-22 09:48:58.281008: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-05-22 09:48:58.286453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-05-22 09:48:58.631613: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3c5c5e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-05-22 09:48:58.631658: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce GTX 1080 Ti, Compute Capability 6.1
2020-05-22 09:48:58.631671: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (1): GeForce GTX 1080 Ti, Compute Capability 6.1
2020-05-22 09:48:58.633921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-05-22 09:48:58.636050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 1 with properties:
pciBusID: 0000:82:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-05-22 09:48:58.636519: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-22 09:48:58.640295: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-05-22 09:48:58.643689: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-05-22 09:48:58.644268: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-05-22 09:48:58.648118: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-05-22 09:48:58.650245: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-05-22 09:48:58.658191: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-05-22 09:48:58.665598: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0, 1
2020-05-22 09:48:58.665657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-22 09:48:58.669790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-05-22 09:48:58.669815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 1
2020-05-22 09:48:58.669826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N N
2020-05-22 09:48:58.669834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 1: N N
2020-05-22 09:48:58.675256: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/device:GPU:0 with 10371 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
2020-05-22 09:48:58.677797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/device:GPU:1 with 10371 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0, compute capability: 6.1)
True
Process finished with exit code 0
從輸出結果的第一行可以看到，PyCharm 使用的解釋器為 Docker 容器中的。當看到輸入結果為 True，則表示可以調用 GPU 了。
參考資料
Ubuntu環境下SSH的安裝及使用
jupyter 遠程訪問配置：KeyError: 'allow_remote_access'