AnaConda環境下安裝librosa包超時

問題重現

大家在AnaConda環境下安裝包的過程中肯定會遇到下載緩慢的問題。

超時信息如下（為了大家方便通過超時信息找到，犧牲了一些頁面體驗，把異常信息貼出來）：

 conda.exceptions.CondaRuntimeError: Runtime error: Could not open '/home/ubuntu/anaconda3/pkgs/icu-64.2-he1b5a44_1.tar.bz2.part' 

for writing (HTTPSConnectionPool(host='conda.anaconda.org', port=443): Read timed out.).
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/exceptions.py", line 473, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/main.py", line 144, in _main
        exit_code = args.func(args, p)
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/main_install.py", line 80, in execute
        install(args, parser, 'install')
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/install.py", line 422, in install
        raise CondaSystemExit('Exiting', e)
      File "/home/ubuntu/anaconda3/lib/python3.5/contextlib.py", line 77, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/common.py", line 573, in json_progress_bars
        yield
      File "/home/ubuntu/anaconda3/lib/python3.5/site-packages/conda/cli/install.py", line 420, in install
        raise CondaRuntimeError('RuntimeError: %s' % e)
    conda.exceptions.CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open '/home/ubuntu/anaconda3/pkgs/icu-64.2-he1b5a44_1.tar.bz2.part' 

for writing (HTTPSConnectionPool(host='conda.anaconda.org', port=443): Read timed out.).

 

或者如下：

Fetching package metadata ...INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
.INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io
.INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io
.........Could not connect to https://conda.anaconda.org/conda-forge/linux-64/
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): conda.anaconda.org
..INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.ustc.edu.cn
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): mirrors.tuna.tsinghua.edu.cn
...INFO requests.packages.urllib3.connectionpool:_get_conn(249): Resetting dropped connection: mirrors.ustc.edu.cn
...INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (1): repo.continuum.io
WARNING requests.packages.urllib3.connectionpool:urlopen(664): Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after 
connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='repo.continuum.io', port=443): Read timed out. (read timeout=3.05)",)': /pkgs/free/linux-64/repodata.json.bz2
INFO requests.packages.urllib3.connectionpool:_new_conn(805): Starting new HTTPS connection (2): repo.continuum.io

解決方法

通常的解決方法是切換到國內的鏡像源

如conda切換到中科大源，命令行下執行

conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/main/
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free/
conda config --add channels  https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/

pip切換源參考這里：pip設置安裝源

 但是有時候即便切換了源，有些頁面因為網絡或者服務器原因下載仍舊是緩慢，直至出現超時。

筆者最近下載librosa，執行以下命令，最終超時下載不了。

conda install librosa -c conda-forge

后來又添加了一個-v參數查看執行命令情況

conda install librosa -v -c conda-forge

發現有執行超時的情況

 后來在stackoverflow上面找到了答案，AnaConda在4.3版本以下讀取超時都是被硬編碼成60秒，這個顯然是不合理的。

I had the same issue. In conda < 4.3.0, the timeout was hardcoded to 60 seconds. In Windows, I'm guessing, that's not enough for huge package with a lot of binary

files like qt since most likely the virus scanner kicks in to check the files. You can hack your ~userid\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda\fetch.py

and change the hard-coded constant yourself, eg. from 60 to 300.

With newer version (I'm not sure when this was added, but I see this in my conda 4.3.22), running conda config --show gave: remote_read_timeout_secs: 60.0.

So I believe you can modify it by adding it to your ~userid\.condarc file.

 

筆者本地的是4.2.0版本Anaconda3-4.2.0-Linux-x86_64.sh

嘗試執行命令報錯：

conda config --set remote_read_timeout_secs 3600

報錯信息：

ubuntu@ubuntu-B85-D3V:~$ conda config --set remote_read_timeout_secs 3600
CondaValueError: Value error: Error key must be one of add_binstar_token, update_dependencies, binstar_upload, 
always_copy, anaconda_upload, add_pip_as_python_dependency, allow_other_channels, channel_priority, changeps1, always_yes, 
shortcuts, channel_alias, use_pip, auto_update_conda, add_anaconda_token, ssl_verify, client_cert_key, client_cert, 
show_channel_urls, offline, allow_softlinks, not remote_read_timeout_secs

也就是當前版本4.2.0還不支持這個remote_read_timeout_secs環境變量設置

筆者在最新版的官方文檔上找到超時相關資料，各位可以根據實際情況酌情使用這兩個參數：

遠程連接超時（默認9.15秒）

# # remote_connect_timeout_secs (float)
# #   The number seconds conda will wait for your client to establish a
# #   connection to a remote url resource.
# # 
# remote_connect_timeout_secs: 9.15

遠程讀取超時（默認60秒）

# # remote_read_timeout_secs (float)
# #   Once conda has connected to a remote resource and sent an HTTP
# #   request, the read timeout is the number of seconds conda will wait for
# #   the server to send a response.
# # 
# remote_read_timeout_secs: 60.0

 最大重試次數（默認3次）

# # remote_max_retries (int)
# #   The maximum number of retries each HTTP connection should attempt.
# # 
# remote_max_retries: 3

 

結論

升級AnaConda版本或者更換鏡像源

wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh
chmod +x Anaconda3-2019.10-Linux-x86_64.sh

安裝過程很簡單，一路回車即可

 

 

 

 

 

 

配置環境變量

vim ~/.bashrc

# added by Anaconda3 4.2.0 installer
export PATH="/home/ubuntu/anaconda3/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-10.2
export PATH="$CUDA_HOME/bin:$PATH"
export LD_LIBRARY_PATH="$CUDA_HOME/lib64:$LD_LIBRARY_PATH"

配置屬性

#使環境變量生效
source ~/.bashrc
#設置讀取超時(1小時3600秒)
source activate xxx
conda config --set remote_read_timeout_secs 3600

 

參考來源：https://stackoverflow.com/questions/42797957/changing-timeout-limit-when-using-conda-install

https://docs.conda.io/projects/conda/en/latest/configuration.html