ImportError: dlopen: cannot load any more object with static TLS 錯誤原因定位及解決辦法


https://github.com/pytorch/pytorch/issues/2575  帖子對於此問題的討論非常深刻,除了更換import順序的通用做法,也指出了問題的根本原因:

 

By suo

Summarizing what I've learned so far. I may be wrong about any of these things, so please correct/take with a grain of salt.

glibc has a table called the DTV. There is a slot for every dlopen'd library. Its use is not important for this discussion.

The DTV is resizable. However, in older versions of glibc, adding a library with static TLS will not resize the DTV, but do a conservative check that amounts to "have a I loaded more than 14 libraries of any kind".

That's why #24911 failed to fix anything, because reducing the amount of thread-local storage is irrelevant.

That's also why changing import order can fix things, because if you change it in a way that loads all your "static TLS" libraries first, then future "dynamic TLS" libraries will resize the DTV like normal

It seems this issue was fixed by a glibc patch in 2014, which eliminates this check and lazily updates the DTV.  (解決方案一:升級python依賴的glibc庫)

 

 

One additional takeaway: this issue was fixed by a 2014 glibc patch. This patch is in the glibc distributed with Xenial.

Trusty is considered unsupported by Ubuntu (the LTS commitment expired in April of this year). We are also removing trusty from our CI (cc @jamesr66a). So if you want this problem to really really (tm) go away, upgrading your linux is another path. (解決方案二:升級操作系統版本,同時升級或者重編譯python開發環境)


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM