聯結地址：https://www.cnblogs.com/dangkai/p/9413932.html 此處簡單有介紹，如果聽不懂，請看下文：鏈接地址
PEP 405 -- Python Virtual Environments
| PEP: | 405 |
|---|---|
| Title: | Python Virtual Environments |
| Author: | Carl Meyer <carl at oddbird.net> |
| BDFL-Delegate: | Nick Coghlan |
| Status: | Final |
| Type: | Standards Track |
| Created: | 13-Jun-2011 |
| Python-Version: | 3.3 |
| Post-History: | 24-Oct-2011, 28-Oct-2011, 06-Mar-2012, 24-May-2012 |
| Resolution: | https://mail.python.org/pipermail/python-dev/2012-May/119668.html |
Contents
摘要
PEP意圖在python中添加一種輕量級的“虛擬環境”機制，該機制具有自己的站點（site directories）目錄；可以選擇與系統站點目錄隔離。每個虛擬環境都有自己的python二進制文件（允許創建具有不同python版本的環境），並且可以在其站點目錄中有自己獨立的一組已安裝的python包，但可以與基本已經安裝的python共享標准庫。
需求動機
現有第三方虛擬環境工具（主要是Ian Bicking的virtualenv）的普及，已經很好地證明了python虛擬環境的實用性。虛擬環境已經廣泛用於依賴關系管理和隔離，在無需系統管理員介入前提下，易於安裝和使用python包，以及跨多個python版本自動測試python軟件等用途。
過去已有的虛擬環境工具缺乏對Python本身行為的支持。諸如rvirtualenv[2]這樣的工具無法將python二進制文件復制到虛擬環境中，因此無法提供與系統站點目錄的可靠隔離。virtualenv復制了python二進制文件，它強制復制python的大部分站點模塊，並手動將一組不斷變化的標准庫模塊symlink/復制到虛擬環境中，以便在每次啟動時執行一個微妙的引導綁定動作。（virtualenv必須復制二進制文件以提供隔離，因為python在搜索sys.prefix之前，python會取消對符號鏈接可執行文件的引用。注意python的啟動過程）
PYTHONHOME環境變量是Python現有的唯一針對虛擬環境的內置解決方案，它需要將整個標准庫復制/符號化鏈接到每個環境中。如果采用復制整個標准庫的方法，就不是一個輕量級的解決方案。對符號鏈接的跨平台支持仍然不一致（即使在支持它們的Windows平台上，創建它們通常需要管理員特權）。
一個與Python集成的虛擬環境機制，並利用現有第三方工具的多年經驗，可以降低維護，提高可靠性，並且更容易為所有Python用戶方便使用。
特點
當執行python二進制文件時，它首先要做的確定其前綴（前綴存儲在sys.prefix中）；然后使用該前綴查找標准庫和其他密鑰文件，並由站點模塊確定站點包目錄的位置。以上是在設置PythonHome環境變量的前提下；假如沒設定環境變量，當前的前綴將從遍歷文件，通過首先在文件系統樹上查找標記文件（os.py），找到前綴。如果還是找不到，則返回到二進制文件中硬編碼的構建時前綴。
PEP的方案是：在此搜索中添加新的第一步。如果pyvenv.cfg文件靠近python.exe文件或其上的一個目錄,（如果可執行文件是symlink，則不會取消引用），則會掃描pyvenv.cfg中的key=value格式行。如果找到了主密鑰，這意味着python二進制文件屬於虛擬環境，並且主密鑰的值是包含用於創建此虛擬環境的python可執行文件的目錄。
在這種情況下，前綴查找將繼續正常使用主鍵的值作為有效的Python二進制位置，該位置可查找基本安裝的前綴。 sys.base_prefix設置為此值，而sys.prefix設置為包含pyvenv.cfg的目錄。
（如果未找到pyvenv.cfg或不包含主鍵，則前綴查找將繼續正常，並且sys.prefix將等於sys.base_prefix。）
此外，添加了sys.base_exec_prefix，並且對sys.exec_prefix進行了類似的處理。 （sys.exec_prefix相當於sys.prefix，但對於特定於平台的文件;默認情況下，它與sys.prefix具有相同的值。）
修改了站點和sysconfig標准庫模塊，以便找到相對於sys.base_prefix / sys.base_exec_prefix的標准庫和頭文件，而site-package目錄（“purelib”和“platlib”，在sysconfig術語中）仍然是找到相對於sys.prefix / sys.exec_prefix。
因此，最簡單形式的Python虛擬環境只包含Python二進制文件的副本或符號鏈接，並附帶pyvenv.cfg文件和site-packages目錄。
從系統站點包隔離
默認情況下，虛擬環境與系統級site-packages目錄完全隔離。
如果pyvenv.cfg文件還包含值為true（不區分大小寫）的鍵include-system-site-packages，則站點模塊還將在虛擬環境站點目錄之后將系統站點目錄添加到sys.path。因此，系統安裝的包仍然是可導入的，但是在虛擬環境中安裝的同名包將優先。
PEP 370用戶級站點包被認為是用於venv目的的系統站點包的一部分：它們不是從隔離的venv中獲得的，而是可以從include-system-site-packages = true venv獲得。
Creating virtual environments
該PEP還建議在標准庫中添加新的venv模塊，以實現虛擬環境的創建。可以使用-m標志執行此模塊：
python3 -m venv /path/to/new/virtual/environment
還提供了pyvenv安裝的腳本，以使這更方便：
pyvenv /path/to/new/virtual/environment
運行此命令將創建目標目錄（創建不存在的任何父目錄）並在其中放置pyvenv.cfg文件，其中主鍵指向運行該命令的Python安裝。它還創建了一個bin /（或Windows上的Scripts）子目錄，其中包含python3可執行文件的副本（或符號鏈接），以及來自打包標准庫模塊的pysetup3腳本（以便於將包從PyPI輕松安裝到新的venv中）。它創建了一個（最初為空）lib / pythonX.Y / site-packages（或Windows上的Lib \ site-packages）子目錄。
如果目標目錄已經存在，則會引發錯誤，除非提供了--clear選項，在這種情況下，目標目錄將被刪除，虛擬環境創建將照常進行。 創建的pyvenv.cfg文件還包括include-system-site-packages密鑰，如果pyvenv使用--system-site-packages選項運行，則設置為true，默認情況下為false。 可以為pyvenv提供多個路徑，在這種情況下，根據給定的選項，在每個提供的路徑上將創建相同的venv。
venv模塊還在venv的bin或Scripts目錄中為POSIX和Windows系統放置“shell激活腳本”。這些腳本只是將虛擬環境的bin（或Scripts）目錄添加到用戶shell PATH的前面。這對於虛擬環境的使用並不是絕對必要的（作為venv的python二進制文件的顯式路徑或者腳本也可以使用），但它很方便。
為了允許pysetup和其他Python包管理器以與安裝到普通Python安裝相同的方式將軟件包安裝到虛擬環境中，並避免在sysconfig中使用sys.base_prefix代替sys.prefix進行特殊套管虛擬環境。適當的，內部虛擬環境布局模仿每個平台上Python安裝本身的布局。因此，POSIX系統上的典型虛擬環境布局將是：
pyvenv.cfg bin/python3 bin/python bin/pysetup3 include/ lib/python3.3/site-packages/
在Windows系統上：
pyvenv.cfg
Scripts/python.exe
Scripts/python3.dll
Scripts/pysetup3.exe
Scripts/pysetup3-script.py
... other DLLs and pyds...
Include/
Lib/site-packages/
安裝到虛擬環境中的第三方軟件包將其Python模塊放置在site-packages目錄中，並將其可執行文件放在bin /或Scripts中。 注意 在正常的Windows系統級安裝中，Python二進制文件本身不會進入“Scripts /”子目錄，就像在默認的venv布局中一樣。這在虛擬環境中很有用，因此用戶只需將單個目錄添加到其shell PATH中，以便有效地“激活”虛擬環境。 注意 在Windows上，還需要將編譯的stdlib模塊中的DLL和pyd文件復制或符號鏈接到env，因為如果從非系統范圍的Python安裝創建venv，Windows將無法找到Python安裝的從venv運行Python時這些文件的副本。
Sysconfig安裝方案和用戶站點
這種方法明確選擇不為venvs引入新的sysconfig安裝方案。相反，通過修改sys.prefix，我們確保在sys.prefix上基於位置的現有安裝方案將僅在venv中工作。安裝到其路徑與sys.prefix無關的其他安裝方案（例如，用戶站點方案）根本不會受到venv的影響。
基於特定於虛擬的sysconfig方案創建Python虛擬環境的替代實現可能是可行的，但它不太健壯，因為需要更多代碼才能知道它是否在虛擬環境中運行。
拷貝和符號連接
The technique in this PEP works equally well in general with a copied or symlinked Python binary (and other needed DLLs on Windows). Symlinking is preferable where possible, because in the case of an upgrade to the underlying Python installation, a Python executable copied in a venv might become out-of-sync with the installed standard library and require manual upgrade.
There are some cross-platform difficulties with symlinks:
- Not all Windows versions support symlinks, and even on those that do, creating them often requires administrator privileges.
- On OS X framework builds of Python, sys.executable is just a stub that executes the real Python binary. Symlinking this stub does not work; it must be copied. (Fortunately the stub is also small, and not changed by bugfix upgrades to Python, so copying it is not an issue).
Thus, this PEP proposes to symlink the binary on all platforms except for Windows, and OS X framework builds. A --symlink option is available to force the use of symlinks on Windows versions that support them, if the appropriate permissions are available. (This option has no effect on OS X framework builds, since symlinking can never work there, and has no advantages).
On Windows, if --symlink is not used, this means that if the underlying Python installation is upgraded, the Python binary and DLLs in the venv should be updated, or there could be issues of mismatch with the upgraded standard library. The pyvenv script accepts a --upgrade option for easily performing this upgrade on an existing venv.
Include files
Current virtualenv handles include files in this way:
On POSIX systems where the installed Python's include files are found in ${base_prefix}/include/pythonX.X, virtualenv creates ${venv}/include/ and symlinks ${base_prefix}/include/pythonX.X to ${venv}/include/pythonX.X. On Windows, where Python's include files are found in {{ sys.prefix }}/Include and symlinks are not reliably available, virtualenv copies {{ sys.prefix }}/Include to ${venv}/Include. This ensures that extension modules built and installed within the virtualenv will always find the Python header files they need in the expected location relative to sys.prefix.
This solution is not ideal when an extension module installs its own header files, as the default installation location for those header files may be a symlink to a system directory that may not be writable. One installer, pip, explicitly works around this by installing header files to a nonstandard location ${venv}/include/site/pythonX.X/, as in Python there's currently no standard abstraction for a site-specific include directory.
This PEP proposes a slightly different approach, though one with essentially the same effect and the same set of advantages and disadvantages. Rather than symlinking or copying include files into the venv, we simply modify the sysconfig schemes so that header files are always sought relative to base_prefix rather than prefix. (We also create an include/ directory within the venv, so installers have somewhere to put include files installed within the env).
Better handling of include files in distutils/packaging and, by extension, pyvenv, is an area that may deserve its own future PEP. For now, we propose that the behavior of virtualenv has thus far proved itself to be at least "good enough" in practice.
API
The high-level method described above makes use of a simple API which provides mechanisms for third-party virtual environment creators to customize environment creation according to their needs.
The venv module contains an EnvBuilder class which accepts the following keyword arguments on instantiation:
- system_site_packages - A Boolean value indicating that the system Python site-packages should be available to the environment. Defaults to False.
- clear - A Boolean value which, if true, will delete any existing target directory instead of raising an exception. Defaults to False.
- symlinks - A Boolean value indicating whether to attempt to symlink the Python binary (and any necessary DLLs or other binaries, e.g. pythonw.exe), rather than copying. Defaults to False.
The instantiated env-builder has a create method, which takes as required argument the path (absolute or relative to the current directory) of the target directory which is to contain the virtual environment. The create method either creates the environment in the specified directory, or raises an appropriate exception.
The venv module also provides a module-level create function as a convenience:
def create(env_dir,
system_site_packages=False, clear=False, use_symlinks=False):
builder = EnvBuilder(
system_site_packages=system_site_packages,
clear=clear,
use_symlinks=use_symlinks)
builder.create(env_dir)
Creators of third-party virtual environment tools are free to use the provided EnvBuilder class as a base class.
The create method of the EnvBuilder class illustrates the hooks available for customization:
def create(self, env_dir):
"""
Create a virtualized Python environment in a directory.
:param env_dir: The target directory to create an environment in.
"""
env_dir = os.path.abspath(env_dir)
context = self.create_directories(env_dir)
self.create_configuration(context)
self.setup_python(context)
self.post_setup(context)
Each of the methods create_directories, create_configuration, setup_python, and post_setup can be overridden. The functions of these methods are:
- create_directories - creates the environment directory and all necessary directories, and returns a context object. This is just a holder for attributes (such as paths), for use by the other methods.
- create_configuration - creates the pyvenv.cfg configuration file in the environment.
- setup_python - creates a copy of the Python executable (and, under Windows, DLLs) in the environment.
- post_setup - A (no-op by default) hook method which can be overridden in third party subclasses to pre-install packages or install scripts in the virtual environment.
In addition, EnvBuilder provides a utility method that can be called from post_setup in subclasses to assist in installing custom scripts into the virtual environment. The method install_scripts accepts as arguments the context object (see above) and a path to a directory. The directory should contain subdirectories "common", "posix", "nt", each containing scripts destined for the bin directory in the environment. The contents of "common" and the directory corresponding to os.name are copied after doing some text replacement of placeholders:
- __VENV_DIR__ is replaced with absolute path of the environment directory.
- __VENV_NAME__ is replaced with the environment name (final path segment of environment directory).
- __VENV_BIN_NAME__ is replaced with the name of the bin directory (either bin or Scripts).
- __VENV_PYTHON__ is replaced with the absolute path of the environment's executable.
The DistributeEnvBuilder subclass in the reference implementation illustrates how the customization hook can be used in practice to pre-install Distribute into the virtual environment. It's not envisaged that DistributeEnvBuilder will be actually added to Python core, but it makes the reference implementation more immediately useful for testing and exploratory purposes.
Backwards Compatibility
Splitting the meanings of sys.prefix
Any virtual environment tool along these lines (which attempts to isolate site-packages, while still making use of the base Python's standard library with no need for it to be symlinked into the virtual environment) is proposing a split between two different meanings (among others) that are currently both wrapped up in sys.prefix: the answers to the questions "Where is the standard library?" and "Where is the site-packages location where third-party modules should be installed?"
This split could be handled by introducing a new sys attribute for either the former prefix or the latter prefix. Either option potentially introduces some backwards-incompatibility with software written to assume the other meaning for sys.prefix. (Such software should preferably be using the APIs in the site and sysconfig modules to answer these questions rather than using sys.prefix directly, in which case there is no backwards-compatibility issue, but in practice sys.prefix is sometimes used.)
The documentation [7] for sys.prefix describes it as "A string giving the site-specific directory prefix where the platform independent Python files are installed," and specifically mentions the standard library and header files as found under sys.prefix. It does not mention site-packages.
Maintaining this documented definition would mean leaving sys.prefix pointing to the base system installation (which is where the standard library and header files are found), and introducing a new value in sys (something like sys.site_prefix) to point to the prefix for site-packages. This would maintain the documented semantics of sys.prefix, but risk breaking isolation if third-party code uses sys.prefix rather than sys.site_prefix or the appropriate site API to find site-packages directories.
The most notable case is probably setuptools [3] and its fork distribute [4], which mostly use distutils and sysconfig APIs, but do use sys.prefix directly to build up a list of site directories for pre-flight checking where pth files can usefully be placed.
Otherwise, a Google Code Search [5] turns up what appears to be a roughly even mix of usage between packages using sys.prefix to build up a site-packages path and packages using it to e.g. eliminate the standard-library from code-execution tracing.
Although it requires modifying the documented definition of sys.prefix, this PEP prefers to have sys.prefix point to the virtual environment (where site-packages is found), and introduce sys.base_prefix to point to the standard library and Python header files. Rationale for this choice:
- It is preferable to err on the side of greater isolation of the virtual environment.
- Virtualenv already modifies sys.prefix to point at the virtual environment, and in practice this has not been a problem.
- No modification is required to setuptools/distribute.
Impact on other Python implementations
The majority of this PEP's changes occur in the standard library, which is shared by other Python implementations and should not present any problem.
Other Python implementations will need to replicate the new sys.prefix-finding behavior of the interpreter bootstrap, including locating and parsing the pyvenv.cfg file, if it is present.
Reference Implementation
The reference implementation is found in a clone of the CPython Mercurial repository [6]. To test it, build and run bin/pyvenv /path/to/new/venv to create a virtual environment.
References
| [1] | http://www.virtualenv.org |
| [2] | https://github.com/kvbik/rvirtualenv |
| [3] | http://peak.telecommunity.com/DevCenter/setuptools |
| [4] | http://packages.python.org/distribute/ |
| [5] | http://www.google.com/codesearch#search/&q=sys.prefix&p=1&type=cs |
| [6] | http://hg.python.org/sandbox/vsajip#venv |
| [7] | http://docs.python.org/dev/library/sys.html#sys.prefix |
Copyright
This document has been placed in the public domain.
Source: https://github.com/python/peps/blob/master/pep-0405.txt