作為OpenStack兩種基本的通信方式(RESTful API與消息總線)之中的一個。理解RESTful API的設計思路和運行過程，有助於我們對OpenStack有更好的理解。RESTful僅僅是設計風格而不是標准，Web服務中通常使用基於HTTP的符合RESTful風格的API。而WSGI(Web ServerGateway Interface)則是python語言中所定義的Webserver和Web應用程序或框架之間的通用接口標准。
在OpenStack中隨處可見基於WSGI的通信，如nova-api。keystone等等。其工作方式為：OpenStack的API服務進程接收到client的HTTP請求時。一個所謂的“路由”模塊會將請求的URL裝換成對應的資源，並路由到合適的操作函數上。
本文將介紹keystone的WSGI通信，並依據怎樣從keystone中獲取token信息的舉例方式進行介紹。
1. 創建WSGI server
keystone是通過/usr/bin/keyston-all腳本進行啟動的。
#/usr/bin/keyston-all
import os
import sys
# If ../keystone/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(__file__),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir,
'keystone',
'__init__.py')):
sys.path.insert(0, possible_topdir)
from keystone.server import eventlet as eventlet_server
if __name__ == '__main__':
eventlet_server.run(possible_topdir)
終於運行到/keystone/server/evenlet.py中的run函數。
#/keystone/server/eventlet.py
def run(possible_topdir):
dev_conf = os.path.join(possible_topdir,
'etc',
'keystone.conf')
config_files = None
if os.path.exists(dev_conf):
config_files = [dev_conf]
common.configure(
version=pbr.version.VersionInfo('keystone').version_string(),
config_files=config_files,
pre_setup_logging_fn=configure_threading)
paste_config = config.find_paste_config()
def create_servers():
admin_worker_count = _get_workers('admin_workers')
public_worker_count = _get_workers('public_workers')
servers = []
servers.append(create_server(paste_config,
'admin',
CONF.eventlet_server.admin_bind_host,
CONF.eventlet_server.admin_port,
admin_worker_count))
servers.append(create_server(paste_config,
'main',
CONF.eventlet_server.public_bind_host,
CONF.eventlet_server.public_port,
public_worker_count))
return servers
_unused, servers = common.setup_backends(
startup_application_fn=create_servers)
serve(*servers)
首先載入配置文件的配置選項，然后找到keystone的paste配置文件(paste_config= config.find_paste_config())。由於OpenStack使用paste的deploy組件來完畢WSGIserver和應用的構建。當中keystone的paste配置文件位於/usr/share/keystone文件夾下。文件名稱為keystone-dist-paste.ini。對於該配置文件的使用，我會在興許內容中進行介紹，更仔細的使用可參考paste的deploy的官方文檔(http://pythonpaste.org/deploy/)。
然后運行common.setup_backends方法載入backend driver和創建admin和main的WSGI的server。這里我們看看common.setup_backends怎樣操作的呢？
#/keystone/server/common.py
def setup_backends(load_extra_backends_fn=lambda: {},
startup_application_fn=lambda: None):
drivers = backends.load_backends()
drivers.update(load_extra_backends_fn())
res = startup_application_fn()
drivers.update(dependency.resolve_future_dependencies())
return drivers, res
#/keystone/backends.py
def load_backends():
# Configure and build the cache
cache.configure_cache_region(cache.REGION)
# Ensure that the identity driver is created before the assignment manager
# and that the assignment driver is created before the resource manager.
# The default resource driver depends on assignment, which in turn
# depends on identity - hence we need to ensure the chain is available.
_IDENTITY_API = identity.Manager()
_ASSIGNMENT_API = assignment.Manager()
DRIVERS = dict(
assignment_api=_ASSIGNMENT_API,
catalog_api=catalog.Manager(),
credential_api=credential.Manager(),
domain_config_api=resource.DomainConfigManager(),
endpoint_filter_api=endpoint_filter.Manager(),
endpoint_policy_api=endpoint_policy.Manager(),
federation_api=federation.Manager(),
id_generator_api=identity.generator.Manager(),
id_mapping_api=identity.MappingManager(),
identity_api=_IDENTITY_API,
oauth_api=oauth1.Manager(),
policy_api=policy.Manager(),
resource_api=resource.Manager(),
revoke_api=revoke.Manager(),
role_api=assignment.RoleManager(),
token_api=token.persistence.Manager(),
trust_api=trust.Manager(),
token_provider_api=token.provider.Manager())
auth.controllers.load_auth_methods()
return DRIVERS
參考這篇文章(http://bingotree.cn/?
p=150)以及《OpenStack設計與實現》，我們知道。在”DRIVERS”字典中，每個鍵值對都定義了一類keystone API實現，它們之間存在相互依賴的可能(所以在這里採用了依賴注入的設計模式)。例如以下：
#/keystone/identity/core.py:Manager
@dependency.provider('identity_api')
@dependency.requires('assignment_api', 'credential_api', 'id_mapping_api',
'resource_api', 'revoke_api')
class Manager(manager.Manager):
依賴注入模式就是在/keystone/common/dependency.py文件里實現的，對於上面的的作用為：假設一個class被加了@dependency.provider(‘xxx’),那么其就會生成一個實例，放置於_REGISTRY = {}全局變量中。名字叫做xxx。以后其它模塊假設想要引用這個模塊，那么僅僅須要在class前面加上@dependency.requires(‘xxx’)或@dependency.optional(‘xxx’)就能夠了。當加上了@dependency.requires后。這個模塊就會有一個叫做xxx的屬性。該屬性的值就是對xxx的引用。
例如說以下這個最簡單的樣例：
@dependency.provider('XXX')
class XXX():
pass
@dependency.requires('XXX')
class YYY():
pass
#然后YYY的實例就有了XXX這個屬性：
yyy = YYY()
print type(yyy.XXX)
當然啦。這里另一個細節那就是在初始化identity這個Manager的時候，我們的@dependency.requires(‘assignment_api’, ‘credential_api’, ‘token_api’)是不滿足要求的，由於這三個API我們還沒通過dependency.provider注冊。所以在源代碼中有這種凝視：”Objects must not rely on the existence of these attributes untilafter ‘resolve_future_dependencies’ has been called; they may not existbeforehand.”。這些require的東東須要等到resolve_future_dependencies被調用后才干被正常使用。
因此調用load_backends方法就載入了興許會使用的backend driver。然后運行res = startup_application_fn()代碼創建WSGI的server。
#/keystone/server/eventlet.py
def create_servers():
admin_worker_count = _get_workers('admin_workers')
public_worker_count = _get_workers('public_workers')
servers = []
servers.append(create_server(paste_config,
'admin',
CONF.eventlet_server.admin_bind_host,
CONF.eventlet_server.admin_port,
admin_worker_count))
servers.append(create_server(paste_config,
'main',
CONF.eventlet_server.public_bind_host,
CONF.eventlet_server.public_port,
public_worker_count))
return servers
#/keystone/server/eventlet.py
def _get_workers(worker_type_config_opt):
# Get the value from config, if the config value is None (not set), return
# the number of cpus with a minimum of 2.
worker_count = CONF.eventlet_server.get(worker_type_config_opt)
if not worker_count:
worker_count = max(2, processutils.get_worker_count())
return worker_count
#/keystone/server/eventlet.py
def create_server(conf, name, host, port, workers):
app = keystone_service.loadapp('config:%s' % conf, name)
server = environment.Server(app, host=host, port=port,
keepalive=CONF.eventlet_server.tcp_keepalive,
keepidle=CONF.eventlet_server.tcp_keepidle)
if CONF.eventlet_server_ssl.enable:
server.set_ssl(CONF.eventlet_server_ssl.certfile,
CONF.eventlet_server_ssl.keyfile,
CONF.eventlet_server_ssl.ca_certs,
CONF.eventlet_server_ssl.cert_required)
return name, ServerWrapper(server, workers)
#/keystone/service.py
def loadapp(conf, name):
# NOTE(blk-u): Save the application being loaded in the controllers module.
# This is similar to how public_app_factory() and v3_app_factory()
# register the version with the controllers module.
controllers.latest_app = deploy.loadapp(conf, name=name)
return controllers.latest_app
當中create_servers方法首先獲取欲創建的admin和main的WSGI server的進程個數(work count)，假設配置文件未定義創建的進程個數。則OpenStack將通過計算計算機的cpu個數與2進行比較。選擇最大的那個數作為創建的進程個數。我的環境沒有在配置文件里進行設置創建進程的個數，且本環境的cpu個數為4，因此這里會創建8個WSGI server進程(4個admin WSGI server和4個main WSGI server)。例如以下：
| [root@jun ~]# ps -elf | grep keystone 5 S keystone 8882 4329 0 80 0 - 103806 poll_s 11:06 ? 00:00:01 keystone-admin -DFOREGROUND 5 S keystone 8883 4329 0 80 0 - 103806 poll_s 11:06 ? 00:00:01 keystone-main -DFOREGROUND 4 S keystone 50511 1 1 80 0 - 86478 poll_s 20:06 ? 00:00:09 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50522 50511 0 80 0 - 114690 ep_pol 20:06 ? 00:00:01 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50523 50511 0 80 0 - 114193 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50524 50511 0 80 0 - 114527 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50525 50511 0 80 0 - 114585 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50526 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50527 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50528 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50529 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 0 S root 51358 7967 0 80 0 - 28161 pipe_w 20:17 pts/2 00:00:00 grep --color=auto keystone |
當中進程號為50511的進程為創建WSGI server的父進程，進程號為50522~50529的進程為8個WSGI server進程。
在得到欲創建的WSGI server的進程數后，則去運行create_server函數，該函數運行app = keystone_service.loadapp('config:%s' % conf, name)代碼來從paste配置文件里生成一個WSGI應用。這里僅僅是載入WSGI應用，並沒有運行生成的WSGI應用，僅僅有在keystone服務進程收到keystoneclient的HTTP請求時，才會觸發使用WSGI應用，對於怎樣使用該WSGI應用，我們會在下一小節進行具體分析。
在載入WSGI應用之后，創建WSGI server，且假設配置文件里使能了SSL，則需對創建的WSGI server進行SSL 設置。
#/keystone/common/environment/eventlet_server.py:Server
class Server(object):
"""Server class to manage multiple WSGI sockets and applications."""
def __init__(self, application, host=None, port=None, keepalive=False,
keepidle=None):
self.application = application
self.host = host or '0.0.0.0'
self.port = port or 0
# Pool for a green thread in which wsgi server will be running
self.pool = eventlet.GreenPool(POOL_SIZE)
self.socket_info = {}
self.greenthread = None
self.do_ssl = False
self.cert_required = False
self.keepalive = keepalive
self.keepidle = keepidle
self.socket = None
當中create_server僅僅是創建創建了一個Server對象。並沒有監聽對應的port(而創建這個Server對象的進程就是WSGI server的父進程，即上面舉例的50511號進程)。
最后create_server將創建的server和workers使用ServerWrapper類進行封裝，並將WSGI server的name(即admin和main)和封裝后的WSGI server進行返回。
最后setup_backends函數運行drivers.update(dependency.resolve_future_dependencies())后返回。回到/keystone/server/eventlet.py的run函數。終於run函數運行serve方法。該方法會創建dmin和main的WSGI server。
#/keystone/server/eventlet.py
def serve(*servers):
logging.warning(_('Running keystone via eventlet is deprecated as of Kilo '
'in favor of running in a WSGI server (e.g. mod_wsgi). '
'Support for keystone under eventlet will be removed in '
'the "M"-Release.'))
if max([server[1].workers for server in servers]) > 1:
launcher = service.ProcessLauncher()
else:
launcher = service.ServiceLauncher()
for name, server in servers:
try:
server.launch_with(launcher)
except socket.error:
logging.exception(_('Failed to start the %(name)s server') % {
'name': name})
raise
# notify calling process we are ready to serve
systemd.notify_once()
for name, server in servers:
launcher.wait()
由於admin和main的WSGI server的works都為4，所以運行lanuncher = service.ProcessLauncher()。即創建一個ProcessLauncher對象。
#/keystone/openstack/common/service.py:ProcessLauncher
class ProcessLauncher(object):
_signal_handlers_set = set()
@classmethod
def _handle_class_signals(cls, *args, **kwargs):
for handler in cls._signal_handlers_set:
handler(*args, **kwargs)
def __init__(self, wait_interval=0.01):
"""Constructor.
:param wait_interval: The interval to sleep for between checks
of child process exit.
"""
self.children = {}
self.sigcaught = None
self.running = True
self.wait_interval = wait_interval
rfd, self.writepipe = os.pipe()
self.readpipe = eventlet.greenio.GreenPipe(rfd, 'r')
self.handle_signal()
然后在對admin和main的WSGI server分別運行server.launch_with(launcher)語句。
#/keystone/server/eventlet.py:ServerWrapper
class ServerWrapper(object):
"""Wraps a Server with some launching info & capabilities."""
def __init__(self, server, workers):
self.server = server
self.workers = workers
def launch_with(self, launcher):
self.server.listen()
if self.workers > 1:
# Use multi-process launcher
launcher.launch_service(self.server, self.workers)
else:
# Use single process launcher
launcher.launch_service(self.server)
#/keystone/common/environment/eventlet_server.py:Server
def listen(self, key=None, backlog=128):
"""Create and start listening on socket.
Call before forking worker processes.
Raises Exception if this has already been called.
"""
# TODO(dims): eventlet's green dns/socket module does not actually
# support IPv6 in getaddrinfo(). We need to get around this in the
# future or monitor upstream for a fix.
# Please refer below link
# (https://bitbucket.org/eventlet/eventlet/
# src/e0f578180d7d82d2ed3d8a96d520103503c524ec/eventlet/support/
# greendns.py?at=0.12#cl-163)
info = socket.getaddrinfo(self.host,
self.port,
socket.AF_UNSPEC,
socket.SOCK_STREAM)[0]
try:
self.socket = eventlet.listen(info[-1], family=info[0],
backlog=backlog)
except EnvironmentError:
LOG.error(_LE("Could not bind to %(host)s:%(port)s"),
{'host': self.host, 'port': self.port})
raise
LOG.info(_LI('Starting %(arg0)s on %(host)s:%(port)s'),
{'arg0': sys.argv[0],
'host': self.host,
'port': self.port})
這里運行launch_with函數時，首先使用WSGI server的父進程對對應的port進行監聽。
| [root@jun ~]# netstat -tnulp | grep 50511 tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 50511/python tcp 0 0 0.0.0.0:35357 0.0.0.0:* LISTEN 50511/python |
如上所看到的。WSGI server的父進程(50511號進程)開啟兩個socket去分別監聽本環境的5000和35357號port，當中5000號port是為main的WSGI server提供的，35357號port為admin的WSGI server提供的。即WSGI server的父進程接收到5000號port的HTTP請求時。則將把該請求轉發給為main開啟的WSGI server去處理。而WSGI server的父進程接收到35357號port的HTTP請求時，則將把該請求轉發給為admin開啟的WSGI server去處理。
在開啟WSGI server的父進程的socket監聽后，才會正式創建admin和main的WSGI server。即運行launcher.launch_service(self.server,self.workers)語句。
#/keystone/openstack/common/service.py:ProcessLauncher
def launch_service(self, service, workers=1):
wrap = ServiceWrapper(service, workers)
LOG.info(_LI('Starting %d workers'), wrap.workers)
while self.running and len(wrap.children) < wrap.workers:
self._start_child(wrap)
Launch_service函數將會依據admin和main的workers創建workers個數的WSGIserver。當中創建一個WSGI server就將其會塞進wrap.children集合中(python的set類型)。
_start_child函數即為創建WSGIserver。
#/keystone/openstack/common/service.py:ProcessLauncher
def _start_child(self, wrap):
if len(wrap.forktimes) > wrap.workers:
# Limit ourselves to one process a second (over the period of
# number of workers * 1 second). This will allow workers to
# start up quickly but ensure we don't fork off children that
# die instantly too quickly.
if time.time() - wrap.forktimes[0] < wrap.workers:
LOG.info(_LI('Forking too fast, sleeping'))
time.sleep(1)
wrap.forktimes.pop(0)
wrap.forktimes.append(time.time())
pid = os.fork()
if pid == 0:
launcher = self._child_process(wrap.service)
while True:
self._child_process_handle_signal()
status, signo = self._child_wait_for_exit_or_signal(launcher)
if not _is_sighup_and_daemon(signo):
break
launcher.restart()
os._exit(status)
LOG.info(_LI('Started child %d'), pid)
wrap.children.add(pid)
self.children[pid] = wrap
return pid
當中在創建的子進程中運行(pid== 0)中運行launcher =self._child_process(wrap.service)創建一個Launcher對象，並載入/keystone/openstack/common/service.py:Services中的run_service函數。那么它怎樣載入run_service函數的呢？例如以下
#/keystone/openstack/common/service.py:ProcessLauncher
def _child_process(self, service):
self._child_process_handle_signal()
# Reopen the eventlet hub to make sure we don't share an epoll
# fd with parent and/or siblings, which would be bad
eventlet.hubs.use_hub()
# Close write to ensure only parent has it open
os.close(self.writepipe)
# Create greenthread to watch for parent to close pipe
eventlet.spawn_n(self._pipe_watcher)
# Reseed random number generator
random.seed()
launcher = Launcher()
launcher.launch_service(service)
return launcher
#/keystone/openstack/common/service.py:Launcher
class Launcher(object):
"""Launch one or more services and wait for them to complete."""
def __init__(self):
"""Initialize the service launcher.
:returns: None
"""
self.services = Services()
self.backdoor_port = eventlet_backdoor.initialize_if_enabled()
def launch_service(self, service):
"""Load and start the given service.
:param service: The service you would like to start.
:returns: None
"""
service.backdoor_port = self.backdoor_port
self.services.add(service)
#/keystone/openstack/common/service.py:Services
class Services(object):
def __init__(self):
self.services = []
self.tg = threadgroup.ThreadGroup()
self.done = event.Event()
def add(self, service):
self.services.append(service)
self.tg.add_thread(self.run_service, service, self.done)
從上面的代碼能夠看出，載入run_service函數是在/keystone/openstack/common/service.py:Launcher中的launch_service函數中進行載入。
當然，這里僅僅是簡單的載入run_service函數。並不會馬上運行該函數。run_service函數的運行是在/keystone/openstack/common/service.py:ProcessLauncher中_start_child中的這條語句運行的：status,signo = self._child_wait_for_exit_or_signal(launcher)。例如以下
#/keystone/openstack/common/service.py:ProcessLauncher
def _child_wait_for_exit_or_signal(self, launcher):
status = 0
signo = 0
# NOTE(johannes): All exceptions are caught to ensure this
# doesn't fallback into the loop spawning children. It would
# be bad for a child to spawn more children.
try:
launcher.wait()
except SignalExit as exc:
signame = _signo_to_signame(exc.signo)
LOG.info(_LI('Child caught %s, exiting'), signame)
status = exc.code
signo = exc.signo
except SystemExit as exc:
status = exc.code
except BaseException:
LOG.exception(_LE('Unhandled exception'))
status = 2
finally:
launcher.stop()
return status, signo
當運行launcher.wait()語句時，將去運行run_service函數。那么run_service函數做了哪些操作呢？
#/keystone/openstack/common/service.py:Services
@staticmethod
def run_service(service, done):
"""Service start wrapper.
:param service: service to run
:param done: event to wait on until a shutdown is triggered
:returns: None
"""
service.start()
done.wait()
這里run_service運行service.start則會去真正的創建WSGI server，這里service即為/keystone/common/environment/eventlet_server.py中的Server對象。即run_service運行Server對象的start函數。
#/keystone/common/environment/eventlet_server.py:Server
def start(self, key=None, backlog=128):
"""Run a WSGI server with the given application."""
if self.socket is None:
self.listen(key=key, backlog=backlog)
dup_socket = self.socket.dup()
if key:
self.socket_info[key] = self.socket.getsockname()
# SSL is enabled
if self.do_ssl:
if self.cert_required:
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
dup_socket = eventlet.wrap_ssl(dup_socket, certfile=self.certfile,
keyfile=self.keyfile,
server_side=True,
cert_reqs=cert_reqs,
ca_certs=self.ca_certs)
# Optionally enable keepalive on the wsgi socket.
if self.keepalive:
dup_socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
if self.keepidle is not None:
dup_socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE,
self.keepidle)
self.greenthread = self.pool.spawn(self._run,
self.application,
dup_socket)
#/keystone/common/environment/eventlet_server.py:Server
def _run(self, application, socket):
"""Start a WSGI server with a new green thread pool."""
logger = log.getLogger('eventlet.wsgi.server')
socket_timeout = CONF.eventlet_server.client_socket_timeout or None
try:
eventlet.wsgi.server(
socket, application, log=EventletFilteringLogger(logger),
debug=False, keepalive=CONF.eventlet_server.wsgi_keep_alive,
socket_timeout=socket_timeout)
except greenlet.GreenletExit:
# Wait until all servers have completed running
pass
except Exception:
LOG.exception(_LE('Server error'))
raise
終於在所創建的每一個子進程中創建一個協程來開啟一個WSGI server，即在_run函數中創建和開啟了一個WSGIserver。
那么在開啟了WSGI server后。當WSGI server的父進程收到有HTTP請求，則將其請求發送到合適的WSGIserver上進行處理，而WSGI server怎樣處理HTTP請求，我們將在下一節進行分析。
2. 分析keystoneclient到keystone的HTTP請求過程
這里我們利用上一篇文章《novalist命令的代碼流程》中的獲取token的HTTP請求作為樣例進行解說。即例如以下的HTTP請求。
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
眼下OpenStack使用paste的deploy組件完畢WSGIserver和應用的構建，且deploy的工作是基於.ini結尾的配置文件。這里基於keystone的配置文件為keystone-dist-paste.ini。其內容例如以下。
| # Keystone PasteDeploy configuration file.
[filter:debug] paste.filter_factory = keystone.common.wsgi:Debug.factory
[filter:request_id] paste.filter_factory = oslo_middleware:RequestId.factory
[filter:build_auth_context] paste.filter_factory = keystone.middleware:AuthContextMiddleware.factory
[filter:token_auth] paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory
[filter:admin_token_auth] paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory
[filter:json_body] paste.filter_factory = keystone.middleware:JsonBodyMiddleware.factory
[filter:user_crud_extension] paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory
[filter:crud_extension] paste.filter_factory = keystone.contrib.admin_crud:CrudExtension.factory
[filter:ec2_extension] paste.filter_factory = keystone.contrib.ec2:Ec2Extension.factory
[filter:ec2_extension_v3] paste.filter_factory = keystone.contrib.ec2:Ec2ExtensionV3.factory
[filter:federation_extension] paste.filter_factory = keystone.contrib.federation.routers:FederationExtension.factory
[filter:oauth1_extension] paste.filter_factory = keystone.contrib.oauth1.routers:OAuth1Extension.factory
[filter:s3_extension] paste.filter_factory = keystone.contrib.s3:S3Extension.factory
[filter:endpoint_filter_extension] paste.filter_factory = keystone.contrib.endpoint_filter.routers:EndpointFilterExtension.factory
[filter:endpoint_policy_extension] paste.filter_factory = keystone.contrib.endpoint_policy.routers:EndpointPolicyExtension.factory
[filter:simple_cert_extension] paste.filter_factory = keystone.contrib.simple_cert:SimpleCertExtension.factory
[filter:revoke_extension] paste.filter_factory = keystone.contrib.revoke.routers:RevokeExtension.factory
[filter:url_normalize] paste.filter_factory = keystone.middleware:NormalizingFilter.factory
[filter:sizelimit] paste.filter_factory = oslo_middleware.sizelimit:RequestBodySizeLimiter.factory
[app:public_service] paste.app_factory = keystone.service:public_app_factory
[app:service_v3] paste.app_factory = keystone.service:v3_app_factory
[app:admin_service] paste.app_factory = keystone.service:admin_app_factory
[pipeline:public_api] # The last item in this pipeline must be public_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension user_crud_extension public_service
[pipeline:admin_api] # The last item in this pipeline must be admin_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension s3_extension crud_extension admin_service
[pipeline:api_v3] # The last item in this pipeline must be service_v3 or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension_v3 s3_extension simple_cert_extension revoke_extension oauth1_extension endpoint_filter_extension endpoint_policy_extension service_v3
[app:public_version_service] paste.app_factory = keystone.service:public_version_app_factory
[app:admin_version_service] paste.app_factory = keystone.service:admin_version_app_factory
[pipeline:public_version_api] pipeline = sizelimit url_normalize public_version_service
[pipeline:admin_version_api] pipeline = sizelimit url_normalize admin_version_service
[composite:main] use = egg:Paste#urlmap /v2.0 = public_api /v3 = api_v3 / = public_version_api
[composite:admin] use = egg:Paste#urlmap /v2.0 = admin_api /v3 = api_v3 / = admin_version_api |
參考書籍《OpenStack設計與實現》知道，paste配置文件分為多個section，每一個section以type：name的格式命名。
(1) type = composite
這個類型的section會把URL請求分發到相應的Application，use表明詳細的分發方式。比方”egg:Paste#urlmap”表示使用Paste包中的urlmap模塊，這個section里的其它形式如”key = value”的行是使用urlmap進行分發時的參數。
(2) type = app
一個app就是一個詳細的WSGI Application。
(3) type = filter-app
接收一個請求后。會首先調用filter-app中的use所指定的app進行過濾，假設這個請求沒有被過濾，就會轉發到next所指定的app進行下一步的處理。
(4) type = filter
與filter-app類型的差別僅僅是沒有next。
(5) type = pipeline
pipeline由一系列filter組成。這個filter鏈條的末尾是一個app。pipeline類型主要是對filter-app進行了簡化。否則，假設多個filter。就須要多個filter-app，然后使用next進行連接。OpenStack的paste的deploy的配置文件主要採用的pipeline的方式。
參看這篇文章http://blog.csdn.net/ztejiagn/article/details/8722765，我們知道pipeline的使用方式。
比如：
| [pipeline:test1] pipeline = filter1 filter2 filter3 app
[filter:filter1] paste.filter_factory = xxx
[filter:filter2] paste.filter_factory = yyy
[filter:filter3] paste.filter_factory = zzz |
如果在ini文件里, 某條pipeline的順序是filter1, filter2, filter3，app, 那么。終於執行的app_real是這樣組織的： app_real =filter1(filter2(filter3(app)))
在app真正被調用的過程中，filter1.__call__(environ,start_response)被首先調用，若某種檢查未通過。filter1做出反應；否則交給filter2.__call__(environ,start_response)進一步處理。若某種檢查未通過，filter2做出反應，中斷鏈條。否則交給filter3.__call__(environ,start_response)處理，若filter3的某種檢查都通過了，最后交給app.__call__(environ,start_response)進行處理。
以下我們詳細分析keystone怎樣處理keystoneclient的HTTP請求。
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
這里，我們的基本url為http://192.168.118.1:5000，而5000port是為main的WSGI server開啟的監聽port，所以當WSGI server的父進程監聽到5000port有HTTP請求，則它將會把HTTP請求轉發給main的WSGI server進行處理，而main的WSGI server則依據keystone-dist-paste.ini配置文件內容對其HTTP請求作對應處理。
1. composite的處理
| [composite:main] use = egg:Paste#urlmap /v2.0 = public_api /v3 = api_v3 / = public_version_api |
由於url為http://192.168.118.1:5000/v2.0/tokens。由於基本url的后面接的信息為/v2.0，所以將到public_api的section作對應操作。
2. pipeline的處理
| [pipeline:public_api] # The last item in this pipeline must be public_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension user_crud_extension public_service |
2.1 sizelimite filter處理
| [filter:sizelimit] paste.filter_factory = oslo_middleware.sizelimit:RequestBodySizeLimiter.factory |
#/oslo_middleware/sizelimit.py:RequestBodySizeLimiter
class RequestBodySizeLimiter(base.Middleware):
"""Limit the size of incoming requests."""
@webob.dec.wsgify
def __call__(self, req):
max_size = CONF.oslo_middleware.max_request_body_size
if (req.content_length is not None and
req.content_length > max_size):
msg = _("Request is too large.")
raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
if req.content_length is None and req.is_body_readable:
limiter = LimitingReader(req.body_file, max_size)
req.body_file = limiter
return self.application
#/oslo_middleware/base.py:Middleware
"""Base class(es) for WSGI Middleware."""
import webob.dec
class Middleware(object):
"""Base WSGI middleware wrapper.
These classes require an application to be initialized that will be called
next. By default the middleware will simply call its wrapped app, or you
can override __call__ to customize its behavior.
"""
@classmethod
def factory(cls, global_conf, **local_conf):
"""Factory method for paste.deploy."""
return cls
def __init__(self, application):
self.application = application
def process_request(self, req):
"""Called on each request.
If this returns None, the next application down the stack will be
executed. If it returns a response then that response will be returned
and execution will stop here.
"""
return None
def process_response(self, response):
"""Do whatever you'd like to the response."""
return response
@webob.dec.wsgify
def __call__(self, req):
response = self.process_request(req)
if response:
return response
response = req.get_response(self.application)
return self.process_response(response)
這里。在paste的deploy的配置文件里。sizelimite filter的paste.filter_factory的value值為factory函數名，在運行paste的deploy.loadapp調用時。則運行了該factory函數，而該函數返回自身類，從而創建了一個自身類。所以當HTTP請求到來時，以函數的形式調用對象時，將會運行該對象的__call__方法。
因此對於本例中的HTTP請求到來時，首先用sizelimite filter去過來HTTP請求，即運行RequestBodySizeLimiter類的__call__方法。
分析該方法可知，sizelimite filter主要推斷request的內容是否超過配置文件keystone.conf設置的最大的request大小。
keystone.conf配置文件默認的max_request_body_size為114688。
此時經過sizelimiterfilter過濾后的request的environ成員變量(字典)的信息為：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': <Response at 0x3d13fd0 200 OK>}, 'REQUEST_METHOD': 'POST', 'PATH_INFO': '/tokens', 'SERVER_PROTOCOL': 'HTTP/1.0', 'REMOTE_ADDR': '192.168.118.1', 'CONTENT_LENGTH': '102', 'HTTP_USER_AGENT': 'python-keystoneclient', 'eventlet.posthooks': [], 'RAW_PATH_INFO': '/v2.0/tokens', 'REMOTE_PORT': '42172', 'eventlet.input': <eventlet.wsgi.Input object at 0x3d13e10>, 'wsgi.url_scheme': 'http', 'webob._body_file': (<_io.BufferedReader>, <eventlet.wsgi.Input object at 0x3d13e10>), 'SERVER_PORT': '5000', 'wsgi.input': <_io.BytesIO object at 0x3d122f0>, 'HTTP_HOST': '192.168.118.1:5000', 'wsgi.multithread': True, 'HTTP_ACCEPT': 'application/json', 'wsgi.version': (1, 0), 'SERVER_NAME': '192.168.118.1', 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once': False, 'wsgi.errors': <open file '<stderr>', mode 'w' at 0x7fda61fb41e0>, 'wsgi.multiprocess': False, 'webob.is_body_seekable': True, 'CONTENT_TYPE': 'application/json'} |
2.2 url_normalize filter處理
| [filter:url_normalize] paste.filter_factory = keystone.middleware:NormalizingFilter.factory |
#/keystone/common/wsgi.py:Middleware
class Middleware(Application):
"""Base WSGI middleware.
These classes require an application to be
initialized that will be called next. By default the middleware will
simply call its wrapped app, or you can override __call__ to customize its
behavior.
"""
@classmethod
def factory(cls, global_config, **local_config):
"""Used for paste app factories in paste.deploy config files.
Any local configuration (that is, values under the [filter:APPNAME]
section of the paste config) will be passed into the `__init__` method
as kwargs.
A hypothetical configuration would look like:
[filter:analytics]
redis_host = 127.0.0.1
paste.filter_factory = keystone.analytics:Analytics.factory
which would result in a call to the `Analytics` class as
import keystone.analytics
keystone.analytics.Analytics(app, redis_host='127.0.0.1')
You could of course re-implement the `factory` method in subclasses,
but using the kwarg passing it shouldn't be necessary.
"""
def _factory(app):
conf = global_config.copy()
conf.update(local_config)
return cls(app, **local_config)
return _factory
@webob.dec.wsgify()
def __call__(self, request):
try:
response = self.process_request(request)
if response:
return response
response = request.get_response(self.application)
return self.process_response(request, response)
except exception.Error as e:
LOG.warning(six.text_type(e))
return render_exception(e, request=request,
user_locale=best_match_language(request))
except TypeError as e:
LOG.exception(six.text_type(e))
return render_exception(exception.ValidationError(e),
request=request,
user_locale=best_match_language(request))
except Exception as e:
LOG.exception(six.text_type(e))
return render_exception(exception.UnexpectedError(exception=e),
request=request,
user_locale=best_match_language(request))
#/keystone/middleware/core.py:NormalizingFilter
class NormalizingFilter(wsgi.Middleware):
"""Middleware filter to handle URL normalization."""
def process_request(self, request):
"""Normalizes URLs."""
# Removes a trailing slash from the given path, if any.
if (len(request.environ['PATH_INFO']) > 1 and
request.environ['PATH_INFO'][-1] == '/'):
request.environ['PATH_INFO'] = request.environ['PATH_INFO'][:-1]
# Rewrites path to root if no path is given.
elif not request.environ['PATH_INFO']:
request.environ['PATH_INFO'] = '/'
url_normalize filter調用父類Middleware的__call__方法，在__call__方法中，調用自身的process_request函數，該函數主要處理sizelimiter filter傳遞下來的request的environ成員變量(字典)中的PATH_INFO的value值，因為本文樣例sizelimiterfilter過濾后request的environ字典中的PATH_INFO的值為：’/tokens’，所以這里並未做不論什么處理。
此時打印出來的request的environ成員變量的值與sizelimiter filter過濾后的同樣，即為：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': <Response at 0x5341310 200 OK>}, 'REQUEST_METHOD': 'POST', 'PATH_INFO': '/tokens', 'SERVER_PROTOCOL': 'HTTP/1.0', 'REMOTE_ADDR': '192.168.118.1', 'CONTENT_LENGTH': '102', 'HTTP_USER_AGENT': 'python-keystoneclient', 'eventlet.posthooks': [], 'RAW_PATH_INFO': '/v2.0/tokens', 'REMOTE_PORT': '58290', 'eventlet.input': <eventlet.wsgi.Input object at 0x5339dd0>, 'wsgi.url_scheme': 'http', 'webob._body_file': (<_io.BufferedReader>, <eventlet.wsgi.Input object at 0x5339dd0>), 'SERVER_PORT': '5000', 'wsgi.input': <_io.BytesIO object at 0x5338350>, 'HTTP_HOST': '192.168.118.1:5000', 'wsgi.multithread': True, 'HTTP_ACCEPT': 'application/json', 'wsgi.version': (1, 0), 'SERVER_NAME': '192.168.118.1', 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once': False, 'wsgi.errors': <open file '<stderr>', mode 'w' at 0x7f82a39e61e0>, 'wsgi.multiprocess': False, 'webob.is_body_seekable': True, 'CONTENT_TYPE': 'application/json'} |
2.3 request_id filter處理
| [filter:request_id] paste.filter_factory = oslo_middleware:RequestId.factory |
#/oslo_middleware/request_id.py:RequestId
ENV_REQUEST_ID = 'openstack.request_id'
class RequestId(base.Middleware):
"""Middleware that ensures request ID.
It ensures to assign request ID for each API request and set it to
request environment. The request ID is also added to API response.
"""
@webob.dec.wsgify
def __call__(self, req):
req_id = context.generate_request_id()
req.environ[ENV_REQUEST_ID] = req_id
response = req.get_response(self.application)
if HTTP_RESP_HEADER_REQUEST_ID not in response.headers:
response.headers.add(HTTP_RESP_HEADER_REQUEST_ID, req_id)
return response
該filter相當於在request的environ成員變量(字典)中的添加key為openstack.request_id，value值req_id(由context的generate_request_id函數產生的由req-開頭的16進制的一串數字)的字典。
終於生成的request的environ成員變量的信息為：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': <Response at 0x52c9f50 200 OK>}, 'REQUEST_METHOD': 'POST', 'PATH_INFO': '/tokens', 'SERVER_PROTOCOL': 'HTTP/1.0', 'REMOTE_ADDR': '192.168.118.1', 'CONTENT_LENGTH': '102', 'HTTP_USER_AGENT': 'python-keystoneclient', 'eventlet.posthooks': [], 'RAW_PATH_INFO': '/v2.0/tokens', 'REMOTE_PORT': '42320', 'eventlet.input': <eventlet.wsgi.Input object at 0x52c9d50>, 'wsgi.url_scheme': 'http', 'webob._body_file': (<_io.BufferedReader>, <eventlet.wsgi.Input object at 0x52c9d50>), 'SERVER_PORT': '5000', 'wsgi.input': <_io.BytesIO object at 0x52ca290>, 'HTTP_HOST': '192.168.118.1:5000', 'wsgi.multithread': True, 'HTTP_ACCEPT': 'application/json', 'openstack.request_id': 'req-c3af38d4-19c7-4454-9938-a2304f8baa3f', 'wsgi.version': (1, 0), 'SERVER_NAME': '192.168.118.1', 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once': False, 'wsgi.errors': <open file '<stderr>', mode 'w' at 0x7f590e2bc1e0>, 'wsgi.multiprocess': False, 'webob.is_body_seekable': True, 'CONTENT_TYPE': 'application/json'} |
2.4 build_auth_context token_auth admin_token_authjson_body filter處理
| [filter:build_auth_context] paste.filter_factory = keystone.middleware:AuthContextMiddleware.factory
[filter:token_auth] paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory
[filter:admin_token_auth] paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory
[filter:json_body] paste.filter_factory = keystone.middleware:JsonBodyMiddleware.factory |
這4個filter事實上也跟前面的filter一樣。沒做不論什么實質性的工作，其代碼流程跟url_normalize filter處理流程類似。我們就不在這里解說了。
2.5 ec2_extension filter處理
| [filter:ec2_extension] paste.filter_factory = keystone.contrib.ec2:Ec2Extension.factory |
#/keystone/contrib/ec2/routers.py:Ec2Extension
class Ec2Extension(wsgi.ExtensionRouter):
def add_routes(self, mapper):
ec2_controller = controllers.Ec2Controller()
# validation
mapper.connect(
'/ec2tokens',
controller=ec2_controller,
action='authenticate',
conditions=dict(method=['POST']))
# crud
mapper.connect(
'/users/{user_id}/credentials/OS-EC2',
controller=ec2_controller,
action='create_credential',
conditions=dict(method=['POST']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2',
controller=ec2_controller,
action='get_credentials',
conditions=dict(method=['GET']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2/{credential_id}',
controller=ec2_controller,
action='get_credential',
conditions=dict(method=['GET']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2/{credential_id}',
controller=ec2_controller,
action='delete_credential',
conditions=dict(method=['DELETE']))
#/keystone/common/wsgi.py:ExtensionRouter
class ExtensionRouter(Router):
"""A router that allows extensions to supplement or overwrite routes.
Expects to be subclassed.
"""
def __init__(self, application, mapper=None):
if mapper is None:
mapper = routes.Mapper()
self.application = application
self.add_routes(mapper)
mapper.connect('{path_info:.*}', controller=self.application)
super(ExtensionRouter, self).__init__(mapper)
def add_routes(self, mapper):
pass
@classmethod
def factory(cls, global_config, **local_config):
"""Used for paste app factories in paste.deploy config files.
Any local configuration (that is, values under the [filter:APPNAME]
section of the paste config) will be passed into the `__init__` method
as kwargs.
A hypothetical configuration would look like:
[filter:analytics]
redis_host = 127.0.0.1
paste.filter_factory = keystone.analytics:Analytics.factory
which would result in a call to the `Analytics` class as
import keystone.analytics
keystone.analytics.Analytics(app, redis_host='127.0.0.1')
You could of course re-implement the `factory` method in subclasses,
but using the kwarg passing it shouldn't be necessary.
"""
def _factory(app):
conf = global_config.copy()
conf.update(local_config)
return cls(app, **local_config)
return _factory
這里引入了python的routes模塊，詳細用法參考官網鏈接http://routes.readthedocs.org/en/latest/index.html。當HTTP請求到來時。route將依據請求的url信息轉換成對應的資源，並路由到合適的函數上。對於ec2_extension filter的重點就在add_routes函數。這里提供了url到對應資源的轉換且能route到合適的函數上，如我們提供HTTP請求的url為http://192.168.118.1:5000/v2.0/ec2tokens。且請求類型為POST，通過-d再加上其它一下驗證參數，這樣將會去運行 Ec2Controller類中的authenticate函數，運行完畢后將得到的ec2類型的token值返回給請求者。
當然，我們舉例的url為http://192.168.118.1:5000/v2.0/tokens。則不會路由到ec2_extension filter上的資源。
2.6 user_crud_extension filter處理
| [filter:user_crud_extension] paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory |
#/keystone/contrib/user_crud/core.py:CrudExtension
class CrudExtension(wsgi.ExtensionRouter):
"""Provides a subset of CRUD operations for internal data types."""
def add_routes(self, mapper):
user_controller = UserController()
mapper.connect('/OS-KSCRUD/users/{user_id}',
controller=user_controller,
action='set_user_password',
conditions=dict(method=['PATCH']))
user_crud_extension filter處理與ec2_extension filter處理類似，且它們有同樣的父類，這里user_crud_extension filter僅僅是添加了一個route。
事實上大部分route都在public_service filter上。
2.7 public_service app處理
| [app:public_service] paste.app_factory = keystone.service:public_app_factory |
#/keystone/service.py
@fail_gracefully
def public_app_factory(global_conf, **local_conf):
controllers.register_version('v2.0')
return wsgi.ComposingRouter(routes.Mapper(),
[assignment.routers.Public(),
token.routers.Router(),
routers.VersionV2('public'),
routers.Extension(False)])
#/keystone/common/wsgi.py:ComposingRouter
class ComposingRouter(Router):
def __init__(self, mapper=None, routers=None):
if mapper is None:
mapper = routes.Mapper()
if routers is None:
routers = []
for router in routers:
router.add_routes(mapper)
super(ComposingRouter, self).__init__(mapper)
從上面能夠public_serviceapp添加了4個route，即assignment.routers.Public()，token.routers.Router(),routers.VersionV2('public')和routers.Extension(False)]，那么我們看看這4個route都添加了哪些資源呢？
#/keystone/assignment/routers.py:Public
class Public(wsgi.ComposableRouter):
def add_routes(self, mapper):
tenant_controller = controllers.TenantAssignment()
mapper.connect('/tenants',
controller=tenant_controller,
action='get_projects_for_token',
conditions=dict(method=['GET']))
#/keystone/token/routers.py:Router
class Router(wsgi.ComposableRouter):
def add_routes(self, mapper):
token_controller = controllers.Auth()
mapper.connect('/tokens',
controller=token_controller,
action='authenticate',
conditions=dict(method=['POST']))
mapper.connect('/tokens/revoked',
controller=token_controller,
action='revocation_list',
conditions=dict(method=['GET']))
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='validate_token',
conditions=dict(method=['GET']))
# NOTE(morganfainberg): For policy enforcement reasons, the
# ``validate_token_head`` method is still used for HEAD requests.
# The controller method makes the same call as the validate_token
# call and lets wsgi.render_response remove the body data.
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='validate_token_head',
conditions=dict(method=['HEAD']))
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='delete_token',
conditions=dict(method=['DELETE']))
mapper.connect('/tokens/{token_id}/endpoints',
controller=token_controller,
action='endpoints',
conditions=dict(method=['GET']))
# Certificates used to verify auth tokens
mapper.connect('/certificates/ca',
controller=token_controller,
action='ca_cert',
conditions=dict(method=['GET']))
mapper.connect('/certificates/signing',
controller=token_controller,
action='signing_cert',
conditions=dict(method=['GET']))
#/keystone/routers.py:VersionV2
class VersionV2(wsgi.ComposableRouter):
def __init__(self, description):
self.description = description
def add_routes(self, mapper):
version_controller = controllers.Version(self.description)
mapper.connect('/',
controller=version_controller,
action='get_version_v2')
#/keystone/routers.py:Extension
class Extension(wsgi.ComposableRouter):
def __init__(self, is_admin=True):
if is_admin:
self.controller = controllers.AdminExtensions()
else:
self.controller = controllers.PublicExtensions()
def add_routes(self, mapper):
extensions_controller = self.controller
mapper.connect('/extensions',
controller=extensions_controller,
action='get_extensions_info',
conditions=dict(method=['GET']))
mapper.connect('/extensions/{extension_alias}',
controller=extensions_controller,
action='get_extension_info',
conditions=dict(method=['GET']))
這些就是public_serviceapp添加的全部route資源，你也能夠依照上面的mapper.connect的方式加入你自己的所須要的資源，然后利用HTTP請求route到你的函數上去運行的對應的操作。
由於我們本文中舉的樣例為：
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
所以運行的是/keystone/token/routers.py:Router的第一個route資源。即
mapper.connect('/tokens',
controller=token_controller,
action='authenticate',
conditions=dict(method=['POST']))
由於token_controller為/keystone/token/controllers.py:Auth對象。所以此時運行該類中的authenticate函數。例如以下
#/keystone/token/controllers.py:Auth
@controller.v2_deprecated
def authenticate(self, context, auth=None):
"""Authenticate credentials and return a token.
Accept auth as a dict that looks like::
{
"auth":{
"passwordCredentials":{
"username":"test_user",
"password":"mypass"
},
"tenantName":"customer-x"
}
}
In this case, tenant is optional, if not provided the token will be
considered "unscoped" and can later be used to get a scoped token.
Alternatively, this call accepts auth with only a token and tenant
that will return a token that is scoped to that tenant.
"""
if auth is None:
raise exception.ValidationError(attribute='auth',
target='request body')
if "token" in auth:
# Try to authenticate using a token
auth_info = self._authenticate_token(
context, auth)
else:
# Try external authentication
try:
auth_info = self._authenticate_external(
context, auth)
except ExternalAuthNotApplicable:
# Try local authentication
auth_info = self._authenticate_local(
context, auth)
user_ref, tenant_ref, metadata_ref, expiry, bind, audit_id = auth_info
# Validate that the auth info is valid and nothing is disabled
try:
self.identity_api.assert_user_enabled(
user_id=user_ref['id'], user=user_ref)
if tenant_ref:
self.resource_api.assert_project_enabled(
project_id=tenant_ref['id'], project=tenant_ref)
except AssertionError as e:
six.reraise(exception.Unauthorized, exception.Unauthorized(e),
sys.exc_info()[2])
# NOTE(morganfainberg): Make sure the data is in correct form since it
# might be consumed external to Keystone and this is a v2.0 controller.
# The user_ref is encoded into the auth_token_data which is returned as
# part of the token data. The token provider doesn't care about the
# format.
user_ref = self.v3_to_v2_user(user_ref)
if tenant_ref:
tenant_ref = self.v3_to_v2_project(tenant_ref)
auth_token_data = self._get_auth_token_data(user_ref,
tenant_ref,
metadata_ref,
expiry,
audit_id)
if tenant_ref:
catalog_ref = self.catalog_api.get_catalog(
user_ref['id'], tenant_ref['id'])
else:
catalog_ref = {}
auth_token_data['id'] = 'placeholder'
if bind:
auth_token_data['bind'] = bind
roles_ref = []
for role_id in metadata_ref.get('roles', []):
role_ref = self.role_api.get_role(role_id)
roles_ref.append(dict(name=role_ref['name']))
(token_id, token_data) = self.token_provider_api.issue_v2_token(
auth_token_data, roles_ref=roles_ref, catalog_ref=catalog_ref)
# NOTE(wanghong): We consume a trust use only when we are using trusts
# and have successfully issued a token.
if CONF.trust.enabled and 'trust_id' in auth:
self.trust_api.consume_use(auth['trust_id'])
return token_data
2.7.1 驗證HTTP請求
這里我們舉例的HTTP請求傳遞給authenticate函數的auth變量的值為：
| {u'tenantName': u'admin', u'passwordCredentials': {u'username': u'admin', u'password': u'admin'}} |
所以，HTTP請求將在_authenticate_local函數中進行驗證。這里是通過傳遞username(或用戶id)和password的方式進行驗證。
另外，對於keystone的token的HTTP請求還有第二種形式。即auth變量的值為例如以下形式：
| "auth": {"tenantName": "admin","token": {"id": "9b390c37959f44b5ad5e8b2aa403267a "}} |
此時完整的HTTP請求為：
| curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin","token": {"id": "9b390c37959f44b5ad5e8b2aa403267a"}}}' |
即不通過username(或用戶id)和password的進行驗證，而是通過未過期的token進行驗證，這樣的形式的HTTP請求將在_authenticate_token函數中進行驗證。
這里我們主要分析怎樣通過username和password對HTTP請求進行驗證。
#/keystone/token/controllers.py:Auth
def _authenticate_local(self, context, auth):
"""Try to authenticate against the identity backend.
Returns auth_token_data, (user_ref, tenant_ref, metadata_ref)
"""
if 'passwordCredentials' not in auth:
raise exception.ValidationError(
attribute='passwordCredentials', target='auth')
if "password" not in auth['passwordCredentials']:
raise exception.ValidationError(
attribute='password', target='passwordCredentials')
password = auth['passwordCredentials']['password']
if password and len(password) > CONF.identity.max_password_length:
raise exception.ValidationSizeError(
attribute='password', size=CONF.identity.max_password_length)
if (not auth['passwordCredentials'].get("userId") and
not auth['passwordCredentials'].get("username")):
raise exception.ValidationError(
attribute='username or userId',
target='passwordCredentials')
user_id = auth['passwordCredentials'].get('userId')
if user_id and len(user_id) > CONF.max_param_size:
raise exception.ValidationSizeError(attribute='userId',
size=CONF.max_param_size)
username = auth['passwordCredentials'].get('username', '')
if username:
if len(username) > CONF.max_param_size:
raise exception.ValidationSizeError(attribute='username',
size=CONF.max_param_size)
try:
user_ref = self.identity_api.get_user_by_name(
username, CONF.identity.default_domain_id)
user_id = user_ref['id']
except exception.UserNotFound as e:
raise exception.Unauthorized(e)
try:
user_ref = self.identity_api.authenticate(
context,
user_id=user_id,
password=password)
except AssertionError as e:
raise exception.Unauthorized(e.args[0])
metadata_ref = {}
tenant_id = self._get_project_id_from_auth(auth)
tenant_ref, metadata_ref['roles'] = self._get_project_roles_and_ref(
user_id, tenant_id)
expiry = provider.default_expire_time()
bind = None
audit_id = None
return (user_ref, tenant_ref, metadata_ref, expiry, bind, audit_id)
_authenticate_local函數前面都是對username(或用戶id)和password的長度進行推斷。
然后運行self.identity_api.get_user_by_name方法，該方法主要目的是獲取用戶的相關信息，然后從中獲取用戶的ID(這是在HTTP請求採用username和password進行驗證的時候調用的接口)。
在獲取用戶的ID后。使用用戶ID和password對HTTP請求進行驗證。
運行例如以下代碼：
user_ref = self.identity_api.authenticate(
context,
user_id=user_id,
password=password)
當中self.identity_api是我們在/keystone/backends.py的load_backends函數中定義的。
即self.identity_api為/keystone/identity/core.py中的Manager對象。
#/keystone/identity/core.py:Manager
@notifications.emit_event('authenticate')
@domains_configured
@exception_translated('assertion')
def authenticate(self, context, user_id, password):
domain_id, driver, entity_id = (
self._get_domain_driver_and_entity_id(user_id))
ref = driver.authenticate(entity_id, password)
return self._set_domain_id_and_mapping(
ref, domain_id, driver, mapping.EntityType.USER)
#/keystone/identity/core.py:Manager
def _get_domain_driver_and_entity_id(self, public_id):
"""Look up details using the public ID.
:param public_id: the ID provided in the call
:returns: domain_id, which can be None to indicate that the driver
in question supports multiple domains
driver selected based on this domain
entity_id which will is understood by the driver.
Use the mapping table to look up the domain, driver and local entity
that is represented by the provided public ID. Handle the situations
were we do not use the mapping (e.g. single driver that understands
UUIDs etc.)
"""
conf = CONF.identity
# First, since we don't know anything about the entity yet, we must
# assume it needs mapping, so long as we are using domain specific
# drivers.
if conf.domain_specific_drivers_enabled:
local_id_ref = self.id_mapping_api.get_id_mapping(public_id)
if local_id_ref:
return (
local_id_ref['domain_id'],
self._select_identity_driver(local_id_ref['domain_id']),
local_id_ref['local_id'])
# So either we are using multiple drivers but the public ID is invalid
# (and hence was not found in the mapping table), or the public ID is
# being handled by the default driver. Either way, the only place left
# to look is in that standard driver. However, we don't yet know if
# this driver also needs mapping (e.g. LDAP in non backward
# compatibility mode).
driver = self.driver
if driver.generates_uuids():
if driver.is_domain_aware:
# No mapping required, and the driver can handle the domain
# information itself. The classic case of this is the
# current SQL driver.
return (None, driver, public_id)
else:
# Although we don't have any drivers of this type, i.e. that
# understand UUIDs but not domains, conceptually you could.
return (conf.default_domain_id, driver, public_id)
# So the only place left to find the ID is in the default driver which
# we now know doesn't generate UUIDs
if not CONF.identity_mapping.backward_compatible_ids:
# We are not running in backward compatibility mode, so we
# must use a mapping.
local_id_ref = self.id_mapping_api.get_id_mapping(public_id)
if local_id_ref:
return (
local_id_ref['domain_id'],
driver,
local_id_ref['local_id'])
else:
raise exception.PublicIDNotFound(id=public_id)
# If we reach here, this means that the default driver
# requires no mapping - but also doesn't understand domains
# (e.g. the classic single LDAP driver situation). Hence we pass
# back the public_ID unmodified and use the default domain (to
# keep backwards compatibility with existing installations).
#
# It is still possible that the public ID is just invalid in
# which case we leave this to the caller to check.
return (conf.default_domain_id, driver, public_id)
這里我們重點關注從_get_domain_driver_and_entity_id函數返回的driver是什么？從上面的代碼能夠看出driver= self.driver，那么我們看看Manger類怎樣定義driver的？
#/keystone/identity/core.py:Manager
@dependency.provider('identity_api')
@dependency.requires('assignment_api', 'credential_api', 'id_mapping_api',
'resource_api', 'revoke_api')
class Manager(manager.Manager):
"""Default pivot point for the Identity backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
This class also handles the support of domain specific backends, by using
the DomainConfigs class. The setup call for DomainConfigs is called
from with the @domains_configured wrapper in a lazy loading fashion
to get around the fact that we can't satisfy the assignment api it needs
from within our __init__() function since the assignment driver is not
itself yet initialized.
Each of the identity calls are pre-processed here to choose, based on
domain, which of the drivers should be called. The non-domain-specific
driver is still in place, and is used if there is no specific driver for
the domain in question (or we are not using multiple domain drivers).
Starting with Juno, in order to be able to obtain the domain from
just an ID being presented as part of an API call, a public ID to domain
and local ID mapping is maintained. This mapping also allows for the local
ID of drivers that do not provide simple UUIDs (such as LDAP) to be
referenced via a public facing ID. The mapping itself is automatically
generated as entities are accessed via the driver.
This mapping is only used when:
- the entity is being handled by anything other than the default driver, or
- the entity is being handled by the default LDAP driver and backward
compatible IDs are not required.
This means that in the standard case of a single SQL backend or the default
settings of a single LDAP backend (since backward compatible IDs is set to
True by default), no mapping is used. An alternative approach would be to
always use the mapping table, but in the cases where we don't need it to
make the public and local IDs the same. It is felt that not using the
mapping by default is a more prudent way to introduce this functionality.
"""
_USER = 'user'
_GROUP = 'group'
def __init__(self):
super(Manager, self).__init__(CONF.identity.driver)
self.domain_configs = DomainConfigs()
self.event_callbacks = {
notifications.ACTIONS.deleted: {
'domain': [self._domain_deleted],
},
}
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
從上述的代碼能夠看出，/keystone/identity/core.py中的Manager類中driver是在其父類/keystone/common/manager.py:Manager中進行定義的。且該driver是在keystone中的配置文件里進行讀取。默認配置例如以下：
cfg.StrOpt('driver',
default=('keystone.identity.backends'
'.sql.Identity'),
help='Identity backend driver.'),
即/keystone/identity/core.py中的Manager類中driver的值是/keystone/identity/backends/sql.py中的Identity對象。
此外。我們/keystone/common/manager.py:Manager中也能夠發現，該類相當於是一個代理類(從__getattr__函數能夠看出)：當子類調用既不在子類自身中也不在父類中的函數時，此時將會去調用定義的driver中的函數。
我們發現driver的值是/keystone/identity/backends/sql.py中的Identity對象。所以/keystone/identity/core.py中Manager類中的authenticate函數此代碼(ref= driver.authenticate(entity_id, password))將調用/keystone/identity/backends/sql.py中的Identity對象中的authenticate函數。
#/keystone/identity/backends/sql.py:Identity
# Identity interface
def authenticate(self, user_id, password):
session = sql.get_session()
user_ref = None
try:
user_ref = self._get_user(session, user_id)
except exception.UserNotFound:
raise AssertionError(_('Invalid user / password'))
if not self._check_password(password, user_ref):
raise AssertionError(_('Invalid user / password'))
return identity.filter_user(user_ref.to_dict())
#/keystone/identity/backends/sql.py:Identity
def _get_user(self, session, user_id):
user_ref = session.query(User).get(user_id)
if not user_ref:
raise exception.UserNotFound(user_id=user_id)
return user_ref
#/keystone/identity/backends/sql.py:User
class User(sql.ModelBase, sql.DictBase):
__tablename__ = 'user'
attributes = ['id', 'name', 'domain_id', 'password', 'enabled',
'default_project_id']
id = sql.Column(sql.String(64), primary_key=True)
name = sql.Column(sql.String(255), nullable=False)
domain_id = sql.Column(sql.String(64), nullable=False)
password = sql.Column(sql.String(128))
enabled = sql.Column(sql.Boolean)
extra = sql.Column(sql.JsonBlob())
default_project_id = sql.Column(sql.String(64))
# Unique constraint across two columns to create the separation
# rather than just only 'name' being unique
__table_args__ = (sql.UniqueConstraint('domain_id', 'name'), {})
def to_dict(self, include_extra_dict=False):
d = super(User, self).to_dict(include_extra_dict=include_extra_dict)
if 'default_project_id' in d and d['default_project_id'] is None:
del d['default_project_id']
return d
/keystone/identity/backends/sql.py中的Identity對象中的authenticate函數首先調用_get_user函數。該函數依據user id從keystone數據庫中的user表查詢user的相關信息，當中查詢的user信息中包含加密后的user的password信息。如我環境中的keystone數據庫中的user表數據。
| MariaDB [keystone]> select * from user; +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ | id | name | extra | password | enabled | domain_id | default_project_id | +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ | 1677a7f58d7d4c98a56f8c2be5e16068 | neutron | {"email": "neutron@localhost"} | $6$rounds=40000$S7Zg3JPb/iFgH0aw$/fnQFBc.pYH8wsOSxBWhDHevIx2zTphl0eSxb6akKNoXUgbUDc9pb5TXNhL3xgGms6RVZm1BFxDK48R2LHPgv0 | 1 | default | c9960b417968496387e6b96a244cc2be | | 7a0fd47f67e94d6e99fe6f0298067102 | cinder | {"email": "cinder@localhost"} | $6$rounds=40000$EZp3iGcBcL9CpXbt$wN/qjTr2m.0PBkXz/HYGcf65QXm0qPqlVTE9n84U1lXqmS1JsdJnxRno8eptuPcVUV91mx39b449FYMfMnuXT/ | 1 | default | c9960b417968496387e6b96a244cc2be | | bde55802097c475592741efc5096d90a | nova | {"email": "nova@localhost"} | $6$rounds=40000$ezqZ86aPM0o1J8DH$hS2cBDqAzminTsL9x7H/kIV6PfLQKude6Z.JQ2ugnW.aBF0n4sevPZVibK30DGrP9/0sY7J5HzBwlmKsCYM8./ | 1 | default | c9960b417968496387e6b96a244cc2be | | d317cfbab4544707a5f323fc1317cf65 | glance | {"email": "glance@localhost"} | $6$rounds=40000$fyCPnCYRF.t6.2Se$PRcMbqExrngnZPAhKiX7BME.MEh7k19VcN.L5GbjeTprXcbODM.ntSd2ezsKuhIx/LMU9ezgPZz0ki92XZTTZ0 | 1 | default | c9960b417968496387e6b96a244cc2be | | e4ea11ef20e0439596b4df43cb7687ac | ceilometer | {"email": "ceilometer@localhost"} | $6$rounds=40000$k9qUckdICo7hY6ZD$4ccb/TkTklY9iRATB9V/DJZoQVigvFGjhJH1nBC19VAW.FIXbCf27yyfIJfpHtCIYR8gsfsQA6cDHNiY3dCHw. | 1 | default | c9960b417968496387e6b96a244cc2be | | f59f17d8e9774eef8730b23ecdc86a4b | admin | {"email": "root@localhost"} | $6$rounds=40000$rPma7p.3qcixVvDW$/rCywiBa74Jg2J7z/m5zH5a.poLSd4haDKgpSJEfa4IK3j.HPbOnClFYO4Nd4ADiTOxX/OUuc2N4Fr8dy/BHx1 | 1 | default | 09e04766c06d477098201683497d3878 | +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ 6 rows in set (0.00 sec) |
#/keystone/identity/backends/sql.py:Identity
def _check_password(self, password, user_ref):
"""Check the specified password against the data store.
Note that we'll pass in the entire user_ref in case the subclass
needs things like user_ref.get('name')
For further justification, please see the follow up suggestion at
https://blueprints.launchpad.net/keystone/+spec/sql-identiy-pam
"""
return utils.check_password(password, user_ref.password)
#/keystone/common/utils.py
def check_password(password, hashed):
"""Check that a plaintext password matches hashed.
hashpw returns the salt value concatenated with the actual hash value.
It extracts the actual salt if this value is then passed as the salt.
"""
if password is None or hashed is None:
return False
password_utf8 = verify_length_and_trunc_password(password).encode('utf-8')
return passlib.hash.sha512_crypt.verify(password_utf8, hashed)
上述便是驗證HTTP請求中的password是否正確的代碼，當然詳細怎么驗證的。我也沒看懂，感興趣的讀者能夠自己再深入研究一下。
驗證完畢成功后。再將對應的user信息設置成合適的格式返回給調用代碼。
再次回到/keystone/token/controllers.py:Auth中的_authenticate_local函數。在將user的信息驗證成功后，以下的代碼將獲取tenant信息，然后提供一個token的過期時間和其它的一些信息返回給authenticate函數(詳細獲取tenant和token過期時間的代碼在這里就不分析了。詳細能夠參考分析驗證userpassword的代碼進行分析)。
當中，我的環境中_authenticate_local函數返回的信息為：
| auth_info: ({'domain_id': u'default', 'name': u'admin', 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'enabled': True, u'email': u'root@localhost', 'default_project_id': u'09e04766c06d477098201683497d3878' },
'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'parent_id': None, 'domain_id': u'default', 'name': u'admin' },
{'roles': [u'397eaf49b01549dab8be01804bec7972']},
datetime.datetime(2016, 3, 24, 13, 8, 30, 626964), None, None ) |
auth_info中的第1個字典為user的信息。第2個字典為tenant的信息。第4個字典為將為獲取的token設置的過期時間。
2.7.2 auth_info格式處理
user_ref = self.v3_to_v2_user(user_ref)
if tenant_ref:
tenant_ref = self.v3_to_v2_project(tenant_ref)
auth_token_data = self._get_auth_token_data(user_ref,
tenant_ref,
metadata_ref,
expiry,
audit_id)
這部分代碼較簡單，讀者能夠自行分析，處理后返回auth_token_data。即例如以下形式：
| auth_token_data: {'expires': datetime.datetime(2016, 3, 24, 13, 8, 30, 626964), 'parent_audit_id': None, 'user': {'username': u'admin', 'name': u'admin', 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'enabled': True, u'email': u'root@localhost', 'tenantId': u'09e04766c06d477098201683497d3878' }, 'tenant': {'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'name': u'admin' }, 'metadata': {'roles': [u'397eaf49b01549dab8be01804bec7972']} } |
2.7.3 獲取catalog信息
if tenant_ref:
catalog_ref = self.catalog_api.get_catalog(
user_ref['id'], tenant_ref['id'])
else:
catalog_ref = {}
由於tenant_ref有值，所以運行if中的語句。當中self.catallog_api為/keystone/catalog/core.py中的Manager對象。
#keystone/catalog/core.py:Manager
def get_catalog(self, user_id, tenant_id):
try:
return self.driver.get_catalog(user_id, tenant_id)
except exception.NotFound:
raise exception.NotFound('Catalog not found for user and tenant')
對於self.driver是什么對象，我們能夠依照2.7.1中的分析方式進行分析(它們採用類似的構造)。
例如以下
#keystone/catalog/core.py:Manager
@dependency.provider('catalog_api')
class Manager(manager.Manager):
"""Default pivot point for the Catalog backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
"""
_ENDPOINT = 'endpoint'
_SERVICE = 'service'
_REGION = 'region'
def __init__(self):
super(Manager, self).__init__(CONF.catalog.driver)
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
所以self.driver為keystone配置文件里catalog section設置的driver對象(查看/keystone/backends.py的load_backends函數中的定義)。
例如以下
cfg.StrOpt('driver',
default='keystone.catalog.backends.sql.Catalog',
help='Catalog backend driver.'),
該driver為/keystone/catalog/backends/sql中的Catalog對象。
#/keystone/catalog/backends/sql.py:Catalog
def get_catalog(self, user_id, tenant_id):
"""Retrieve and format the V2 service catalog.
:param user_id: The id of the user who has been authenticated for
creating service catalog.
:param tenant_id: The id of the project. 'tenant_id' will be None
in the case this being called to create a catalog to go in a
domain scoped token. In this case, any endpoint that requires
a tenant_id as part of their URL will be skipped (as would a whole
service if, as a consequence, it has no valid endpoints).
:returns: A nested dict representing the service catalog or an
empty dict.
"""
substitutions = dict(
itertools.chain(six.iteritems(CONF),
six.iteritems(CONF.eventlet_server)))
substitutions.update({'user_id': user_id})
silent_keyerror_failures = []
if tenant_id:
substitutions.update({'tenant_id': tenant_id})
else:
silent_keyerror_failures = ['tenant_id']
session = sql.get_session()
endpoints = (session.query(Endpoint).
options(sql.joinedload(Endpoint.service)).
filter(Endpoint.enabled == true()).all())
catalog = {}
for endpoint in endpoints:
if not endpoint.service['enabled']:
continue
try:
formatted_url = core.format_url(
endpoint['url'], substitutions,
silent_keyerror_failures=silent_keyerror_failures)
if formatted_url is not None:
url = formatted_url
else:
continue
except exception.MalformedEndpoint:
continue # this failure is already logged in format_url()
region = endpoint['region_id']
service_type = endpoint.service['type']
default_service = {
'id': endpoint['id'],
'name': endpoint.service.extra.get('name', ''),
'publicURL': ''
}
catalog.setdefault(region, {})
catalog[region].setdefault(service_type, default_service)
interface_url = '%sURL' % endpoint['interface']
catalog[region][service_type][interface_url] = url
return catalog
從上面的代碼能夠看出，get_catalog函數主要將keystone數據庫的endpoint表的信息進行組裝，然后將其返回給authenticate函數。當中endpoint表的定義例如以下
#/keystone/catalog/backends/sql.py:Endpoint
class Endpoint(sql.ModelBase, sql.DictBase):
__tablename__ = 'endpoint'
attributes = ['id', 'interface', 'region_id', 'service_id', 'url',
'legacy_endpoint_id', 'enabled']
id = sql.Column(sql.String(64), primary_key=True)
legacy_endpoint_id = sql.Column(sql.String(64))
interface = sql.Column(sql.String(8), nullable=False)
region_id = sql.Column(sql.String(255),
sql.ForeignKey('region.id',
ondelete='RESTRICT'),
nullable=True,
default=None)
service_id = sql.Column(sql.String(64),
sql.ForeignKey('service.id'),
nullable=False)
url = sql.Column(sql.Text(), nullable=False)
enabled = sql.Column(sql.Boolean, nullable=False, default=True,
server_default=sqlalchemy.sql.expression.true())
extra = sql.Column(sql.JsonBlob())
數據庫中endpoint表的信息例如以下。
| MariaDB [keystone]> select * from endpoint; +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+ | id | legacy_endpoint_id | interface | service_id | url | extra | enabled | region_id | +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+ | 03966fa6606945b985d8ff3ba1912f00 | 5e483b6c64fd44158215b3e285a3614a | admin | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 0ab5626e37714ead9c1be3a7fb322d66 | 5e483b6c64fd44158215b3e285a3614a | internal | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 11b8c840c71c4258a644bf93ddee8d0f | da4a4ab4857840b0aa9d7c5acc4c82e3 | admin | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:35357/v2.0 | {} | 1 | RegionOne | | 16b0ed99d09044229a7c67992f514aa3 | 5e483b6c64fd44158215b3e285a3614a | public | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 1a22050d1c404a1fa257b7de8453f7b5 | 6870936c855741319bdbd915a9284f19 | admin | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 1b6b0ae5906345d3a6ec0352c97e724d | 58abac45c14c44879d39e5c9c65dab61 | internal | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Cloud | {} | 1 | RegionOne | | 3572f3870f21445bb82c5e622d87f5cd | 58abac45c14c44879d39e5c9c65dab61 | public | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Cloud | {} | 1 | RegionOne | | 3d9c418b79f641a291a885fe20ec6a84 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | admin | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | 4095ec6b1d9c4e3ba87994a90a0d1ed5 | 278489f69a414670b5e66e03db79e580 | internal | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | | 40a799e5ac0f4731bd2206299f303917 | da4a4ab4857840b0aa9d7c5acc4c82e3 | public | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:5000/v2.0 | {} | 1 | RegionOne | | 589ae871719c46308e7179271c66e43a | 58abac45c14c44879d39e5c9c65dab61 | admin | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Admin | {} | 1 | RegionOne | | 63e42f266d9b432abdbf6f62da6bffaa | 6870936c855741319bdbd915a9284f19 | internal | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 65560ec5eed64581a72f4ac3d1fa519d | 4cbcc6153af54f40b4bf5e7fe3e60533 | admin | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | 65c0727bdee74006bec9fbd3ca6f90a6 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | internal | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | 6ecc803364da42b7a250bf9fe2e71cc4 | 45bf231a5a9a41eea1b879132b152dda | internal | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | 6fb8352c59f141dfb69725f0a7a81280 | 6870936c855741319bdbd915a9284f19 | public | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 7cd86837358f4c19bb551fe3c36fadf9 | 2e8e4627677f4cec8e4aaee2c61e064b | public | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | 7f980243a75740378e1ac27c0b9cc8fd | 45bf231a5a9a41eea1b879132b152dda | public | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | 8fb40f64a7024518bce652cbf1bba9cd | 4cbcc6153af54f40b4bf5e7fe3e60533 | public | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | a4b0a96daca1412891c299783527f648 | 45bf231a5a9a41eea1b879132b152dda | admin | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | be2bdd1ed6fc4232b46be46548a292e9 | da4a4ab4857840b0aa9d7c5acc4c82e3 | internal | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:5000/v2.0 | {} | 1 | RegionOne | | c38c9bf499b64d8d8eee0a0e226e84df | 2e8e4627677f4cec8e4aaee2c61e064b | admin | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | c77d7cfc578f4299ae3e6e0c78e23cc9 | 2e8e4627677f4cec8e4aaee2c61e064b | internal | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | c7c6ca1359ab424e9fdac3c407ce45c9 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | public | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | db973fd1d8cd46699325be0912abda75 | 278489f69a414670b5e66e03db79e580 | admin | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | | de47b4c075ab4402bf57111bc84dcb50 | 4cbcc6153af54f40b4bf5e7fe3e60533 | internal | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | e29c0a5af91b4e32834b5b93629bffe5 | 278489f69a414670b5e66e03db79e580 | public | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+
MariaDB [keystone]> select count(*) from endpoint; +----------+ | count(*) | +----------+ | 27 | +----------+ 1 row in set (0.05 sec) |
即在數據庫中endpoint表有27條信息。
每一個服務會占3條endpoint(URL)信息，即adminURL, publicURL和internalURL。所以總共同擁有9個服務的endpoint在OpenStack上。這9個service在keystone數據庫中的service表中保存。
| MariaDB [keystone]> select * from service; +----------------------------------+-----------+---------+---------------------------------------------------------------------+ | id | type | enabled | extra | +----------------------------------+-----------+---------+---------------------------------------------------------------------+ | 2a2b254ccf6742a2af7f2736c5246b13 | metering | 1 | {"name": "ceilometer", "description": "Openstack Metering Service"} | | 591dfc7f5b434674bf3566646ffb37ec | image | 1 | {"name": "glance", "description": "OpenStack Image Service"} | | 7a4407c4eb7a4f2c97813b0d0a9729ab | ec2 | 1 | {"name": "nova_ec2", "description": "EC2 Service"} | | 9e0e0000b90645cc8a5fcbcc7ad3b02c | identity | 1 | {"name": "keystone", "description": "OpenStack Identity Service"} | | bcc3368ce9384e63ac5f8f4c894f232a | volumev2 | 1 | {"name": "cinderv2", "description": "Cinder Service v2"} | | c333b7e90e8742c6adf726f199338f04 | computev3 | 1 | {"name": "novav3", "description": "Openstack Compute Service v3"} | | c75d6dc0a8574c979ccdc1ea44ad41b2 | network | 1 | {"name": "neutron", "description": "Neutron Networking Service"} | | c90e95ef71164659beb0fa413b4a292f | volume | 1 | {"name": "cinder", "description": "Cinder Service"} | | f3f4b449f7244b28b65940af94fd8a47 | compute | 1 | {"name": "nova", "description": "Openstack Compute Service"} | +----------------------------------+-----------+---------+---------------------------------------------------------------------+
MariaDB [keystone]> select count(*) from service; +----------+ | count(*) | +----------+ | 9 | +----------+ 1 row in set (0.07 sec) |
終於經過get_catalog函數對endpoint信息進行格式化后的返回信息例如以下：
| {u'RegionOne': { u'compute': {u'adminURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'id': u'03966fa6606945b985d8ff3ba1912f00', u'internalURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'name': u'nova' }, u'network': {u'adminURL': u'http://192.168.118.1:9696/', 'publicURL': u'http://192.168.118.1:9696/', 'id': u'6ecc803364da42b7a250bf9fe2e71cc4', u'internalURL': u'http://192.168.118.1:9696/', 'name': u'neutron' }, u'volumev2': {u'adminURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'id': u'1a22050d1c404a1fa257b7de8453f7b5', u'internalURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'name': u'cinderv2' }, u'computev3': {u'adminURL': u'http://192.168.118.1:8774/v3', 'publicURL': u'http://192.168.118.1:8774/v3', 'id': u'4095ec6b1d9c4e3ba87994a90a0d1ed5', u'internalURL': u'http://192.168.118.1:8774/v3', 'name': u'novav3' }, u'image': {u'adminURL': u'http://192.168.118.1:9292', 'publicURL': u'http://192.168.118.1:9292', 'id': u'7cd86837358f4c19bb551fe3c36fadf9', u'internalURL': u'http://192.168.118.1:9292', 'name': u'glance' }, u'metering': {u'adminURL': u'http://192.168.118.1:8777', 'publicURL': u'http://192.168.118.1:8777', 'id': u'3d9c418b79f641a291a885fe20ec6a84', u'internalURL': u'http://192.168.118.1:8777', 'name': u'ceilometer' }, u'volume': {u'adminURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'id': u'65560ec5eed64581a72f4ac3d1fa519d', u'internalURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'name': u'cinder' }, u'ec2': {u'adminURL': u'http://192.168.118.1:8773/services/Admin', 'publicURL': u'http://192.168.118.1:8773/services/Cloud', 'id': u'1b6b0ae5906345d3a6ec0352c97e724d', u'internalURL': u'http://192.168.118.1:8773/services/Cloud', 'name': u'nova_ec2' }, u'identity': {u'adminURL': u'http://192.168.118.1:35357/v2.0', 'publicURL': u'http://192.168.118.1:5000/v2.0', 'id': u'11b8c840c71c4258a644bf93ddee8d0f', u'internalURL': u'http://192.168.118.1:5000/v2.0', 'name': u'keystone' } } } |
2.7.4 生成token id
生成token id運行的代碼例如以下
(token_id, token_data) = self.token_provider_api.issue_v2_token(
auth_token_data, roles_ref=roles_ref, catalog_ref=catalog_ref)
當中self.toekn_provider_api為/keystone/token/provider中的Manager對象 (查看/keystone/backends.py的load_backends函數中的定義)。
#/keystone/token/provider.py:Manager
def issue_v2_token(self, token_ref, roles_ref=None, catalog_ref=None):
token_id, token_data = self.driver.issue_v2_token(
token_ref, roles_ref, catalog_ref)
if self._needs_persistence:
data = dict(key=token_id,
id=token_id,
expires=token_data['access']['token']['expires'],
user=token_ref['user'],
tenant=token_ref['tenant'],
metadata=token_ref['metadata'],
token_data=token_data,
bind=token_ref.get('bind'),
trust_id=token_ref['metadata'].get('trust_id'),
token_version=self.V2)
self._create_token(token_id, data)
return token_id, token_data
self.driver的定義查看例如以下代碼。
#/keystone/token/provider.py:Manager
@dependency.provider('token_provider_api')
@dependency.requires('assignment_api', 'revoke_api')
class Manager(manager.Manager):
"""Default pivot point for the token provider backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
"""
V2 = V2
V3 = V3
VERSIONS = VERSIONS
INVALIDATE_PROJECT_TOKEN_PERSISTENCE = 'invalidate_project_tokens'
INVALIDATE_USER_TOKEN_PERSISTENCE = 'invalidate_user_tokens'
_persistence_manager = None
def __init__(self):
super(Manager, self).__init__(CONF.token.provider)
self._register_callback_listeners()
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
cfg.StrOpt('provider',
default='keystone.token.providers.uuid.Provider',
help='Controls the token construction, validation, and '
'revocation operations. Core providers are '
'"keystone.token.providers.[fernet|pkiz|pki|uuid].'
'Provider".'),
所以driver為/keystone/token/providers/uuid.py中的Provider對象。
#/keystone/token/providers/uuid.py:Provider
class Provider(common.BaseProvider):
def __init__(self, *args, **kwargs):
super(Provider, self).__init__(*args, **kwargs)
def _get_token_id(self, token_data):
return uuid.uuid4().hex
def needs_persistence(self):
"""Should the token be written to a backend."""
return True
#/keystone/token/providers/common.py:BaseProvider
def issue_v2_token(self, token_ref, roles_ref=None,
catalog_ref=None):
metadata_ref = token_ref['metadata']
trust_ref = None
if CONF.trust.enabled and metadata_ref and 'trust_id' in metadata_ref:
trust_ref = self.trust_api.get_trust(metadata_ref['trust_id'])
token_data = self.v2_token_data_helper.format_token(
token_ref, roles_ref, catalog_ref, trust_ref)
token_id = self._get_token_id(token_data)
token_data['access']['token']['id'] = token_id
return token_id, token_data
終於token id通過_get_token_id函數進行生成，且為16進制的id(uuid.uuid4().hex)。且將生成的token id塞進返回給authenticate函數的token_data中。
終於的token_data信息為：
| {'access': {'token': {'issued_at': '2016-03-24T12:08:30.807133', 'expires': '2016-03-24T13:08:30Z', 'id': 'fcb6cd4b7c3e400baca630adf8d878cf', 'tenant': {'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'name': u'admin' }, 'audit_ids': ['yjGRG7A-S6asfuvYLESR-g'] }, 'serviceCatalog': [ {'endpoints': [{u'adminURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'id': u'03966fa6606945b985d8ff3ba1912f00', 'publicURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'compute', 'name': u'nova' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:9696/', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:9696/', 'id': u'6ecc803364da42b7a250bf9fe2e71cc4', 'publicURL': u'http://192.168.118.1:9696/' }], 'endpoints_links': [], 'type': u'network', 'name': u'neutron' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'id': u'1a22050d1c404a1fa257b7de8453f7b5', 'publicURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'volumev2', 'name': u'cinderv2' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8774/v3', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8774/v3', 'id': u'4095ec6b1d9c4e3ba87994a90a0d1ed5', 'publicURL': u'http://192.168.118.1:8774/v3' }], 'endpoints_links': [], 'type': u'computev3', 'name': u'novav3' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:9292', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:9292', 'id': u'7cd86837358f4c19bb551fe3c36fadf9', 'publicURL': u'http://192.168.118.1:9292' }], 'endpoints_links': [], 'type': u'image', 'name': u'glance' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8777', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8777', 'id': u'3d9c418b79f641a291a885fe20ec6a84', 'publicURL': u'http://192.168.118.1:8777' }], 'endpoints_links': [], 'type': u'metering', 'name': u'ceilometer' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'id': u'65560ec5eed64581a72f4ac3d1fa519d', 'publicURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'volume', 'name': u'cinder' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8773/services/Admin', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8773/services/Cloud', 'id': u'1b6b0ae5906345d3a6ec0352c97e724d', 'publicURL': u'http://192.168.118.1:8773/services/Cloud' }], 'endpoints_links': [], 'type': u'ec2', 'name': u'nova_ec2' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:35357/v2.0', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:5000/v2.0', 'id': u'11b8c840c71c4258a644bf93ddee8d0f', 'publicURL': u'http://192.168.118.1:5000/v2.0' }], 'endpoints_links': [], 'type': u'identity', 'name': u'keystone' } ], 'user': {'username': u'admin', 'roles_links': [], 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'roles': [{'name': u'admin'}], 'name': u'admin' }, 'metadata': {'is_admin': 0, 'roles': [u'397eaf49b01549dab8be01804bec7972']} } } |
所以當HTTP請求獲取token信息時，keystone將會把token id和 各種服務的endpoint信息也一起返回給HTTP請求者。將上述的token_data(keystone返回的)與《nova list命令的代碼流程分析》文章中keystoneclient收到的auth_ref信息做比較可知，兩者信息差點兒全然同樣。
眼下。authenticate函數的代碼流程分析完畢。
即驗證用戶信息，獲取endpoint列表。生成token id。
在本文中，我們舉例的HTTP請求為獲取token信息的請求：
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
終於調用到/keystone/token/controllers.py中Auth類中的authenticate函數。當然keystone還提供了非常多其它的api接口，對於怎樣發送正確的HTTP請求，請詳細參看OpenStack官網(http://developer.openstack.org/api-ref-identity-admin-v2.html)
3. 總結
本文分析了keystone的WSGI代碼流程。
1. 分析了OpenStack創建WSGI server的代碼流程。
WSGI server的父進程監聽5000和35357port獲取keystone HTTP請求，然后將HTTP請求下發到合適的WSGIserver上。
2. 分析了keystone的HTTP請求的處理流程
本文通過舉例獲取token信息的HTTP請求來分析keystone的處理流程。即當HTTP請求到來時。一個所謂的”路由”模塊會將請求的URL轉換成對應的資源。並路由到合適的操作函數上。