Ansible源碼分析【第三篇】： Ansible源碼分析之ansible命令

Ansible命令講解

ansible 0.8 版本

研究ansible 0.8 版本源碼

wget https://pypi.python.org/packages/source/s/setuptools/setuptools-20.9.0.tar.gz#md5=e5f4d80fcb68bcac179b327bf1791dec
tar -zxf setuptools-20.9.0.tar.gz
cd setuptools-20.9.0
python setup.py install
cd ..
tar -zxf setuptools-0.6c11.tar.gz
cd setuptools-0.6c11
python setup.py install
cd ..

easy_install jinja2

easy_install pyyaml

unzip ansible-0.8.zip
cd ansible-0.8
python setup.py install

ansible模塊安裝位置：/usr/lib/python2.6/site-packages/

#!/usr/bin/env python

# (c) 2012, Michael DeHaan <[e-mail]>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.

########################################################

import sys
import getpass

from ansible.runner import Runner      　　　　# API模塊
import ansible.constants as C　　　　　　        # 定義命令行的配置參數（包含一些可以配置的參數，一些不可配置的參數）
from ansible import utils　　　　　　　　　　  # 工具集合，即常用的函數模塊
from ansible import errors　　　　　　　　　   # 常見的ansible錯誤類型定義
from ansible import callbacks                             
from ansible import inventory　　　　　　　　 # 主機列表清單

########################################################

class Cli(object):
    ''' code behind bin/ansible '''

    # ----------------------------------------------

    def __init__(self):
        self.stats = callbacks.AggregateStats()　　　　　　# 統計playbook運行時各種統計信息
        self.callbacks = callbacks.CliRunnerCallbacks()　　 # 回調函數供/usr/bin/ansible使用

    # ----------------------------------------------

    def parse(self):　　　　　　　　　　　　　　　　　　# bin/ansible命令行幫助信息
        ''' create an options parser for bin/ansible '''

        parser = utils.base_parser(constants=C, runas_opts=True, subset_opts=True, async_opts=True,
            output_opts=True, connect_opts=True, usage='%prog <host-pattern> [options]')
        parser.add_option('-a', '--args', dest='module_args',
            help="module arguments", default=C.DEFAULT_MODULE_ARGS)
        parser.add_option('-m', '--module-name', dest='module_name',
            help="module name to execute (default=%s)" % C.DEFAULT_MODULE_NAME,
            default=C.DEFAULT_MODULE_NAME)
        options, args = parser.parse_args()
　　 print(options, args)   # 添加這行代碼，可以查看ansible從終端獲取的到有配置項，例如：
[root@zabbix ~]# ansible all -m shell -a 'date'
(<Values at 0x24e25f0: {'subset': None, 'ask_pass': False, 'remote_user': 'root', 'seconds': 0, 'poll_interval': 15, 'sudo': False, 'tree': None, 'private_key_file': None, 'one_line': None, 'connection': 'paramiko', 'inventory': '/etc/ansible/hosts', 'timeout': 10, 'module_name': 'shell', 'module_path': '/usr/share/ansible/', 'sudo_user': None, 'forks': 5, 'ask_sudo_pass': False, 'module_args': 'date'}>, ['all'])
        self.callbacks.options = options

        if len(args) == 0 or len(args) > 1:   # 判斷命令行輸入的參數個數，不滿足條件則輸出幫助信息，並退出
            parser.print_help()
            sys.exit(1)
        return (options, args)

    # ----------------------------------------------

    def run(self, options, args):
        ''' use Runner lib to do SSH things '''

        pattern = args[0]
        print(pattern)  　　　　# 獲取要執行命令的主機或者主機組，本例為test主機
        inventory_manager = inventory.Inventory(options.inventory)   # 執行inventory目錄下的__init__.py文件中的Inventory類獲取主機列表文件，根據上一步獲取的主機列表文件而定，一般默認為/etc/ansible/hosts
        其中options為上一步傳遞過來的字典格式的參數內容，options.inventory 即獲取參數字典inventory 這個key的值

        hosts = inventory_manager.list_hosts(pattern)
        # 即調用inventory目錄下的__init__.py文件中的Inventory類的list_hosts方法，list_hosts方法調用get_hosts方法，get_hosts方法又調用了_get_host方法
　　 print(hosts)  　　# 獲取主機列表 ['test1', 'test2']，即/etc/ansible/hosts里面主機第一類的內容

　　 # 主機不存在的情況
        if len(hosts) == 0:
            print >>sys.stderr, "No hosts matched"
            sys.exit(1)
        
        # 獲取密碼內容
        sshpass = None
        sudopass = None
        if options.ask_pass:
            sshpass = getpass.getpass(prompt="SSH password: ")
        if options.ask_sudo_pass:
            sudopass = getpass.getpass(prompt="sudo password: ")
            options.sudo = True
        if options.sudo_user:
            options.sudo = True
        options.sudo_user = options.sudo_user or C.DEFAULT_SUDO_USER
        if options.tree:
            utils.prepare_writeable_dir(options.tree)
        # 將參數封裝給runner模塊
        runner = Runner(
            module_name=options.module_name, module_path=options.module_path,
            module_args=options.module_args,
            remote_user=options.remote_user, remote_pass=sshpass,
            inventory=inventory_manager, timeout=options.timeout,
            private_key_file=options.private_key_file,
            forks=options.forks,
            pattern=pattern,
            callbacks=self.callbacks, sudo=options.sudo,
            sudo_pass=sudopass,sudo_user=options.sudo_user,
            transport=options.connection, subset=options.subset
        )

        if options.seconds:
            print "background launch...\n\n"
            results, poller = runner.run_async(options.seconds)
            results = self.poll_while_needed(poller, options)
        else:
            results = runner.run()   # 調用lib/ansible/runner/__init__.py中Runner類的run方法
　　　 print(results)
            # 字典格式的返回結果
{
    'dark': {
        'test1': {
            
        },
        'test2': {
            
        }
    },
    'contacted': {
        '172.16.1.151': {
            u'changed': True,
            u'end': u'2016-08-0406: 07: 56.768411',
            u'stdout': u'2016\u5e7408\u670804\u65e5\u661f\u671f\u56db06: 07: 56CST',
            u'cmd': u'date',
            u'rc': 0,
            u'start': u'2016-08-0406: 07: 56.765674',
            u'stderr': u'',
            u'delta': u'0: 00: 00.002737',
            'invocation': {
                'module_name': u'shell',
                'module_args': u'date'
            }
        }
    }
}


        return (runner, results)

    # ----------------------------------------------

    def poll_while_needed(self, poller, options):
        ''' summarize results from Runner '''

        # BACKGROUND POLL LOGIC when -B and -P are specified
        if options.seconds and options.poll_interval > 0:
            poller.wait(options.seconds, options.poll_interval)

        return poller.results


########################################################

if __name__ == '__main__':
    cli = Cli()　　　　　　　　　　　　　　　　# 實例化自定義命令行界面類
    (options, args) = cli.parse()　　　　　　　 # 通過命令行獲取執行參數（可配置的及不可配置的參數） 
    try:
        (runner, results) = cli.run(options, args)　　# 去執行命令，並返回結果，字典格式
        for result in results['contacted'].values():     # 將字典格式的返回結果做處理，最后將結果顯示給用戶
            if 'failed' in result or result.get('rc', 0) != 0:
                sys.exit(2)
        if results['dark']:
            sys.exit(2)
    except errors.AnsibleError, e:
        # Generic handler for ansible specific errors
        print "ERROR: %s" % str(e)
        sys.exit(1)

 

ansible1.9.0.0版本

#!/usr/bin/env python

# (c) 2012, Michael DeHaan <[e-mail]>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.

########################################################

__requires__ = ['ansible']
try:
    import pkg_resources
except Exception:
    # Use pkg_resources to find the correct versions of libraries and set
    # sys.path appropriately when there are multiversion installs.  But we
    # have code that better expresses the errors in the places where the code
    # is actually used (the deps are optional for many code paths) so we don't
    # want to fail here.
    pass

# 導入常用的Python模塊
import os
import sys

from ansible.runner import Runner    # Python的API模塊
import ansible.constants as C
from ansible import utils
from ansible import errors
from ansible import callbacks
from ansible import inventory
########################################################

class Cli(object):
    ''' code behind bin/ansible '''

    # ----------------------------------------------

    def __init__(self):
        self.stats = callbacks.AggregateStats()
        self.callbacks = callbacks.CliRunnerCallbacks()
        if C.DEFAULT_LOAD_CALLBACK_PLUGINS:
            callbacks.load_callback_plugins()

    # ----------------------------------------------

    def parse(self):
        ''' create an options parser for bin/ansible '''

        parser = utils.base_parser(
            constants=C,
            runas_opts=True,
            subset_opts=True,
            async_opts=True,
            output_opts=True,
            connect_opts=True,
            check_opts=True,
            diff_opts=False,
            usage='%prog <host-pattern> [options]'
        )

        parser.add_option('-a', '--args', dest='module_args',
            help="module arguments", default=C.DEFAULT_MODULE_ARGS)
        parser.add_option('-m', '--module-name', dest='module_name',
            help="module name to execute (default=%s)" % C.DEFAULT_MODULE_NAME,
            default=C.DEFAULT_MODULE_NAME)

        options, args = parser.parse_args()
        self.callbacks.options = options

        if len(args) == 0 or len(args) > 1:
            parser.print_help()
            sys.exit(1)

        # privlege escalation command line arguments need to be mutually exclusive
        utils.check_mutually_exclusive_privilege(options, parser)

        if (options.ask_vault_pass and options.vault_password_file):
            parser.error("--ask-vault-pass and --vault-password-file are mutually exclusive")

        return (options, args)

    # ----------------------------------------------

    def run(self, options, args):
        ''' use Runner lib to do SSH things '''

        pattern = args[0]

        sshpass = becomepass = vault_pass = become_method = None

        # Never ask for an SSH password when we run with local connection
        if options.connection == "local":
            options.ask_pass = False
        else:
            options.ask_pass = options.ask_pass or C.DEFAULT_ASK_PASS

        options.ask_vault_pass = options.ask_vault_pass or C.DEFAULT_ASK_VAULT_PASS

        # become
        utils.normalize_become_options(options)
        prompt_method = utils.choose_pass_prompt(options)
        (sshpass, becomepass, vault_pass) = utils.ask_passwords(ask_pass=options.ask_pass, become_ask_pass=options.become_ask_pass, ask_vault_pass=options.ask_vault_pass, become_method=prompt_method)

        # read vault_pass from a file
        if not options.ask_vault_pass and options.vault_password_file:
            vault_pass = utils.read_vault_file(options.vault_password_file)

        extra_vars = utils.parse_extra_vars(options.extra_vars, vault_pass)

        inventory_manager = inventory.Inventory(options.inventory, vault_password=vault_pass)
        if options.subset:
            inventory_manager.subset(options.subset)
        hosts = inventory_manager.list_hosts(pattern)

        if len(hosts) == 0:
            callbacks.display("No hosts matched", stderr=True)
            sys.exit(0)

        if options.listhosts:
            for host in hosts:
                callbacks.display('    %s' % host)
            sys.exit(0)

        if options.module_name in ['command','shell'] and not options.module_args:
            callbacks.display("No argument passed to %s module" % options.module_name, color='red', stderr=True)
            sys.exit(1)

        if options.tree:
            utils.prepare_writeable_dir(options.tree)

        runner = Runner(
            module_name=options.module_name,
            module_path=options.module_path,
            module_args=options.module_args,
            remote_user=options.remote_user,
            remote_pass=sshpass,
            inventory=inventory_manager,
            timeout=options.timeout,
            private_key_file=options.private_key_file,
            forks=options.forks,
            pattern=pattern,
            callbacks=self.callbacks,
            transport=options.connection,
            subset=options.subset,
            check=options.check,
            diff=options.check,
            vault_pass=vault_pass,
            become=options.become,
            become_method=options.become_method,
            become_pass=becomepass,
            become_user=options.become_user,
            extra_vars=extra_vars,
        )

        if options.seconds:
            callbacks.display("background launch...\n\n", color='cyan')
            results, poller = runner.run_async(options.seconds)
            results = self.poll_while_needed(poller, options)
        else:
            results = runner.run()

        return (runner, results)

    # ----------------------------------------------

    def poll_while_needed(self, poller, options):
        ''' summarize results from Runner '''

        # BACKGROUND POLL LOGIC when -B and -P are specified
        if options.seconds and options.poll_interval > 0:
            poller.wait(options.seconds, options.poll_interval)

        return poller.results


########################################################

if __name__ == '__main__':
    callbacks.display("", log_only=True)
    callbacks.display(" ".join(sys.argv), log_only=True)
    callbacks.display("", log_only=True)

    cli = Cli()
    (options, args) = cli.parse()
    try:
        (runner, results) = cli.run(options, args)
        for result in results['contacted'].values():
            if 'failed' in result or result.get('rc', 0) != 0:
                sys.exit(2)
        if results['dark']:
            sys.exit(3)
    except errors.AnsibleError, e:
        # Generic handler for ansible specific errors
        callbacks.display("ERROR: %s" % str(e), stderr=True, color='red')
        sys.exit(1)

 

ansible-2.1.0.0-0.1.rc1 版本

#!/usr/bin/env python

# (c) 2012, Michael DeHaan <[e-mail]>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.

########################################################
from __future__ import (absolute_import, division, print_function)    
# __future__模塊：提供某些將要引入的特性的模塊  詳解附錄__future__模塊講解

__metaclass__ = type

__requires__ = ['ansible']
try:
    import pkg_resources
except Exception:
    # Use pkg_resources to find the correct versions of libraries and set
    # sys.path appropriately when there are multiversion installs.  But we
    # have code that better expresses the errors in the places where the code
    # is actually used (the deps are optional for many code paths) so we don't
    # want to fail here.
    pass

# 引入常用的Python模塊
import os
import shutil
import sys
import traceback

# for debug
from multiprocessing import Lock
debug_lock = Lock()

import ansible.constants as C
from ansible.errors import AnsibleError, AnsibleOptionsError, AnsibleParserError
from ansible.utils.display import Display
from ansible.utils.unicode import to_unicode


########################################
### OUTPUT OF LAST RESORT ###
# 定義一個類，用於最后輸出結果排序

class LastResort(object):
    def display(self, msg):
        print(msg, file=sys.stderr)

    def error(self, msg, wrap_text=None):
        print(msg, file=sys.stderr)


########################################

if __name__ == '__main__':

    display = LastResort()    # 實例化上邊定義的LastResort類
    cli = None
    me = os.path.basename(sys.argv[0])    # ansible命令本身

    try:
        display = Display()
        display.debug("starting run")

        sub = None
        try:
            if me.find('-') != -1:
                target = me.split('-')
                if len(target) > 1:
                    sub = target[1]
                    myclass = "%sCLI" % sub.capitalize()
                    mycli = getattr(__import__("ansible.cli.%s" % sub, fromlist=[myclass]), myclass)
            elif me == 'ansible':
                from ansible.cli.adhoc import AdHocCLI as mycli
            else:
                raise AnsibleError("Unknown Ansible alias: %s" % me)
        except ImportError as e:
            if e.message.endswith(' %s' % sub):
                raise AnsibleError("Ansible sub-program not implemented: %s" % me)
            else:
                raise

        cli = mycli(sys.argv)
        cli.parse()
        exit_code = cli.run()

    except AnsibleOptionsError as e:
        cli.parser.print_help()
        display.error(to_unicode(e), wrap_text=False)
        exit_code = 5
    except AnsibleParserError as e:
        display.error(to_unicode(e), wrap_text=False)
        exit_code = 4
# TQM takes care of these, but leaving comment to reserve the exit codes
#    except AnsibleHostUnreachable as e:
#        display.error(str(e))
#        exit_code = 3
#    except AnsibleHostFailed as e:
#        display.error(str(e))
#        exit_code = 2
    except AnsibleError as e:
        display.error(to_unicode(e), wrap_text=False)
        exit_code = 1
    except KeyboardInterrupt:
        display.error("User interrupted execution")
        exit_code = 99
    except Exception as e:
        have_cli_options = cli is not None and cli.options is not None
        display.error("Unexpected Exception: %s" % to_unicode(e), wrap_text=False)
        if not have_cli_options or have_cli_options and cli.options.verbosity > 2:
            display.display(u"the full traceback was:\n\n%s" % to_unicode(traceback.format_exc()))
        else:
            display.display("to see the full traceback, use -vvv")
        exit_code = 250
    finally:
        # Remove ansible tempdir
        shutil.rmtree(C.DEFAULT_LOCAL_TMP, True)

    sys.exit(exit_code)

 附錄

1、__future__模塊

__future__模塊：提供某些將要引入的特性，基本上都是Python 3的特性

>>>import __future__

>>> __future__.
__future__.CO_FUTURE_ABSOLUTE_IMPORT 　　　　  __future__.all_feature_names
__future__.CO_FUTURE_DIVISION         　　　　　　　 __future__.division
__future__.CO_FUTURE_PRINT_FUNCTION   　　　　　__future__.generators
__future__.CO_FUTURE_UNICODE_LITERALS 　　　　  __future__.nested_scopes
__future__.CO_FUTURE_WITH_STATEMENT   　　　　  __future__.print_function
__future__.CO_GENERATOR_ALLOWED       　　　　　 __future__.unicode_literals
__future__.CO_NESTED                  　　　　　　　　      __future__.with_statement
__future__.absolute_import

可以導入的新特性

>>> for i in __future__.all_feature_names:
...     print(i)
...
nested_scopes
generators
division 
absolute_import
with_statement
print_function
unicode_literals
barry_as_FLUFL
generator_stop

對於功能如下：

division

新的除法特性，本來的除號`/`對於分子分母是整數的情況會取整，但新特性中在此情況下的除法不會取整，取整的使用`//`。

print_function 

新的print是一個函數，如果導入此特性，之前的print語句就不能用了。

unicode_literals

這個是對字符串使用unicode字符

nested_scopes

　　這個是修改嵌套函數或lambda函數中變量的搜索順序，從`當前函數命名空間->模塊命名空間`的順序更改為了`當前函數命名空間->父函數命名空間->模塊命名空間`,python2.7.5中默認使用

generators

生成器，對應yield的語法，python2.7.5中默認使用

with_statement

是使用with關鍵字，python2.7.5是默認使用

2、methclass

methclass作用

1. 你可以自由的、動態的修改/增加/刪除 類的或者實例中的方法或者屬性
2. 批量的對某些方法使用decorator，而不需要每次都在方法的上面加入@decorator_func
3. 當引入第三方庫的時候，如果該庫某些類需要patch的時候可以用metaclass
4. 可以用於序列化(參見yaml這個庫的實現，我沒怎么仔細看）
5. 提供接口注冊，接口格式檢查等
6. 自動委托(auto delegate)
7. more...

 

3、__requires__

__requires__是一個控制台生成腳本的一部分。Python本身沒有意義,只有setuptools庫本身使用這些信息。

 

4、pkg_resources

5、traceback

追蹤異常

6、multiprocessing

多進程並發

參考資料

__future__模塊講解：http://www.cnblogs.com/ksedz/archive/2013/07/15/3190208.html

methclass講解：http://www.cnblogs.com/huangcong/archive/2011/08/28/2156307.html