芝麻HTTP:Ansible扩展
2018-06-18 01:03:16来源:未知 阅读 ()
Ansible简介
Ansible是由Python开发的一个运维工具,因为工作需要接触到Ansible,经常会集成一些东西到Ansible,所以对Ansible的了解越来越多。
那Ansible到底是什么呢?在我的理解中,原来需要登录到服务器上,然后执行一堆命令才能完成一些操作。而Ansible就是来代替我们去执行那些命令。并且可以通过Ansible控制多台机器,在机器上进行任务的编排和执行,在Ansible中称为playbook。
那Ansible是如何做到的呢?简单点说,就是Ansible将我们要执行的命令生成一个脚本,然后通过sftp将脚本上传到要执行命令的服务器上,然后在通过ssh协议,执行这个脚本并将执行结果返回。
那Ansible具体是怎么做到的呢?下面从模块和插件来看一下Ansible是如何完成一个模块的执行
PS:下面的分析都是在对Ansible有一些具体使用经验之后,通过阅读源代码进一步得出的执行结论,所以希望在看本文时,是建立在对Ansible有一定了解的基础上,最起码对于Ansible的一些概念有了解,例如inventory,module,playbooks等
Ansible模块
模块是Ansible执行的最小单位,可以是由Python编写,也可以是Shell编写,也可以是由其他语言编写。模块中定义了具体的操作步骤以及实际使用过程中所需要的参数
执行的脚本就是根据模块生成一个可执行的脚本。
那Ansible是怎么样将这个脚本上传到服务器上,然后执行获取结果的呢?
Ansible插件
connection插件
连接插件,根据指定的ssh参数连接指定的服务器,并切提供实际执行命令的接口
shell插件
命令插件,根据sh类型,来生成用于connection时要执行的命令
strategy插件
执行策略插件,默认情况下是线性插件,就是一个任务接着一个任务的向下执行,此插件将任务丢到执行器去执行。
action插件
动作插件,实质就是任务模块的所有动作,如果ansible的模块没有特别编写的action插件,默认情况下是normal或者async(这两个根据模块是否async来选择),normal和async中定义的就是模块的执行步骤。例如,本地创建临时文件,上传临时文件,执行脚本,删除脚本等等,如果想在所有的模块中增加一些特殊步骤,可以通过增加action插件的方式来扩展。
Ansible执行模块流程
- ansible命令实质是通过ansible/cli/adhoc.py来运行,同时会收集参数信息
- 设置Play信息,然后通过TaskQueueManager进行run,
- TaskQueueManager需要Inventory(节点仓库),variable_manager(收集变量),options(命令行中指定的参数),stdout_callback(回调函数)
- 在task_queue_manager.py中找到run中
- 初始化时会设置队列
- 会根据options,,variable_manager,passwords等信息设置成一个PlayContext信息(playbooks/playcontext.py)
- 设置插件(plugins)信息callback_loader(回调), strategy_loader(执行策略), module_loader(任务模块)
- 通过strategy_loader(strategy插件)的run(默认的strategy类型是linear,线性执行),去按照顺序执行所有的任务(执行一个模块,可能会执行多个任务)
- 在strategy_loader插件run之后,会判断action类型。如果是meta类型的话会单独执行(不是具体的ansible模块时),而其他模块时,会加载到队列_queue_task
- 在队列中会调用WorkerProcess去处理,在workerproces实际的run之后,会使用TaskExecutor进行执行
- 在TaskExecutor中会设置connection插件,并且根据task的类型(模块。或是include等)获取action插件,就是对应的模块,如果模块有自定义的执行,则会执行自定义的action,如果没有的会使用normal或者async,这个是根据是否是任务的async属性来决定
- 在Action插件中定义着执行的顺序,及具体操作,例如生成临时目录,生成临时脚本,所以要在统一的模式下,集成一些额外的处理时,可以重写Action的方法
- 通过Connection插件来执行Action的各个操作步骤
扩展Ansible实例
执行节点Python环境扩展
实际需求中,我们扩展的一些Ansible模块需要使用三方库,但每个节点中安装这些库有些不易于管理。ansible执行模块的实质就是在节点的python环境下执行生成的脚本,所以我们采取的方案是,指定节点上的Python环境,将局域网内一个python环境作为nfs共享。通过扩展Action插件,增加节点上挂载nfs,待执行结束后再将节点上的nfs卸载。具体实施步骤如下:
扩展代码:
重写ActionBase的execute_module方法
# execute_module from __future__ import (absolute_import, division, print_function) __metaclass__ = type import json import pipes from ansible.compat.six import text_type, iteritems from ansible import constants as C from ansible.errors import AnsibleError from ansible.release import __version__ try: from __main__ import display except ImportError: from ansible.utils.display import Display display = Display() class MagicStackBase(object): def _mount_nfs(self, ansible_nfs_src, ansible_nfs_dest): cmd = ['mount',ansible_nfs_src, ansible_nfs_dest] cmd = [pipes.quote(c) for c in cmd] cmd = ' '.join(cmd) result = self._low_level_execute_command(cmd=cmd, sudoable=True) return result def _umount_nfs(self, ansible_nfs_dest): cmd = ['umount', ansible_nfs_dest] cmd = [pipes.quote(c) for c in cmd] cmd = ' '.join(cmd) result = self._low_level_execute_command(cmd=cmd, sudoable=True) return result def _execute_module(self, module_name=None, module_args=None, tmp=None, task_vars=None, persist_files=False, delete_remote_tmp=True): ''' Transfer and run a module along with its arguments. ''' # display.v(task_vars) if task_vars is None: task_vars = dict() # if a module name was not specified for this execution, use # the action from the task if module_name is None: module_name = self._task.action if module_args is None: module_args = self._task.args # set check mode in the module arguments, if required if self._play_context.check_mode: if not self._supports_check_mode: raise AnsibleError("check mode is not supported for this operation") module_args['_ansible_check_mode'] = True else: module_args['_ansible_check_mode'] = False # Get the connection user for permission checks remote_user = task_vars.get('ansible_ssh_user') or self._play_context.remote_user # set no log in the module arguments, if required module_args['_ansible_no_log'] = self._play_context.no_log or C.DEFAULT_NO_TARGET_SYSLOG # set debug in the module arguments, if required module_args['_ansible_debug'] = C.DEFAULT_DEBUG # let module know we are in diff mode module_args['_ansible_diff'] = self._play_context.diff # let module know our verbosity module_args['_ansible_verbosity'] = display.verbosity # give the module information about the ansible version module_args['_ansible_version'] = __version__ # set the syslog facility to be used in the module module_args['_ansible_syslog_facility'] = task_vars.get('ansible_syslog_facility', C.DEFAULT_SYSLOG_FACILITY) # let module know about filesystems that selinux treats specially module_args['_ansible_selinux_special_fs'] = C.DEFAULT_SELINUX_SPECIAL_FS (module_style, shebang, module_data) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars) if not shebang: raise AnsibleError("module (%s) is missing interpreter line" % module_name) # get nfs info for mount python packages ansible_nfs_src = task_vars.get("ansible_nfs_src", None) ansible_nfs_dest = task_vars.get("ansible_nfs_dest", None) # a remote tmp path may be necessary and not already created remote_module_path = None args_file_path = None if not tmp and self._late_needs_tmp_path(tmp, module_style): tmp = self._make_tmp_path(remote_user) if tmp: remote_module_filename = self._connection._shell.get_remote_filename(module_name) remote_module_path = self._connection._shell.join_path(tmp, remote_module_filename) if module_style in ['old', 'non_native_want_json']: # we'll also need a temp file to hold our module arguments args_file_path = self._connection._shell.join_path(tmp, 'args') if remote_module_path or module_style != 'new': display.debug("transferring module to remote") self._transfer_data(remote_module_path, module_data) if module_style == 'old': # we need to dump the module args to a k=v string in a file on # the remote system, which can be read and parsed by the module args_data = "" for k,v in iteritems(module_args): args_data += '%s=%s ' % (k, pipes.quote(text_type(v))) self._transfer_data(args_file_path, args_data) elif module_style == 'non_native_want_json': self._transfer_data(args_file_path, json.dumps(module_args)) display.debug("done transferring module to remote") environment_string = self._compute_environment_string() remote_files = None if args_file_path: remote_files = tmp, remote_module_path, args_file_path elif remote_module_path: remote_files = tmp, remote_module_path # Fix permissions of the tmp path and tmp files. This should be # called after all files have been transferred. if remote_files: self._fixup_perms2(remote_files, remote_user) # mount nfs if ansible_nfs_src and ansible_nfs_dest: result = self._mount_nfs(ansible_nfs_src, ansible_nfs_dest) if result['rc'] != 0: raise AnsibleError("mount nfs failed!!! {0}".format(result['stderr'])) cmd = "" in_data = None if self._connection.has_pipelining and self._play_context.pipelining and not C.DEFAULT_KEEP_REMOTE_FILES and module_style == 'new': in_data = module_data else: if remote_module_path: cmd = remote_module_path rm_tmp = None if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp: if not self._play_context.become or self._play_context.become_user == 'root': # not sudoing or sudoing to root, so can cleanup files in the same step rm_tmp = tmp cmd = self._connection._shell.build_module_command(environment_string, shebang, cmd, arg_path=args_file_path, rm_tmp=rm_tmp) cmd = cmd.strip() sudoable = True if module_name == "accelerate": # always run the accelerate module as the user # specified in the play, not the sudo_user sudoable = False res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data) # umount nfs if ansible_nfs_src and ansible_nfs_dest: result = self._umount_nfs(ansible_nfs_dest) if result['rc'] != 0: raise AnsibleError("umount nfs failed!!! {0}".format(result['stderr'])) if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp: if self._play_context.become and self._play_context.become_user != 'root': # not sudoing to root, so maybe can't delete files as that other user # have to clean up temp files as original user in a second step tmp_rm_cmd = self._connection._shell.remove(tmp, recurse=True) tmp_rm_res = self._low_level_execute_command(tmp_rm_cmd, sudoable=False) tmp_rm_data = self._parse_returned_data(tmp_rm_res) if tmp_rm_data.get('rc', 0) != 0: display.warning('Error deleting remote temporary files (rc: {0}, stderr: {1})'.format(tmp_rm_res.get('rc'), tmp_rm_res.get('stderr', 'No error string available.'))) # parse the main result data = self._parse_returned_data(res) # pre-split stdout into lines, if stdout is in the data and there # isn't already a stdout_lines value there if 'stdout' in data and 'stdout_lines' not in data: data['stdout_lines'] = data.get('stdout', u'').splitlines() display.debug("done with _execute_module (%s, %s)" % (module_name, module_args)) return data
集成到normal.py和async.py中,记住要将这两个插件在ansible.cfg中进行配置
from __future__ import (absolute_import, division, print_function) __metaclass__ = type from ansible.plugins.action import ActionBase from ansible.utils.vars import merge_hash from common.ansible_plugins import MagicStackBase class ActionModule(MagicStackBase, ActionBase): def run(self, tmp=None, task_vars=None): if task_vars is None: task_vars = dict() results = super(ActionModule, self).run(tmp, task_vars) # remove as modules might hide due to nolog del results['invocation']['module_args'] results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars)) # Remove special fields from the result, which can only be set # internally by the executor engine. We do this only here in # the 'normal' action, as other action plugins may set this. # # We don't want modules to determine that running the module fires # notify handlers. That's for the playbook to decide. for field in ('_ansible_notify',): if field in results: results.pop(field) return results
- 配置ansible.cfg,将扩展的插件指定为ansible需要的action插件
- 重写插件方法,重点是execute_module
- 执行命令中需要指定Python环境,将需要的参数添加进去nfs挂载和卸载的参数
ansible 51 -m mysql_db -a "state=dump name=all target=/tmp/test.sql" -i hosts -u root -v -e "ansible_nfs_src=172.16.30.170:/web/proxy_env/lib64/python2.7/site-packages ansible_nfs_dest=/root/.pyenv/versions/2.7.10/lib/python2.7/site-packages ansible_python_interpreter=/root/.pyenv/versions/2.7.10/bin/python"
标签:
版权申明:本站文章部分自网络,如有侵权,请联系:west999com@outlook.com
特别注意:本站所有转载文章言论不代表本站观点,本站所提供的摄影照片,插画,设计作品,如需使用,请与原作者联系,版权归原作者所有
- python 之 前端开发(HTTP协议、head标签、img标签、a标签、 2019-08-13
- Python Http-server 使用 2019-07-24
- HTTP: Request中的post和get区别 2019-07-24
- RobotFramework + HTTP接口自动化实现 2019-07-24
- 解决pyhton aiohttp ssl:None [[SSL: CERTIFICATE_VERIFY_FA 2019-07-24
IDC资讯: 主机资讯 注册资讯 托管资讯 vps资讯 网站建设
网站运营: 建站经验 策划盈利 搜索优化 网站推广 免费资源
网络编程: Asp.Net编程 Asp编程 Php编程 Xml编程 Access Mssql Mysql 其它
服务器技术: Web服务器 Ftp服务器 Mail服务器 Dns服务器 安全防护
软件技巧: 其它软件 Word Excel Powerpoint Ghost Vista QQ空间 QQ FlashGet 迅雷
网页制作: FrontPages Dreamweaver Javascript css photoshop fireworks Flash