transifex-client-0.10/0000755000175000001440000000000012247615466015560 5ustar mpessasusers00000000000000transifex-client-0.10/setup.py0000755000175000001440000000337212247614034017270 0ustar mpessasusers00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import os from codecs import BOM from setuptools import setup from txclib import get_version readme_file = open(u'README.rst') long_description = readme_file.read() readme_file.close() if long_description.startswith(BOM): long_description = long_description.lstrip(BOM) long_description = long_description.decode('utf-8') package_data = { '': ['LICENSE', 'README.rst'], } scripts = ['tx'] install_requires = [] extra_args = {} import platform if platform.system() == 'Windows': import py2exe extra_args = { 'console': ['tx'], 'options': {'py2exe': {'bundle_files': 1}}, 'zipfile': None, } setup( name="transifex-client", version=get_version(), scripts=scripts, description="A command line interface for Transifex", long_description=long_description, author="Transifex", author_email="admin@transifex.com", url="https://www.transifex.com", license="GPLv2", dependency_links=[ ], setup_requires=[ ], install_requires=install_requires, tests_require=["mock", ], data_files=[ ], test_suite="tests", zip_safe=False, packages=[ 'txclib', 'txclib.packages', 'txclib.packages.urllib3', 'txclib.packages.urllib3.contrib', 'txclib.packages.urllib3.packages', 'txclib.packages.urllib3.packages.ssl_match_hostname', ], include_package_data=True, package_data=package_data, keywords=('translation', 'localization', 'internationalization',), classifiers=[ 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', ], **extra_args ) transifex-client-0.10/README.rst0000644000175000001440000000214412247321676017245 0ustar mpessasusers00000000000000 ============================= Transifex Command-Line Tool ============================= The Transifex Command-line Client is a command line tool that enables you to easily manage your translations within a project without the need of an elaborate UI system. You can use the command line client to easily create new resources, map locale files to translations and synchronize your Transifex project with your local repository and vice verca. Translators and localization managers can also use it to handle large volumes of translation files easily and without much hassle. Check the full documentation at http://support.transifex.com/customer/portal/topics/440187-transifex-client/articles. Installing ========== You can install the latest version of transifex-client running ``pip install transifex-client`` or ``easy_install transifex-client`` You can also install the `in-development version`_ of transifex-client with ``pip install transifex-client==dev`` or ``easy_install transifex-client==dev``. .. _in-development version: http://github.com/transifex/transifex-client/tarball/master#egg=transifex-client-dev transifex-client-0.10/LICENSE0000644000175000001440000004314212234723375016564 0ustar mpessasusers00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. transifex-client-0.10/PKG-INFO0000644000175000001440000000346612247615466016666 0ustar mpessasusers00000000000000Metadata-Version: 1.1 Name: transifex-client Version: 0.10 Summary: A command line interface for Transifex Home-page: https://www.transifex.com Author: Transifex Author-email: admin@transifex.com License: GPLv2 Description: ============================= Transifex Command-Line Tool ============================= The Transifex Command-line Client is a command line tool that enables you to easily manage your translations within a project without the need of an elaborate UI system. You can use the command line client to easily create new resources, map locale files to translations and synchronize your Transifex project with your local repository and vice verca. Translators and localization managers can also use it to handle large volumes of translation files easily and without much hassle. Check the full documentation at http://support.transifex.com/customer/portal/topics/440187-transifex-client/articles. Installing ========== You can install the latest version of transifex-client running ``pip install transifex-client`` or ``easy_install transifex-client`` You can also install the `in-development version`_ of transifex-client with ``pip install transifex-client==dev`` or ``easy_install transifex-client==dev``. .. _in-development version: http://github.com/transifex/transifex-client/tarball/master#egg=transifex-client-dev Keywords: translation,localization,internationalization Platform: UNKNOWN Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 transifex-client-0.10/tx0000755000175000001440000000717012247321676016143 0ustar mpessasusers00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from optparse import OptionParser, OptionValueError import os import sys import ssl import errno from txclib import utils from txclib import get_version from txclib.log import set_log_level, logger reload(sys) # WTF? Otherwise setdefaultencoding doesn't work # This block ensures that ^C interrupts are handled quietly. try: import signal def exithandler(signum,frame): signal.signal(signal.SIGINT, signal.SIG_IGN) signal.signal(signal.SIGTERM, signal.SIG_IGN) sys.exit(1) signal.signal(signal.SIGINT, exithandler) signal.signal(signal.SIGTERM, exithandler) if hasattr(signal, 'SIGPIPE'): signal.signal(signal.SIGPIPE, signal.SIG_DFL) except KeyboardInterrupt: sys.exit(1) # When we open file with f = codecs.open we specifi FROM what encoding to read # This sets the encoding for the strings which are created with f.read() sys.setdefaultencoding('utf-8') def main(argv): """ Here we parse the flags (short, long) and we instantiate the classes. """ usage = "usage: %prog [options] command [cmd_options]" description = "This is the Transifex command line client which"\ " allows you to manage your translations locally and sync"\ " them with the master Transifex server.\nIf you'd like to"\ " check the available commands issue `%prog help` or if you"\ " just want help with a specific command issue `%prog help"\ " command`" parser = OptionParser( usage=usage, version=get_version(), description=description ) parser.disable_interspersed_args() parser.add_option( "-d", "--debug", action="store_true", dest="debug", default=False, help=("enable debug messages") ) parser.add_option( "-q", "--quiet", action="store_true", dest="quiet", default=False, help="don't print status messages to stdout" ) parser.add_option( "-r", "--root", action="store", dest="root_dir", type="string", default=None, help="change root directory (default is cwd)" ) parser.add_option( "--traceback", action="store_true", dest="trace", default=False, help="print full traceback on exceptions" ) parser.add_option( "--disable-colors", action="store_true", dest="color_disable", default=(os.name == 'nt' or not sys.stdout.isatty()), help="disable colors in the output of commands" ) (options, args) = parser.parse_args() if len(args) < 1: parser.error("No command was given") utils.DISABLE_COLORS = options.color_disable # set log level if options.quiet: set_log_level('WARNING') elif options.debug: set_log_level('DEBUG') # find .tx path_to_tx = options.root_dir or utils.find_dot_tx() cmd = args[0] try: utils.exec_command(cmd, args[1:], path_to_tx) except ssl.SSLError as e: if 'certificate verify failed' in e.strerror: logger.error( 'Error: Could not verify the SSL certificate of the remote host' ) else: logger.error(errno.errorcode[e.errno]) sys.exit(1) except utils.UnknownCommandError: logger.error("tx: Command %s not found" % cmd) except SystemExit: sys.exit() except: import traceback if options.trace: traceback.print_exc() else: formatted_lines = traceback.format_exc().splitlines() logger.error(formatted_lines[-1]) sys.exit(1) # Run baby :) ... run if __name__ == "__main__": # sys.argv[0] is the name of the script that we’re running. main(sys.argv[1:]) transifex-client-0.10/setup.cfg0000644000175000001440000000007312247615466017401 0ustar mpessasusers00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 transifex-client-0.10/txclib/0000755000175000001440000000000012247615466017045 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/utils.py0000644000175000001440000001337212247321676020562 0ustar mpessasusers00000000000000import os, sys, re, errno import ssl try: from json import loads as parse_json, dumps as compile_json except ImportError: from simplejson import loads as parse_json, dumps as compile_json from txclib.packages import urllib3 from txclib.urls import API_URLS from txclib.exceptions import UnknownCommandError from txclib.paths import posix_path, native_path, posix_sep from txclib.web import user_agent_identifier from txclib.log import logger def find_dot_tx(path=os.path.curdir, previous=None): """Return the path where .tx folder is found. The 'path' should be a DIRECTORY. This process is functioning recursively from the current directory to each one of the ancestors dirs. """ path = os.path.abspath(path) if path == previous: return None joined = os.path.join(path, ".tx") if os.path.isdir(joined): return path else: return find_dot_tx(os.path.dirname(path), path) ################################################# # Parse file filter expressions and create regex def regex_from_filefilter(file_filter, root_path = os.path.curdir): """Create proper regex from expression.""" # Force expr to be a valid regex expr (escaped) but keep intact expr_re = re.escape( posix_path(os.path.join(root_path, native_path(file_filter))) ) expr_re = expr_re.replace("\\", '').replace( '', '([^%(sep)s]+)' % { 'sep': re.escape(posix_sep)}) return "^%s$" % expr_re TX_URLS = { 'resource': '(?Phttps?://(\w|\.|:|-)+)/projects/p/(?P(\w|-)+)/resource/(?P(\w|-)+)/?$', 'release': '(?Phttps?://(\w|\.|:|-)+)/projects/p/(?P(\w|-)+)/r/(?P(\w|-)+)/?$', 'project': '(?Phttps?://(\w|\.|:|-)+)/projects/p/(?P(\w|-)+)/?$', } def parse_tx_url(url): """ Try to match given url to any of the valid url patterns specified in TX_URLS. If not match is found, we raise exception """ for type_ in TX_URLS.keys(): pattern = TX_URLS[type_] m = re.match(pattern, url) if m: return type_, m.groupdict() raise Exception( "tx: Malformed url given. Please refer to our docs: http://bit.ly/txautor" ) def get_details(api_call, username, password, *args, **kwargs): """ Get the tx project info through the API. This function can also be used to check the existence of a project. """ url = (API_URLS[api_call] % (kwargs)).encode('UTF-8') conn = urllib3.connection_from_url(kwargs['hostname']) headers = urllib3.util.make_headers( basic_auth='{0}:{1}'.format(username, password), accept_encoding=True, user_agent=user_agent_identifier(), ) try: r = conn.request('GET', url, headers=headers) remote_project = parse_json(r.data) r.close() return remote_project except ssl.SSLError: logger.error("Invalid SSL certificate") raise except Exception, e: logger.debug(unicode(e)) raise def valid_slug(slug): """ Check if a slug contains only valid characters. Valid chars include [-_\w] """ try: a, b = slug.split('.') except ValueError: return False else: if re.match("^[A-Za-z0-9_-]*$", a) and re.match("^[A-Za-z0-9_-]*$", b): return True return False def discover_commands(): """ Inspect commands.py and find all available commands """ import inspect from txclib import commands command_table = {} fns = inspect.getmembers(commands, inspect.isfunction) for name, fn in fns: if name.startswith("cmd_"): command_table.update({ name.split("cmd_")[1]:fn }) return command_table def exec_command(command, *args, **kwargs): """ Execute given command """ commands = discover_commands() try: cmd_fn = commands[command] except KeyError: raise UnknownCommandError cmd_fn(*args,**kwargs) def mkdir_p(path): try: if path: os.makedirs(path) except OSError, exc: # Python >2.5 if exc.errno == errno.EEXIST: pass else: raise def confirm(prompt='Continue?', default=True): """ Prompt the user for a Yes/No answer. Args: prompt: The text displayed to the user ([Y/n] will be appended) default: If the default value will be yes or no """ valid_yes = ['Y', 'y', 'Yes', 'yes', ] valid_no = ['N', 'n', 'No', 'no', ] if default: prompt = prompt + '[Y/n]' valid_yes.append('') else: prompt = prompt + '[y/N]' valid_no.append('') ans = raw_input(prompt) while (ans not in valid_yes and ans not in valid_no): ans = raw_input(prompt) return ans in valid_yes # Stuff for command line colored output COLORS = [ 'BLACK', 'RED', 'GREEN', 'YELLOW', 'BLUE', 'MAGENTA', 'CYAN', 'WHITE' ] DISABLE_COLORS = False def color_text(text, color_name, bold=False): """ This command can be used to colorify command line output. If the shell doesn't support this or the --disable-colors options has been set, it just returns the plain text. Usage: print "%s" % color_text("This text is red", "RED") """ if color_name in COLORS and not DISABLE_COLORS: return '\033[%s;%sm%s\033[0m' % ( int(bold), COLORS.index(color_name) + 30, text) else: return text def files_in_project(curpath): """ Iterate over the files in the project. Return each file under ``curpath`` with its absolute name. """ for root, dirs, files in os.walk(curpath, followlinks=True): for f in files: yield os.path.abspath(os.path.join(root, f)) transifex-client-0.10/txclib/log.py0000644000175000001440000000171312247321676020177 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- """ Add logging capabilities to tx-client. """ import sys import logging _logger = logging.getLogger('txclib.packages.urllib3') _logger.setLevel(logging.CRITICAL) _logger = logging.getLogger('txclib') _logger.setLevel(logging.INFO) _formatter = logging.Formatter('%(message)s') _error_handler = logging.StreamHandler(sys.stderr) _error_handler.setLevel(logging.ERROR) _error_handler.setFormatter(_formatter) _logger.addHandler(_error_handler) _msg_handler = logging.StreamHandler(sys.stdout) _msg_handler.setLevel(logging.DEBUG) _msg_handler.setFormatter(_formatter) _msg_filter = logging.Filter() _msg_filter.filter = lambda r: r.levelno < logging.ERROR _msg_handler.addFilter(_msg_filter) _logger.addHandler(_msg_handler) logger = _logger def set_log_level(level): """Set the level for the logger. Args: level: A string among DEBUG, INFO, WARNING, ERROR, CRITICAL. """ logger.setLevel(getattr(logging, level)) transifex-client-0.10/txclib/commands.py0000644000175000001440000005232312247365732021223 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- """ In this file we have all the top level commands for the transifex client. Since we're using a way to automatically list them and execute them, when adding code to this file you must take care of the following: * Added functions must begin with 'cmd_' followed by the actual name of the command being used in the command line (eg cmd_init) * The description for each function that we display to the user is read from the func_doc attribute which reads the doc string. So, when adding docstring to a new function make sure you add an oneliner which is descriptive and is meant to be seen by the user. * When including libraries, it's best if you include modules instead of functions because that way our function resolution will work faster and the chances of overlapping are minimal * All functions should use the OptionParser and should have a usage and descripition field. """ import os import re, shutil import sys from optparse import OptionParser, OptionGroup import ConfigParser from txclib import utils, project from txclib.utils import parse_json, compile_json, files_in_project from txclib.config import OrderedRawConfigParser from txclib.exceptions import UnInitializedError from txclib.parsers import delete_parser, help_parser, parse_csv_option, \ status_parser, pull_parser, set_parser, push_parser, init_parser from txclib.paths import posix_path from txclib.log import logger def cmd_init(argv, path_to_tx): "Initialize a new transifex project." parser = init_parser() (options, args) = parser.parse_args(argv) if len(args) > 1: parser.error("Too many arguments were provided. Aborting...") if args: path_to_tx = args[0] else: path_to_tx = os.getcwd() if os.path.isdir(os.path.join(path_to_tx,".tx")): logger.info("tx: There is already a tx folder!") reinit = raw_input("Do you want to delete it and reinit the project? [y/N]: ") while (reinit != 'y' and reinit != 'Y' and reinit != 'N' and reinit != 'n' and reinit != ''): reinit = raw_input("Do you want to delete it and reinit the project? [y/N]: ") if not reinit or reinit in ['N', 'n', 'NO', 'no', 'No']: return # Clean the old settings # FIXME: take a backup else: rm_dir = os.path.join(path_to_tx, ".tx") shutil.rmtree(rm_dir) logger.info("Creating .tx folder...") os.mkdir(os.path.join(path_to_tx,".tx")) # Handle the credentials through transifexrc home = os.path.expanduser("~") txrc = os.path.join(home, ".transifexrc") config = OrderedRawConfigParser() default_transifex = "https://www.transifex.com" transifex_host = options.host or raw_input("Transifex instance [%s]: " % default_transifex) if not transifex_host: transifex_host = default_transifex if not transifex_host.startswith(('http://', 'https://')): transifex_host = 'https://' + transifex_host config_file = os.path.join(path_to_tx, ".tx", "config") if not os.path.exists(config_file): # The path to the config file (.tx/config) logger.info("Creating skeleton...") config = OrderedRawConfigParser() config.add_section('main') config.set('main', 'host', transifex_host) # Touch the file if it doesn't exist logger.info("Creating config file...") fh = open(config_file, 'w') config.write(fh) fh.close() prj = project.Project(path_to_tx) prj.getset_host_credentials(transifex_host, user=options.user, password=options.password) prj.save() logger.info("Done.") def cmd_set(argv, path_to_tx): "Add local or remote files under transifex" parser = set_parser() (options, args) = parser.parse_args(argv) # Implement options/args checks # TODO !!!!!!! if options.local: try: expression = args[0] except IndexError: parser.error("Please specify an expression.") if not options.resource: parser.error("Please specify a resource") if not options.source_language: parser.error("Please specify a source language.") if not '' in expression: parser.error("The expression you have provided is not valid.") if not utils.valid_slug(options.resource): parser.error("Invalid resource slug. The format is "\ ". and the valid characters include [_-\w].") _auto_local(path_to_tx, options.resource, source_language=options.source_language, expression = expression, source_file=options.source_file, execute=options.execute, regex=False) if options.execute: _set_minimum_perc(options.resource, options.minimum_perc, path_to_tx) _set_mode(options.resource, options.mode, path_to_tx) _set_type(options.resource, options.i18n_type, path_to_tx) return if options.remote: try: url = args[0] except IndexError: parser.error("Please specify an remote url") _auto_remote(path_to_tx, url) _set_minimum_perc(options.resource, options.minimum_perc, path_to_tx) _set_mode(options.resource, options.mode, path_to_tx) return if options.is_source: resource = options.resource if not resource: parser.error("You must specify a resource name with the" " -r|--resource flag.") lang = options.language if not lang: parser.error("Please specify a source language.") if len(args) != 1: parser.error("Please specify a file.") if not utils.valid_slug(resource): parser.error("Invalid resource slug. The format is "\ ". and the valid characters include [_-\w].") file = args[0] # Calculate relative path path_to_file = os.path.relpath(file, path_to_tx) _set_source_file(path_to_tx, resource, options.language, path_to_file) elif options.resource or options.language: resource = options.resource lang = options.language if len(args) != 1: parser.error("Please specify a file") # Calculate relative path path_to_file = os.path.relpath(args[0], path_to_tx) try: _go_to_dir(path_to_tx) except UnInitializedError, e: utils.logger.error(e) return if not utils.valid_slug(resource): parser.error("Invalid resource slug. The format is "\ ". and the valid characters include [_-\w].") _set_translation(path_to_tx, resource, lang, path_to_file) _set_mode(options.resource, options.mode, path_to_tx) _set_type(options.resource, options.i18n_type, path_to_tx) _set_minimum_perc(options.resource, options.minimum_perc, path_to_tx) logger.info("Done.") return def _auto_local(path_to_tx, resource, source_language, expression, execute=False, source_file=None, regex=False): """Auto configure local project.""" # The path everything will be relative to curpath = os.path.abspath(os.curdir) # Force expr to be a valid regex expr (escaped) but keep intact expr_re = utils.regex_from_filefilter(expression, curpath) expr_rec = re.compile(expr_re) if not execute: logger.info("Only printing the commands which will be run if the " "--execute switch is specified.") # First, let's construct a dictionary of all matching files. # Note: Only the last matching file of a language will be stored. translation_files = {} for f_path in files_in_project(curpath): match = expr_rec.match(posix_path(f_path)) if match: lang = match.group(1) if lang == source_language and not source_file: source_file = f_path else: translation_files[lang] = f_path if not source_file: raise Exception("Could not find a source language file. Please run" " set --source manually and then re-run this command or provide" " the source file with the -s flag.") if execute: logger.info("Updating source for resource %s ( %s -> %s )." % (resource, source_language, os.path.relpath(source_file, path_to_tx))) _set_source_file(path_to_tx, resource, source_language, os.path.relpath(source_file, path_to_tx)) else: logger.info('\ntx set --source -r %(res)s -l %(lang)s %(file)s\n' % { 'res': resource, 'lang': source_language, 'file': os.path.relpath(source_file, curpath)}) prj = project.Project(path_to_tx) root_dir = os.path.abspath(path_to_tx) if execute: try: prj.config.get("%s" % resource, "source_file") except ConfigParser.NoSectionError: raise Exception("No resource with slug \"%s\" was found.\nRun 'tx set --auto" "-local -r %s \"expression\"' to do the initial configuration." % resource) # Now let's handle the translation files. if execute: logger.info("Updating file expression for resource %s ( %s )." % (resource, expression)) # Eval file_filter relative to root dir file_filter = posix_path( os.path.relpath(os.path.join(curpath, expression), path_to_tx) ) prj.config.set("%s" % resource, "file_filter", file_filter) else: for (lang, f_path) in sorted(translation_files.items()): logger.info('tx set -r %(res)s -l %(lang)s %(file)s' % { 'res': resource, 'lang': lang, 'file': os.path.relpath(f_path, curpath)}) if execute: prj.save() def _auto_remote(path_to_tx, url): """ Initialize a remote release/project/resource to the current directory. """ logger.info("Auto configuring local project from remote URL...") type, vars = utils.parse_tx_url(url) prj = project.Project(path_to_tx) username, password = prj.getset_host_credentials(vars['hostname']) if type == 'project': logger.info("Getting details for project %s" % vars['project']) proj_info = utils.get_details('project_details', username, password, hostname = vars['hostname'], project = vars['project']) resources = [ '.'.join([vars['project'], r['slug']]) for r in proj_info['resources'] ] logger.info("%s resources found. Configuring..." % len(resources)) elif type == 'release': logger.info("Getting details for release %s" % vars['release']) rel_info = utils.get_details('release_details', username, password, hostname = vars['hostname'], project = vars['project'], release = vars['release']) resources = [] for r in rel_info['resources']: if r.has_key('project'): resources.append('.'.join([r['project']['slug'], r['slug']])) else: resources.append('.'.join([vars['project'], r['slug']])) logger.info("%s resources found. Configuring..." % len(resources)) elif type == 'resource': logger.info("Getting details for resource %s" % vars['resource']) resources = [ '.'.join([vars['project'], vars['resource']]) ] else: raise("Url '%s' is not recognized." % url) for resource in resources: logger.info("Configuring resource %s." % resource) proj, res = resource.split('.') res_info = utils.get_details('resource_details', username, password, hostname = vars['hostname'], project = proj, resource=res) try: source_lang = res_info['source_language_code'] i18n_type = res_info['i18n_type'] except KeyError: raise Exception("Remote server seems to be running an unsupported version" " of Transifex. Either update your server software of fallback" " to a previous version of transifex-client.") prj.set_remote_resource( resource=resource, host = vars['hostname'], source_lang = source_lang, i18n_type = i18n_type) prj.save() def cmd_push(argv, path_to_tx): "Push local files to remote server" parser = push_parser() (options, args) = parser.parse_args(argv) force_creation = options.force_creation languages = parse_csv_option(options.languages) resources = parse_csv_option(options.resources) skip = options.skip_errors prj = project.Project(path_to_tx) if not (options.push_source or options.push_translations): parser.error("You need to specify at least one of the -s|--source," " -t|--translations flags with the push command.") prj.push( force=force_creation, resources=resources, languages=languages, skip=skip, source=options.push_source, translations=options.push_translations, no_interactive=options.no_interactive ) logger.info("Done.") def cmd_pull(argv, path_to_tx): "Pull files from remote server to local repository" parser = pull_parser() (options, args) = parser.parse_args(argv) if options.fetchall and options.languages: parser.error("You can't user a language filter along with the"\ " -a|--all option") languages = parse_csv_option(options.languages) resources = parse_csv_option(options.resources) pseudo = options.pseudo skip = options.skip_errors minimum_perc = options.minimum_perc or None try: _go_to_dir(path_to_tx) except UnInitializedError, e: utils.logger.error(e) return # instantiate the project.Project prj = project.Project(path_to_tx) prj.pull( languages=languages, resources=resources, overwrite=options.overwrite, fetchall=options.fetchall, fetchsource=options.fetchsource, force=options.force, skip=skip, minimum_perc=minimum_perc, mode=options.mode, pseudo=pseudo ) logger.info("Done.") def _set_source_file(path_to_tx, resource, lang, path_to_file): """Reusable method to set source file.""" proj, res = resource.split('.') if not proj or not res: raise Exception("\"%s.%s\" is not a valid resource identifier. It should" " be in the following format project_slug.resource_slug." % (proj, res)) if not lang: raise Exception("You haven't specified a source language.") try: _go_to_dir(path_to_tx) except UnInitializedError, e: utils.logger.error(e) return if not os.path.exists(path_to_file): raise Exception("tx: File ( %s ) does not exist." % os.path.join(path_to_tx, path_to_file)) # instantiate the project.Project prj = project.Project(path_to_tx) root_dir = os.path.abspath(path_to_tx) if root_dir not in os.path.normpath(os.path.abspath(path_to_file)): raise Exception("File must be under the project root directory.") logger.info("Setting source file for resource %s.%s ( %s -> %s )." % ( proj, res, lang, path_to_file)) path_to_file = os.path.relpath(path_to_file, root_dir) prj = project.Project(path_to_tx) # FIXME: Check also if the path to source file already exists. try: try: prj.config.get("%s.%s" % (proj, res), "source_file") except ConfigParser.NoSectionError: prj.config.add_section("%s.%s" % (proj, res)) except ConfigParser.NoOptionError: pass finally: prj.config.set( "%s.%s" % (proj, res), "source_file", posix_path(path_to_file) ) prj.config.set("%s.%s" % (proj, res), "source_lang", lang) prj.save() def _set_translation(path_to_tx, resource, lang, path_to_file): """Reusable method to set translation file.""" proj, res = resource.split('.') if not project or not resource: raise Exception("\"%s\" is not a valid resource identifier. It should" " be in the following format project_slug.resource_slug." % resource) try: _go_to_dir(path_to_tx) except UnInitializedError, e: utils.logger.error(e) return # Warn the user if the file doesn't exist if not os.path.exists(path_to_file): logger.info("Warning: File '%s' doesn't exist." % path_to_file) # instantiate the project.Project prj = project.Project(path_to_tx) root_dir = os.path.abspath(path_to_tx) if root_dir not in os.path.normpath(os.path.abspath(path_to_file)): raise Exception("File must be under the project root directory.") if lang == prj.config.get("%s.%s" % (proj, res), "source_lang"): raise Exception("tx: You cannot set translation file for the source language." " Source languages contain the strings which will be translated!") logger.info("Updating translations for resource %s ( %s -> %s )." % (resource, lang, path_to_file)) path_to_file = os.path.relpath(path_to_file, root_dir) prj.config.set( "%s.%s" % (proj, res), "trans.%s" % lang, posix_path(path_to_file) ) prj.save() def cmd_status(argv, path_to_tx): "Print status of current project" parser = status_parser() (options, args) = parser.parse_args(argv) resources = parse_csv_option(options.resources) prj = project.Project(path_to_tx) resources = prj.get_chosen_resources(resources) resources_num = len(resources) for idx, res in enumerate(resources): p, r = res.split('.') logger.info("%s -> %s (%s of %s)" % (p, r, idx + 1, resources_num)) logger.info("Translation Files:") slang = prj.get_resource_option(res, 'source_lang') sfile = prj.get_resource_option(res, 'source_file') or "N/A" lang_map = prj.get_resource_lang_mapping(res) logger.info(" - %s: %s (%s)" % (utils.color_text(slang, "RED"), sfile, utils.color_text("source", "YELLOW"))) files = prj.get_resource_files(res) fkeys = files.keys() fkeys.sort() for lang in fkeys: local_lang = lang if lang in lang_map.values(): local_lang = lang_map.flip[lang] logger.info(" - %s: %s" % (utils.color_text(local_lang, "RED"), files[lang])) logger.info("") def cmd_help(argv, path_to_tx): """List all available commands""" parser = help_parser() (options, args) = parser.parse_args(argv) if len(args) > 1: parser.error("Multiple arguments received. Exiting...") # Get all commands fns = utils.discover_commands() # Print help for specific command if len(args) == 1: try: fns[argv[0]](['--help'], path_to_tx) except KeyError: utils.logger.error("Command %s not found" % argv[0]) # or print summary of all commands # the code below will only be executed if the KeyError exception is thrown # becuase in all other cases the function called with --help will exit # instead of return here keys = fns.keys() keys.sort() logger.info("Transifex command line client.\n") logger.info("Available commands are:") for key in keys: logger.info(" %-15s\t%s" % (key, fns[key].func_doc)) logger.info("\nFor more information run %s command --help" % sys.argv[0]) def cmd_delete(argv, path_to_tx): "Delete an accessible resource or translation in a remote server." parser = delete_parser() (options, args) = parser.parse_args(argv) languages = parse_csv_option(options.languages) resources = parse_csv_option(options.resources) skip = options.skip_errors force = options.force_delete prj = project.Project(path_to_tx) prj.delete(resources, languages, skip, force) logger.info("Done.") def _go_to_dir(path): """Change the current working directory to the directory specified as argument. Args: path: The path to chdor to. Raises: UnInitializedError, in case the directory has not been initialized. """ if path is None: raise UnInitializedError( "Directory has not been initialzied. " "Did you forget to run 'tx init' first?" ) os.chdir(path) def _set_minimum_perc(resource, value, path_to_tx): """Set the minimum percentage in the .tx/config file.""" args = (resource, 'minimum_perc', value, path_to_tx, 'set_min_perc') _set_project_option(*args) def _set_mode(resource, value, path_to_tx): """Set the mode in the .tx/config file.""" args = (resource, 'mode', value, path_to_tx, 'set_default_mode') _set_project_option(*args) def _set_type(resource, value, path_to_tx): """Set the i18n type in the .tx/config file.""" args = (resource, 'type', value, path_to_tx, 'set_i18n_type') _set_project_option(*args) def _set_project_option(resource, name, value, path_to_tx, func_name): """Save the option to the project config file.""" if value is None: return if not resource: logger.debug("Setting the %s for all resources." % name) resources = [] else: logger.debug("Setting the %s for resource %s." % (name, resource)) resources = [resource, ] prj = project.Project(path_to_tx) getattr(prj, func_name)(resources, value) prj.save() transifex-client-0.10/txclib/processors.py0000644000175000001440000000272412234723375021621 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- """ Module for API-related calls. """ import urlparse def hostname_tld_migration(hostname): """ Migrate transifex.net to transifex.com. :param hostname: The hostname to migrate (if needed). :returns: A hostname with the transifex.com domain (if needed). """ parts = urlparse.urlparse(hostname) if parts.hostname.endswith('transifex.net'): hostname = hostname.replace('transifex.net', 'transifex.com', 1) return hostname def hostname_ssl_migration(hostname): """ Migrate Transifex hostnames to use HTTPS. :param hostname: The hostname to migrate (if needed). :returns: A https hostname (if needed). """ parts = urlparse.urlparse(hostname) is_transifex = ( parts.hostname[-14:-3] == '.transifex.' or parts.hostname == 'transifex.net' or parts.hostname == 'transifex.com' ) is_https = parts.scheme == 'https' if is_transifex and not is_https: if not parts.scheme: hostname = 'https:' + hostname else: hostname = hostname.replace(parts.scheme, 'https', 1) return hostname def visit_hostname(hostname): """ Have a chance to visit a hostname before actually using it. :param hostname: The original hostname. :returns: The hostname with the necessary changes. """ for processor in [hostname_ssl_migration, hostname_tld_migration, ]: hostname = processor(hostname) return hostname transifex-client-0.10/txclib/packages/0000755000175000001440000000000012247615466020623 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/0000755000175000001440000000000012247615466022177 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/util.py0000644000175000001440000005001512247321676023524 0ustar mpessasusers00000000000000# urllib3/util.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php from base64 import b64encode from binascii import hexlify, unhexlify from collections import namedtuple from hashlib import md5, sha1 from socket import error as SocketError, _GLOBAL_DEFAULT_TIMEOUT import time try: from select import poll, POLLIN except ImportError: # `poll` doesn't exist on OSX and other platforms poll = False try: from select import select except ImportError: # `select` doesn't exist on AppEngine. select = False try: # Test for SSL features SSLContext = None HAS_SNI = False import ssl from ssl import wrap_socket, CERT_NONE, PROTOCOL_SSLv23 from ssl import SSLContext # Modern SSL? from ssl import HAS_SNI # Has SNI? except ImportError: pass from .packages import six from .exceptions import LocationParseError, SSLError, TimeoutStateError _Default = object() # The default timeout to use for socket connections. This is the attribute used # by httplib to define the default timeout def current_time(): """ Retrieve the current time, this function is mocked out in unit testing. """ return time.time() class Timeout(object): """ Utility object for storing timeout values. Example usage: .. code-block:: python timeout = urllib3.util.Timeout(connect=2.0, read=7.0) pool = HTTPConnectionPool('www.google.com', 80, timeout=timeout) pool.request(...) # Etc, etc :param connect: The maximum amount of time to wait for a connection attempt to a server to succeed. Omitting the parameter will default the connect timeout to the system default, probably `the global default timeout in socket.py `_. None will set an infinite timeout for connection attempts. :type connect: integer, float, or None :param read: The maximum amount of time to wait between consecutive read operations for a response from the server. Omitting the parameter will default the read timeout to the system default, probably `the global default timeout in socket.py `_. None will set an infinite timeout. :type read: integer, float, or None :param total: The maximum amount of time to wait for an HTTP request to connect and return. This combines the connect and read timeouts into one. In the event that both a connect timeout and a total are specified, or a read timeout and a total are specified, the shorter timeout will be applied. Defaults to None. :type total: integer, float, or None .. note:: Many factors can affect the total amount of time for urllib3 to return an HTTP response. Specifically, Python's DNS resolver does not obey the timeout specified on the socket. Other factors that can affect total request time include high CPU load, high swap, the program running at a low priority level, or other behaviors. The observed running time for urllib3 to return a response may be greater than the value passed to `total`. In addition, the read and total timeouts only measure the time between read operations on the socket connecting the client and the server, not the total amount of time for the request to return a complete response. As an example, you may want a request to return within 7 seconds or fail, so you set the ``total`` timeout to 7 seconds. If the server sends one byte to you every 5 seconds, the request will **not** trigger time out. This case is admittedly rare. """ #: A sentinel object representing the default timeout value DEFAULT_TIMEOUT = _GLOBAL_DEFAULT_TIMEOUT def __init__(self, connect=_Default, read=_Default, total=None): self._connect = self._validate_timeout(connect, 'connect') self._read = self._validate_timeout(read, 'read') self.total = self._validate_timeout(total, 'total') self._start_connect = None def __str__(self): return '%s(connect=%r, read=%r, total=%r)' % ( type(self).__name__, self._connect, self._read, self.total) @classmethod def _validate_timeout(cls, value, name): """ Check that a timeout attribute is valid :param value: The timeout value to validate :param name: The name of the timeout attribute to validate. This is used for clear error messages :return: the value :raises ValueError: if the type is not an integer or a float, or if it is a numeric value less than zero """ if value is _Default: return cls.DEFAULT_TIMEOUT if value is None or value is cls.DEFAULT_TIMEOUT: return value try: float(value) except (TypeError, ValueError): raise ValueError("Timeout value %s was %s, but it must be an " "int or float." % (name, value)) try: if value < 0: raise ValueError("Attempted to set %s timeout to %s, but the " "timeout cannot be set to a value less " "than 0." % (name, value)) except TypeError: # Python 3 raise ValueError("Timeout value %s was %s, but it must be an " "int or float." % (name, value)) return value @classmethod def from_float(cls, timeout): """ Create a new Timeout from a legacy timeout value. The timeout value used by httplib.py sets the same timeout on the connect(), and recv() socket requests. This creates a :class:`Timeout` object that sets the individual timeouts to the ``timeout`` value passed to this function. :param timeout: The legacy timeout value :type timeout: integer, float, sentinel default object, or None :return: a Timeout object :rtype: :class:`Timeout` """ return Timeout(read=timeout, connect=timeout) def clone(self): """ Create a copy of the timeout object Timeout properties are stored per-pool but each request needs a fresh Timeout object to ensure each one has its own start/stop configured. :return: a copy of the timeout object :rtype: :class:`Timeout` """ # We can't use copy.deepcopy because that will also create a new object # for _GLOBAL_DEFAULT_TIMEOUT, which socket.py uses as a sentinel to # detect the user default. return Timeout(connect=self._connect, read=self._read, total=self.total) def start_connect(self): """ Start the timeout clock, used during a connect() attempt :raises urllib3.exceptions.TimeoutStateError: if you attempt to start a timer that has been started already. """ if self._start_connect is not None: raise TimeoutStateError("Timeout timer has already been started.") self._start_connect = current_time() return self._start_connect def get_connect_duration(self): """ Gets the time elapsed since the call to :meth:`start_connect`. :return: the elapsed time :rtype: float :raises urllib3.exceptions.TimeoutStateError: if you attempt to get duration for a timer that hasn't been started. """ if self._start_connect is None: raise TimeoutStateError("Can't get connect duration for timer " "that has not started.") return current_time() - self._start_connect @property def connect_timeout(self): """ Get the value to use when setting a connection timeout. This will be a positive float or integer, the value None (never timeout), or the default system timeout. :return: the connect timeout :rtype: int, float, :attr:`Timeout.DEFAULT_TIMEOUT` or None """ if self.total is None: return self._connect if self._connect is None or self._connect is self.DEFAULT_TIMEOUT: return self.total return min(self._connect, self.total) @property def read_timeout(self): """ Get the value for the read timeout. This assumes some time has elapsed in the connection timeout and computes the read timeout appropriately. If self.total is set, the read timeout is dependent on the amount of time taken by the connect timeout. If the connection time has not been established, a :exc:`~urllib3.exceptions.TimeoutStateError` will be raised. :return: the value to use for the read timeout :rtype: int, float, :attr:`Timeout.DEFAULT_TIMEOUT` or None :raises urllib3.exceptions.TimeoutStateError: If :meth:`start_connect` has not yet been called on this object. """ if (self.total is not None and self.total is not self.DEFAULT_TIMEOUT and self._read is not None and self._read is not self.DEFAULT_TIMEOUT): # in case the connect timeout has not yet been established. if self._start_connect is None: return self._read return max(0, min(self.total - self.get_connect_duration(), self._read)) elif self.total is not None and self.total is not self.DEFAULT_TIMEOUT: return max(0, self.total - self.get_connect_duration()) else: return self._read class Url(namedtuple('Url', ['scheme', 'auth', 'host', 'port', 'path', 'query', 'fragment'])): """ Datastructure for representing an HTTP URL. Used as a return value for :func:`parse_url`. """ slots = () def __new__(cls, scheme=None, auth=None, host=None, port=None, path=None, query=None, fragment=None): return super(Url, cls).__new__(cls, scheme, auth, host, port, path, query, fragment) @property def hostname(self): """For backwards-compatibility with urlparse. We're nice like that.""" return self.host @property def request_uri(self): """Absolute path including the query string.""" uri = self.path or '/' if self.query is not None: uri += '?' + self.query return uri @property def netloc(self): """Network location including host and port""" if self.port: return '%s:%d' % (self.host, self.port) return self.host def split_first(s, delims): """ Given a string and an iterable of delimiters, split on the first found delimiter. Return two split parts and the matched delimiter. If not found, then the first part is the full input string. Example: :: >>> split_first('foo/bar?baz', '?/=') ('foo', 'bar?baz', '/') >>> split_first('foo/bar?baz', '123') ('foo/bar?baz', '', None) Scales linearly with number of delims. Not ideal for large number of delims. """ min_idx = None min_delim = None for d in delims: idx = s.find(d) if idx < 0: continue if min_idx is None or idx < min_idx: min_idx = idx min_delim = d if min_idx is None or min_idx < 0: return s, '', None return s[:min_idx], s[min_idx+1:], min_delim def parse_url(url): """ Given a url, return a parsed :class:`.Url` namedtuple. Best-effort is performed to parse incomplete urls. Fields not provided will be None. Partly backwards-compatible with :mod:`urlparse`. Example: :: >>> parse_url('http://google.com/mail/') Url(scheme='http', host='google.com', port=None, path='/', ...) >>> parse_url('google.com:80') Url(scheme=None, host='google.com', port=80, path=None, ...) >>> parse_url('/foo?bar') Url(scheme=None, host=None, port=None, path='/foo', query='bar', ...) """ # While this code has overlap with stdlib's urlparse, it is much # simplified for our needs and less annoying. # Additionally, this implementations does silly things to be optimal # on CPython. scheme = None auth = None host = None port = None path = None fragment = None query = None # Scheme if '://' in url: scheme, url = url.split('://', 1) # Find the earliest Authority Terminator # (http://tools.ietf.org/html/rfc3986#section-3.2) url, path_, delim = split_first(url, ['/', '?', '#']) if delim: # Reassemble the path path = delim + path_ # Auth if '@' in url: auth, url = url.split('@', 1) # IPv6 if url and url[0] == '[': host, url = url.split(']', 1) host += ']' # Port if ':' in url: _host, port = url.split(':', 1) if not host: host = _host if not port.isdigit(): raise LocationParseError("Failed to parse: %s" % url) port = int(port) elif not host and url: host = url if not path: return Url(scheme, auth, host, port, path, query, fragment) # Fragment if '#' in path: path, fragment = path.split('#', 1) # Query if '?' in path: path, query = path.split('?', 1) return Url(scheme, auth, host, port, path, query, fragment) def get_host(url): """ Deprecated. Use :func:`.parse_url` instead. """ p = parse_url(url) return p.scheme or 'http', p.hostname, p.port def make_headers(keep_alive=None, accept_encoding=None, user_agent=None, basic_auth=None): """ Shortcuts for generating request headers. :param keep_alive: If ``True``, adds 'connection: keep-alive' header. :param accept_encoding: Can be a boolean, list, or string. ``True`` translates to 'gzip,deflate'. List will get joined by comma. String will be used as provided. :param user_agent: String representing the user-agent you want, such as "python-urllib3/0.6" :param basic_auth: Colon-separated username:password string for 'authorization: basic ...' auth header. Example: :: >>> make_headers(keep_alive=True, user_agent="Batman/1.0") {'connection': 'keep-alive', 'user-agent': 'Batman/1.0'} >>> make_headers(accept_encoding=True) {'accept-encoding': 'gzip,deflate'} """ headers = {} if accept_encoding: if isinstance(accept_encoding, str): pass elif isinstance(accept_encoding, list): accept_encoding = ','.join(accept_encoding) else: accept_encoding = 'gzip,deflate' headers['accept-encoding'] = accept_encoding if user_agent: headers['user-agent'] = user_agent if keep_alive: headers['connection'] = 'keep-alive' if basic_auth: headers['authorization'] = 'Basic ' + \ b64encode(six.b(basic_auth)).decode('utf-8') return headers def is_connection_dropped(conn): # Platform-specific """ Returns True if the connection is dropped and should be closed. :param conn: :class:`httplib.HTTPConnection` object. Note: For platforms like AppEngine, this will always return ``False`` to let the platform handle connection recycling transparently for us. """ sock = getattr(conn, 'sock', False) if not sock: # Platform-specific: AppEngine return False if not poll: if not select: # Platform-specific: AppEngine return False try: return select([sock], [], [], 0.0)[0] except SocketError: return True # This version is better on platforms that support it. p = poll() p.register(sock, POLLIN) for (fno, ev) in p.poll(0.0): if fno == sock.fileno(): # Either data is buffered (bad), or the connection is dropped. return True def resolve_cert_reqs(candidate): """ Resolves the argument to a numeric constant, which can be passed to the wrap_socket function/method from the ssl module. Defaults to :data:`ssl.CERT_NONE`. If given a string it is assumed to be the name of the constant in the :mod:`ssl` module or its abbrevation. (So you can specify `REQUIRED` instead of `CERT_REQUIRED`. If it's neither `None` nor a string we assume it is already the numeric constant which can directly be passed to wrap_socket. """ if candidate is None: return CERT_NONE if isinstance(candidate, str): res = getattr(ssl, candidate, None) if res is None: res = getattr(ssl, 'CERT_' + candidate) return res return candidate def resolve_ssl_version(candidate): """ like resolve_cert_reqs """ if candidate is None: return PROTOCOL_SSLv23 if isinstance(candidate, str): res = getattr(ssl, candidate, None) if res is None: res = getattr(ssl, 'PROTOCOL_' + candidate) return res return candidate def assert_fingerprint(cert, fingerprint): """ Checks if given fingerprint matches the supplied certificate. :param cert: Certificate as bytes object. :param fingerprint: Fingerprint as string of hexdigits, can be interspersed by colons. """ # Maps the length of a digest to a possible hash function producing # this digest. hashfunc_map = { 16: md5, 20: sha1 } fingerprint = fingerprint.replace(':', '').lower() digest_length, rest = divmod(len(fingerprint), 2) if rest or digest_length not in hashfunc_map: raise SSLError('Fingerprint is of invalid length.') # We need encode() here for py32; works on py2 and p33. fingerprint_bytes = unhexlify(fingerprint.encode()) hashfunc = hashfunc_map[digest_length] cert_digest = hashfunc(cert).digest() if not cert_digest == fingerprint_bytes: raise SSLError('Fingerprints did not match. Expected "{0}", got "{1}".' .format(hexlify(fingerprint_bytes), hexlify(cert_digest))) def is_fp_closed(obj): """ Checks whether a given file-like object is closed. :param obj: The file-like object to check. """ if hasattr(obj, 'fp'): # Object is a container for another file-like object that gets released # on exhaustion (e.g. HTTPResponse) return obj.fp is None return obj.closed if SSLContext is not None: # Python 3.2+ def ssl_wrap_socket(sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None): """ All arguments except `server_hostname` have the same meaning as for :func:`ssl.wrap_socket` :param server_hostname: Hostname of the expected certificate """ context = SSLContext(ssl_version) context.verify_mode = cert_reqs if ca_certs: try: context.load_verify_locations(ca_certs) # Py32 raises IOError # Py33 raises FileNotFoundError except Exception as e: # Reraise as SSLError raise SSLError(e) if certfile: # FIXME: This block needs a test. context.load_cert_chain(certfile, keyfile) if HAS_SNI: # Platform-specific: OpenSSL with enabled SNI return context.wrap_socket(sock, server_hostname=server_hostname) return context.wrap_socket(sock) else: # Python 3.1 and earlier def ssl_wrap_socket(sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None): return wrap_socket(sock, keyfile=keyfile, certfile=certfile, ca_certs=ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version) transifex-client-0.10/txclib/packages/urllib3/packages/0000755000175000001440000000000012247615466023755 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/packages/ssl_match_hostname/0000755000175000001440000000000012247615466027630 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/packages/ssl_match_hostname/__init__.py0000644000175000001440000000672612247321676031751 0ustar mpessasusers00000000000000"""The match_hostname() function from Python 3.2, essential when using SSL.""" import re __version__ = '3.2.2' class CertificateError(ValueError): pass def _dnsname_match(dn, hostname, max_wildcards=1): """Matching according to RFC 6125, section 6.4.3 http://tools.ietf.org/html/rfc6125#section-6.4.3 """ pats = [] if not dn: return False parts = dn.split(r'.') leftmost = parts[0] wildcards = leftmost.count('*') if wildcards > max_wildcards: # Issue #17980: avoid denials of service by refusing more # than one wildcard per fragment. A survery of established # policy among SSL implementations showed it to be a # reasonable choice. raise CertificateError( "too many wildcards in certificate DNS name: " + repr(dn)) # speed up common case w/o wildcards if not wildcards: return dn.lower() == hostname.lower() # RFC 6125, section 6.4.3, subitem 1. # The client SHOULD NOT attempt to match a presented identifier in which # the wildcard character comprises a label other than the left-most label. if leftmost == '*': # When '*' is a fragment by itself, it matches a non-empty dotless # fragment. pats.append('[^.]+') elif leftmost.startswith('xn--') or hostname.startswith('xn--'): # RFC 6125, section 6.4.3, subitem 3. # The client SHOULD NOT attempt to match a presented identifier # where the wildcard character is embedded within an A-label or # U-label of an internationalized domain name. pats.append(re.escape(leftmost)) else: # Otherwise, '*' matches any dotless string, e.g. www* pats.append(re.escape(leftmost).replace(r'\*', '[^.]*')) # add the remaining fragments, ignore any wildcards for frag in parts[1:]: pats.append(re.escape(frag)) pat = re.compile(r'\A' + r'\.'.join(pats) + r'\Z', re.IGNORECASE) return pat.match(hostname) def match_hostname(cert, hostname): """Verify that *cert* (in decoded format as returned by SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 and RFC 6125 rules are followed, but IP addresses are not accepted for *hostname*. CertificateError is raised on failure. On success, the function returns nothing. """ if not cert: raise ValueError("empty or no certificate") dnsnames = [] san = cert.get('subjectAltName', ()) for key, value in san: if key == 'DNS': if _dnsname_match(value, hostname): return dnsnames.append(value) if not dnsnames: # The subject is only checked when there is no dNSName entry # in subjectAltName for sub in cert.get('subject', ()): for key, value in sub: # XXX according to RFC 2818, the most specific Common Name # must be used. if key == 'commonName': if _dnsname_match(value, hostname): return dnsnames.append(value) if len(dnsnames) > 1: raise CertificateError("hostname %r " "doesn't match either of %s" % (hostname, ', '.join(map(repr, dnsnames)))) elif len(dnsnames) == 1: raise CertificateError("hostname %r " "doesn't match %r" % (hostname, dnsnames[0])) else: raise CertificateError("no appropriate commonName or " "subjectAltName fields were found") transifex-client-0.10/txclib/packages/urllib3/packages/ordered_dict.py0000644000175000001440000002135012247321676026754 0ustar mpessasusers00000000000000# Backport of OrderedDict() class that runs on Python 2.4, 2.5, 2.6, 2.7 and pypy. # Passes Python2.7's test suite and incorporates all the latest updates. # Copyright 2009 Raymond Hettinger, released under the MIT License. # http://code.activestate.com/recipes/576693/ try: from thread import get_ident as _get_ident except ImportError: from dummy_thread import get_ident as _get_ident try: from _abcoll import KeysView, ValuesView, ItemsView except ImportError: pass class OrderedDict(dict): 'Dictionary that remembers insertion order' # An inherited dict maps keys to values. # The inherited dict provides __getitem__, __len__, __contains__, and get. # The remaining methods are order-aware. # Big-O running times for all methods are the same as for regular dictionaries. # The internal self.__map dictionary maps keys to links in a doubly linked list. # The circular doubly linked list starts and ends with a sentinel element. # The sentinel element never gets deleted (this simplifies the algorithm). # Each link is stored as a list of length three: [PREV, NEXT, KEY]. def __init__(self, *args, **kwds): '''Initialize an ordered dictionary. Signature is the same as for regular dictionaries, but keyword arguments are not recommended because their insertion order is arbitrary. ''' if len(args) > 1: raise TypeError('expected at most 1 arguments, got %d' % len(args)) try: self.__root except AttributeError: self.__root = root = [] # sentinel node root[:] = [root, root, None] self.__map = {} self.__update(*args, **kwds) def __setitem__(self, key, value, dict_setitem=dict.__setitem__): 'od.__setitem__(i, y) <==> od[i]=y' # Setting a new item creates a new link which goes at the end of the linked # list, and the inherited dictionary is updated with the new key/value pair. if key not in self: root = self.__root last = root[0] last[1] = root[0] = self.__map[key] = [last, root, key] dict_setitem(self, key, value) def __delitem__(self, key, dict_delitem=dict.__delitem__): 'od.__delitem__(y) <==> del od[y]' # Deleting an existing item uses self.__map to find the link which is # then removed by updating the links in the predecessor and successor nodes. dict_delitem(self, key) link_prev, link_next, key = self.__map.pop(key) link_prev[1] = link_next link_next[0] = link_prev def __iter__(self): 'od.__iter__() <==> iter(od)' root = self.__root curr = root[1] while curr is not root: yield curr[2] curr = curr[1] def __reversed__(self): 'od.__reversed__() <==> reversed(od)' root = self.__root curr = root[0] while curr is not root: yield curr[2] curr = curr[0] def clear(self): 'od.clear() -> None. Remove all items from od.' try: for node in self.__map.itervalues(): del node[:] root = self.__root root[:] = [root, root, None] self.__map.clear() except AttributeError: pass dict.clear(self) def popitem(self, last=True): '''od.popitem() -> (k, v), return and remove a (key, value) pair. Pairs are returned in LIFO order if last is true or FIFO order if false. ''' if not self: raise KeyError('dictionary is empty') root = self.__root if last: link = root[0] link_prev = link[0] link_prev[1] = root root[0] = link_prev else: link = root[1] link_next = link[1] root[1] = link_next link_next[0] = root key = link[2] del self.__map[key] value = dict.pop(self, key) return key, value # -- the following methods do not depend on the internal structure -- def keys(self): 'od.keys() -> list of keys in od' return list(self) def values(self): 'od.values() -> list of values in od' return [self[key] for key in self] def items(self): 'od.items() -> list of (key, value) pairs in od' return [(key, self[key]) for key in self] def iterkeys(self): 'od.iterkeys() -> an iterator over the keys in od' return iter(self) def itervalues(self): 'od.itervalues -> an iterator over the values in od' for k in self: yield self[k] def iteritems(self): 'od.iteritems -> an iterator over the (key, value) items in od' for k in self: yield (k, self[k]) def update(*args, **kwds): '''od.update(E, **F) -> None. Update od from dict/iterable E and F. If E is a dict instance, does: for k in E: od[k] = E[k] If E has a .keys() method, does: for k in E.keys(): od[k] = E[k] Or if E is an iterable of items, does: for k, v in E: od[k] = v In either case, this is followed by: for k, v in F.items(): od[k] = v ''' if len(args) > 2: raise TypeError('update() takes at most 2 positional ' 'arguments (%d given)' % (len(args),)) elif not args: raise TypeError('update() takes at least 1 argument (0 given)') self = args[0] # Make progressively weaker assumptions about "other" other = () if len(args) == 2: other = args[1] if isinstance(other, dict): for key in other: self[key] = other[key] elif hasattr(other, 'keys'): for key in other.keys(): self[key] = other[key] else: for key, value in other: self[key] = value for key, value in kwds.items(): self[key] = value __update = update # let subclasses override update without breaking __init__ __marker = object() def pop(self, key, default=__marker): '''od.pop(k[,d]) -> v, remove specified key and return the corresponding value. If key is not found, d is returned if given, otherwise KeyError is raised. ''' if key in self: result = self[key] del self[key] return result if default is self.__marker: raise KeyError(key) return default def setdefault(self, key, default=None): 'od.setdefault(k[,d]) -> od.get(k,d), also set od[k]=d if k not in od' if key in self: return self[key] self[key] = default return default def __repr__(self, _repr_running={}): 'od.__repr__() <==> repr(od)' call_key = id(self), _get_ident() if call_key in _repr_running: return '...' _repr_running[call_key] = 1 try: if not self: return '%s()' % (self.__class__.__name__,) return '%s(%r)' % (self.__class__.__name__, self.items()) finally: del _repr_running[call_key] def __reduce__(self): 'Return state information for pickling' items = [[k, self[k]] for k in self] inst_dict = vars(self).copy() for k in vars(OrderedDict()): inst_dict.pop(k, None) if inst_dict: return (self.__class__, (items,), inst_dict) return self.__class__, (items,) def copy(self): 'od.copy() -> a shallow copy of od' return self.__class__(self) @classmethod def fromkeys(cls, iterable, value=None): '''OD.fromkeys(S[, v]) -> New ordered dictionary with keys from S and values equal to v (which defaults to None). ''' d = cls() for key in iterable: d[key] = value return d def __eq__(self, other): '''od.__eq__(y) <==> od==y. Comparison to another OD is order-sensitive while comparison to a regular mapping is order-insensitive. ''' if isinstance(other, OrderedDict): return len(self)==len(other) and self.items() == other.items() return dict.__eq__(self, other) def __ne__(self, other): return not self == other # -- the following methods are only used in Python 2.7 -- def viewkeys(self): "od.viewkeys() -> a set-like object providing a view on od's keys" return KeysView(self) def viewvalues(self): "od.viewvalues() -> an object providing a view on od's values" return ValuesView(self) def viewitems(self): "od.viewitems() -> a set-like object providing a view on od's items" return ItemsView(self) transifex-client-0.10/txclib/packages/urllib3/packages/six.py0000644000175000001440000002655412247321676025143 0ustar mpessasusers00000000000000"""Utilities for writing code that runs on Python 2 and 3""" #Copyright (c) 2010-2011 Benjamin Peterson #Permission is hereby granted, free of charge, to any person obtaining a copy of #this software and associated documentation files (the "Software"), to deal in #the Software without restriction, including without limitation the rights to #use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of #the Software, and to permit persons to whom the Software is furnished to do so, #subject to the following conditions: #The above copyright notice and this permission notice shall be included in all #copies or substantial portions of the Software. #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR #IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS #FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR #COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER #IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN #CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. import operator import sys import types __author__ = "Benjamin Peterson " __version__ = "1.2.0" # Revision 41c74fef2ded # True if we are running on Python 3. PY3 = sys.version_info[0] == 3 if PY3: string_types = str, integer_types = int, class_types = type, text_type = str binary_type = bytes MAXSIZE = sys.maxsize else: string_types = basestring, integer_types = (int, long) class_types = (type, types.ClassType) text_type = unicode binary_type = str if sys.platform.startswith("java"): # Jython always uses 32 bits. MAXSIZE = int((1 << 31) - 1) else: # It's possible to have sizeof(long) != sizeof(Py_ssize_t). class X(object): def __len__(self): return 1 << 31 try: len(X()) except OverflowError: # 32-bit MAXSIZE = int((1 << 31) - 1) else: # 64-bit MAXSIZE = int((1 << 63) - 1) del X def _add_doc(func, doc): """Add documentation to a function.""" func.__doc__ = doc def _import_module(name): """Import module, returning the module after the last dot.""" __import__(name) return sys.modules[name] class _LazyDescr(object): def __init__(self, name): self.name = name def __get__(self, obj, tp): result = self._resolve() setattr(obj, self.name, result) # This is a bit ugly, but it avoids running this again. delattr(tp, self.name) return result class MovedModule(_LazyDescr): def __init__(self, name, old, new=None): super(MovedModule, self).__init__(name) if PY3: if new is None: new = name self.mod = new else: self.mod = old def _resolve(self): return _import_module(self.mod) class MovedAttribute(_LazyDescr): def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): super(MovedAttribute, self).__init__(name) if PY3: if new_mod is None: new_mod = name self.mod = new_mod if new_attr is None: if old_attr is None: new_attr = name else: new_attr = old_attr self.attr = new_attr else: self.mod = old_mod if old_attr is None: old_attr = name self.attr = old_attr def _resolve(self): module = _import_module(self.mod) return getattr(module, self.attr) class _MovedItems(types.ModuleType): """Lazy loading of moved objects""" _moved_attributes = [ MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), MovedAttribute("map", "itertools", "builtins", "imap", "map"), MovedAttribute("reload_module", "__builtin__", "imp", "reload"), MovedAttribute("reduce", "__builtin__", "functools"), MovedAttribute("StringIO", "StringIO", "io"), MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), MovedModule("builtins", "__builtin__"), MovedModule("configparser", "ConfigParser"), MovedModule("copyreg", "copy_reg"), MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), MovedModule("http_cookies", "Cookie", "http.cookies"), MovedModule("html_entities", "htmlentitydefs", "html.entities"), MovedModule("html_parser", "HTMLParser", "html.parser"), MovedModule("http_client", "httplib", "http.client"), MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), MovedModule("cPickle", "cPickle", "pickle"), MovedModule("queue", "Queue"), MovedModule("reprlib", "repr"), MovedModule("socketserver", "SocketServer"), MovedModule("tkinter", "Tkinter"), MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), MovedModule("tkinter_tix", "Tix", "tkinter.tix"), MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), MovedModule("tkinter_colorchooser", "tkColorChooser", "tkinter.colorchooser"), MovedModule("tkinter_commondialog", "tkCommonDialog", "tkinter.commondialog"), MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), MovedModule("tkinter_font", "tkFont", "tkinter.font"), MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", "tkinter.simpledialog"), MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), MovedModule("winreg", "_winreg"), ] for attr in _moved_attributes: setattr(_MovedItems, attr.name, attr) del attr moves = sys.modules[__name__ + ".moves"] = _MovedItems("moves") def add_move(move): """Add an item to six.moves.""" setattr(_MovedItems, move.name, move) def remove_move(name): """Remove item from six.moves.""" try: delattr(_MovedItems, name) except AttributeError: try: del moves.__dict__[name] except KeyError: raise AttributeError("no such move, %r" % (name,)) if PY3: _meth_func = "__func__" _meth_self = "__self__" _func_code = "__code__" _func_defaults = "__defaults__" _iterkeys = "keys" _itervalues = "values" _iteritems = "items" else: _meth_func = "im_func" _meth_self = "im_self" _func_code = "func_code" _func_defaults = "func_defaults" _iterkeys = "iterkeys" _itervalues = "itervalues" _iteritems = "iteritems" try: advance_iterator = next except NameError: def advance_iterator(it): return it.next() next = advance_iterator if PY3: def get_unbound_function(unbound): return unbound Iterator = object def callable(obj): return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) else: def get_unbound_function(unbound): return unbound.im_func class Iterator(object): def next(self): return type(self).__next__(self) callable = callable _add_doc(get_unbound_function, """Get the function out of a possibly unbound function""") get_method_function = operator.attrgetter(_meth_func) get_method_self = operator.attrgetter(_meth_self) get_function_code = operator.attrgetter(_func_code) get_function_defaults = operator.attrgetter(_func_defaults) def iterkeys(d): """Return an iterator over the keys of a dictionary.""" return iter(getattr(d, _iterkeys)()) def itervalues(d): """Return an iterator over the values of a dictionary.""" return iter(getattr(d, _itervalues)()) def iteritems(d): """Return an iterator over the (key, value) pairs of a dictionary.""" return iter(getattr(d, _iteritems)()) if PY3: def b(s): return s.encode("latin-1") def u(s): return s if sys.version_info[1] <= 1: def int2byte(i): return bytes((i,)) else: # This is about 2x faster than the implementation above on 3.2+ int2byte = operator.methodcaller("to_bytes", 1, "big") import io StringIO = io.StringIO BytesIO = io.BytesIO else: def b(s): return s def u(s): return unicode(s, "unicode_escape") int2byte = chr import StringIO StringIO = BytesIO = StringIO.StringIO _add_doc(b, """Byte literal""") _add_doc(u, """Text literal""") if PY3: import builtins exec_ = getattr(builtins, "exec") def reraise(tp, value, tb=None): if value.__traceback__ is not tb: raise value.with_traceback(tb) raise value print_ = getattr(builtins, "print") del builtins else: def exec_(code, globs=None, locs=None): """Execute code in a namespace.""" if globs is None: frame = sys._getframe(1) globs = frame.f_globals if locs is None: locs = frame.f_locals del frame elif locs is None: locs = globs exec("""exec code in globs, locs""") exec_("""def reraise(tp, value, tb=None): raise tp, value, tb """) def print_(*args, **kwargs): """The new-style print function.""" fp = kwargs.pop("file", sys.stdout) if fp is None: return def write(data): if not isinstance(data, basestring): data = str(data) fp.write(data) want_unicode = False sep = kwargs.pop("sep", None) if sep is not None: if isinstance(sep, unicode): want_unicode = True elif not isinstance(sep, str): raise TypeError("sep must be None or a string") end = kwargs.pop("end", None) if end is not None: if isinstance(end, unicode): want_unicode = True elif not isinstance(end, str): raise TypeError("end must be None or a string") if kwargs: raise TypeError("invalid keyword arguments to print()") if not want_unicode: for arg in args: if isinstance(arg, unicode): want_unicode = True break if want_unicode: newline = unicode("\n") space = unicode(" ") else: newline = "\n" space = " " if sep is None: sep = space if end is None: end = newline for i, arg in enumerate(args): if i: write(sep) write(arg) write(end) _add_doc(reraise, """Reraise an exception.""") def with_metaclass(meta, base=object): """Create a base class with a metaclass.""" return meta("NewBase", (base,), {}) transifex-client-0.10/txclib/packages/urllib3/packages/__init__.py0000644000175000001440000000011212247321676026055 0ustar mpessasusers00000000000000from __future__ import absolute_import from . import ssl_match_hostname transifex-client-0.10/txclib/packages/urllib3/exceptions.py0000644000175000001440000000631212247321676024731 0ustar mpessasusers00000000000000# urllib3/exceptions.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php ## Base Exceptions class HTTPError(Exception): "Base exception used by this module." pass class PoolError(HTTPError): "Base exception for errors caused within a pool." def __init__(self, pool, message): self.pool = pool HTTPError.__init__(self, "%s: %s" % (pool, message)) def __reduce__(self): # For pickling purposes. return self.__class__, (None, None) class RequestError(PoolError): "Base exception for PoolErrors that have associated URLs." def __init__(self, pool, url, message): self.url = url PoolError.__init__(self, pool, message) def __reduce__(self): # For pickling purposes. return self.__class__, (None, self.url, None) class SSLError(HTTPError): "Raised when SSL certificate fails in an HTTPS connection." pass class ProxyError(HTTPError): "Raised when the connection to a proxy fails." pass class DecodeError(HTTPError): "Raised when automatic decoding based on Content-Type fails." pass ## Leaf Exceptions class MaxRetryError(RequestError): "Raised when the maximum number of retries is exceeded." def __init__(self, pool, url, reason=None): self.reason = reason message = "Max retries exceeded with url: %s" % url if reason: message += " (Caused by %s: %s)" % (type(reason), reason) else: message += " (Caused by redirect)" RequestError.__init__(self, pool, url, message) class HostChangedError(RequestError): "Raised when an existing pool gets a request for a foreign host." def __init__(self, pool, url, retries=3): message = "Tried to open a foreign host with url: %s" % url RequestError.__init__(self, pool, url, message) self.retries = retries class TimeoutStateError(HTTPError): """ Raised when passing an invalid state to a timeout """ pass class TimeoutError(HTTPError): """ Raised when a socket timeout error occurs. Catching this error will catch both :exc:`ReadTimeoutErrors ` and :exc:`ConnectTimeoutErrors `. """ pass class ReadTimeoutError(TimeoutError, RequestError): "Raised when a socket timeout occurs while receiving data from a server" pass # This timeout error does not have a URL attached and needs to inherit from the # base HTTPError class ConnectTimeoutError(TimeoutError): "Raised when a socket timeout occurs while connecting to a server" pass class EmptyPoolError(PoolError): "Raised when a pool runs out of connections and no more are allowed." pass class ClosedPoolError(PoolError): "Raised when a request enters a pool after the pool has been closed." pass class LocationParseError(ValueError, HTTPError): "Raised when get_host or similar fails to parse the URL input." def __init__(self, location): message = "Failed to parse: %s" % location HTTPError.__init__(self, message) self.location = location transifex-client-0.10/txclib/packages/urllib3/contrib/0000755000175000001440000000000012247615466023637 5ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/contrib/ntlmpool.py0000644000175000001440000001120512247321676026051 0ustar mpessasusers00000000000000# urllib3/contrib/ntlmpool.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php """ NTLM authenticating pool, contributed by erikcederstran Issue #10, see: http://code.google.com/p/urllib3/issues/detail?id=10 """ try: from http.client import HTTPSConnection except ImportError: from httplib import HTTPSConnection from logging import getLogger from ntlm import ntlm from urllib3 import HTTPSConnectionPool log = getLogger(__name__) class NTLMConnectionPool(HTTPSConnectionPool): """ Implements an NTLM authentication version of an urllib3 connection pool """ scheme = 'https' def __init__(self, user, pw, authurl, *args, **kwargs): """ authurl is a random URL on the server that is protected by NTLM. user is the Windows user, probably in the DOMAIN\\username format. pw is the password for the user. """ super(NTLMConnectionPool, self).__init__(*args, **kwargs) self.authurl = authurl self.rawuser = user user_parts = user.split('\\', 1) self.domain = user_parts[0].upper() self.user = user_parts[1] self.pw = pw def _new_conn(self): # Performs the NTLM handshake that secures the connection. The socket # must be kept open while requests are performed. self.num_connections += 1 log.debug('Starting NTLM HTTPS connection no. %d: https://%s%s' % (self.num_connections, self.host, self.authurl)) headers = {} headers['Connection'] = 'Keep-Alive' req_header = 'Authorization' resp_header = 'www-authenticate' conn = HTTPSConnection(host=self.host, port=self.port) # Send negotiation message headers[req_header] = ( 'NTLM %s' % ntlm.create_NTLM_NEGOTIATE_MESSAGE(self.rawuser)) log.debug('Request headers: %s' % headers) conn.request('GET', self.authurl, None, headers) res = conn.getresponse() reshdr = dict(res.getheaders()) log.debug('Response status: %s %s' % (res.status, res.reason)) log.debug('Response headers: %s' % reshdr) log.debug('Response data: %s [...]' % res.read(100)) # Remove the reference to the socket, so that it can not be closed by # the response object (we want to keep the socket open) res.fp = None # Server should respond with a challenge message auth_header_values = reshdr[resp_header].split(', ') auth_header_value = None for s in auth_header_values: if s[:5] == 'NTLM ': auth_header_value = s[5:] if auth_header_value is None: raise Exception('Unexpected %s response header: %s' % (resp_header, reshdr[resp_header])) # Send authentication message ServerChallenge, NegotiateFlags = \ ntlm.parse_NTLM_CHALLENGE_MESSAGE(auth_header_value) auth_msg = ntlm.create_NTLM_AUTHENTICATE_MESSAGE(ServerChallenge, self.user, self.domain, self.pw, NegotiateFlags) headers[req_header] = 'NTLM %s' % auth_msg log.debug('Request headers: %s' % headers) conn.request('GET', self.authurl, None, headers) res = conn.getresponse() log.debug('Response status: %s %s' % (res.status, res.reason)) log.debug('Response headers: %s' % dict(res.getheaders())) log.debug('Response data: %s [...]' % res.read()[:100]) if res.status != 200: if res.status == 401: raise Exception('Server rejected request: wrong ' 'username or password') raise Exception('Wrong server response: %s %s' % (res.status, res.reason)) res.fp = None log.debug('Connection established') return conn def urlopen(self, method, url, body=None, headers=None, retries=3, redirect=True, assert_same_host=True): if headers is None: headers = {} headers['Connection'] = 'Keep-Alive' return super(NTLMConnectionPool, self).urlopen(method, url, body, headers, retries, redirect, assert_same_host) transifex-client-0.10/txclib/packages/urllib3/contrib/pyopenssl.py0000644000175000001440000002734412247321676026254 0ustar mpessasusers00000000000000'''SSL with SNI-support for Python 2. This needs the following packages installed: * pyOpenSSL (tested with 0.13) * ndg-httpsclient (tested with 0.3.2) * pyasn1 (tested with 0.1.6) To activate it call :func:`~urllib3.contrib.pyopenssl.inject_into_urllib3`. This can be done in a ``sitecustomize`` module, or at any other time before your application begins using ``urllib3``, like this:: try: import urllib3.contrib.pyopenssl urllib3.contrib.pyopenssl.inject_into_urllib3() except ImportError: pass Now you can use :mod:`urllib3` as you normally would, and it will support SNI when the required modules are installed. ''' from ndg.httpsclient.ssl_peer_verification import SUBJ_ALT_NAME_SUPPORT from ndg.httpsclient.subj_alt_name import SubjectAltName import OpenSSL.SSL from pyasn1.codec.der import decoder as der_decoder from socket import _fileobject import ssl from cStringIO import StringIO from .. import connectionpool from .. import util __all__ = ['inject_into_urllib3', 'extract_from_urllib3'] # SNI only *really* works if we can read the subjectAltName of certificates. HAS_SNI = SUBJ_ALT_NAME_SUPPORT # Map from urllib3 to PyOpenSSL compatible parameter-values. _openssl_versions = { ssl.PROTOCOL_SSLv23: OpenSSL.SSL.SSLv23_METHOD, ssl.PROTOCOL_SSLv3: OpenSSL.SSL.SSLv3_METHOD, ssl.PROTOCOL_TLSv1: OpenSSL.SSL.TLSv1_METHOD, } _openssl_verify = { ssl.CERT_NONE: OpenSSL.SSL.VERIFY_NONE, ssl.CERT_OPTIONAL: OpenSSL.SSL.VERIFY_PEER, ssl.CERT_REQUIRED: OpenSSL.SSL.VERIFY_PEER + OpenSSL.SSL.VERIFY_FAIL_IF_NO_PEER_CERT, } orig_util_HAS_SNI = util.HAS_SNI orig_connectionpool_ssl_wrap_socket = connectionpool.ssl_wrap_socket def inject_into_urllib3(): 'Monkey-patch urllib3 with PyOpenSSL-backed SSL-support.' connectionpool.ssl_wrap_socket = ssl_wrap_socket util.HAS_SNI = HAS_SNI def extract_from_urllib3(): 'Undo monkey-patching by :func:`inject_into_urllib3`.' connectionpool.ssl_wrap_socket = orig_connectionpool_ssl_wrap_socket util.HAS_SNI = orig_util_HAS_SNI ### Note: This is a slightly bug-fixed version of same from ndg-httpsclient. def get_subj_alt_name(peer_cert): # Search through extensions dns_name = [] if not SUBJ_ALT_NAME_SUPPORT: return dns_name general_names = SubjectAltName() for i in range(peer_cert.get_extension_count()): ext = peer_cert.get_extension(i) ext_name = ext.get_short_name() if ext_name != 'subjectAltName': continue # PyOpenSSL returns extension data in ASN.1 encoded form ext_dat = ext.get_data() decoded_dat = der_decoder.decode(ext_dat, asn1Spec=general_names) for name in decoded_dat: if not isinstance(name, SubjectAltName): continue for entry in range(len(name)): component = name.getComponentByPosition(entry) if component.getName() != 'dNSName': continue dns_name.append(str(component.getComponent())) return dns_name class fileobject(_fileobject): def read(self, size=-1): # Use max, disallow tiny reads in a loop as they are very inefficient. # We never leave read() with any leftover data from a new recv() call # in our internal buffer. rbufsize = max(self._rbufsize, self.default_bufsize) # Our use of StringIO rather than lists of string objects returned by # recv() minimizes memory usage and fragmentation that occurs when # rbufsize is large compared to the typical return value of recv(). buf = self._rbuf buf.seek(0, 2) # seek end if size < 0: # Read until EOF self._rbuf = StringIO() # reset _rbuf. we consume it via buf. while True: try: data = self._sock.recv(rbufsize) except OpenSSL.SSL.WantReadError: continue if not data: break buf.write(data) return buf.getvalue() else: # Read until size bytes or EOF seen, whichever comes first buf_len = buf.tell() if buf_len >= size: # Already have size bytes in our buffer? Extract and return. buf.seek(0) rv = buf.read(size) self._rbuf = StringIO() self._rbuf.write(buf.read()) return rv self._rbuf = StringIO() # reset _rbuf. we consume it via buf. while True: left = size - buf_len # recv() will malloc the amount of memory given as its # parameter even though it often returns much less data # than that. The returned data string is short lived # as we copy it into a StringIO and free it. This avoids # fragmentation issues on many platforms. try: data = self._sock.recv(left) except OpenSSL.SSL.WantReadError: continue if not data: break n = len(data) if n == size and not buf_len: # Shortcut. Avoid buffer data copies when: # - We have no data in our buffer. # AND # - Our call to recv returned exactly the # number of bytes we were asked to read. return data if n == left: buf.write(data) del data # explicit free break assert n <= left, "recv(%d) returned %d bytes" % (left, n) buf.write(data) buf_len += n del data # explicit free #assert buf_len == buf.tell() return buf.getvalue() def readline(self, size=-1): buf = self._rbuf buf.seek(0, 2) # seek end if buf.tell() > 0: # check if we already have it in our buffer buf.seek(0) bline = buf.readline(size) if bline.endswith('\n') or len(bline) == size: self._rbuf = StringIO() self._rbuf.write(buf.read()) return bline del bline if size < 0: # Read until \n or EOF, whichever comes first if self._rbufsize <= 1: # Speed up unbuffered case buf.seek(0) buffers = [buf.read()] self._rbuf = StringIO() # reset _rbuf. we consume it via buf. data = None recv = self._sock.recv while True: try: while data != "\n": data = recv(1) if not data: break buffers.append(data) except OpenSSL.SSL.WantReadError: continue break return "".join(buffers) buf.seek(0, 2) # seek end self._rbuf = StringIO() # reset _rbuf. we consume it via buf. while True: try: data = self._sock.recv(self._rbufsize) except OpenSSL.SSL.WantReadError: continue if not data: break nl = data.find('\n') if nl >= 0: nl += 1 buf.write(data[:nl]) self._rbuf.write(data[nl:]) del data break buf.write(data) return buf.getvalue() else: # Read until size bytes or \n or EOF seen, whichever comes first buf.seek(0, 2) # seek end buf_len = buf.tell() if buf_len >= size: buf.seek(0) rv = buf.read(size) self._rbuf = StringIO() self._rbuf.write(buf.read()) return rv self._rbuf = StringIO() # reset _rbuf. we consume it via buf. while True: try: data = self._sock.recv(self._rbufsize) except OpenSSL.SSL.WantReadError: continue if not data: break left = size - buf_len # did we just receive a newline? nl = data.find('\n', 0, left) if nl >= 0: nl += 1 # save the excess data to _rbuf self._rbuf.write(data[nl:]) if buf_len: buf.write(data[:nl]) break else: # Shortcut. Avoid data copy through buf when returning # a substring of our first recv(). return data[:nl] n = len(data) if n == size and not buf_len: # Shortcut. Avoid data copy through buf when # returning exactly all of our first recv(). return data if n >= left: buf.write(data[:left]) self._rbuf.write(data[left:]) break buf.write(data) buf_len += n #assert buf_len == buf.tell() return buf.getvalue() class WrappedSocket(object): '''API-compatibility wrapper for Python OpenSSL's Connection-class.''' def __init__(self, connection, socket): self.connection = connection self.socket = socket def fileno(self): return self.socket.fileno() def makefile(self, mode, bufsize=-1): return fileobject(self.connection, mode, bufsize) def settimeout(self, timeout): return self.socket.settimeout(timeout) def sendall(self, data): return self.connection.sendall(data) def close(self): return self.connection.shutdown() def getpeercert(self, binary_form=False): x509 = self.connection.get_peer_certificate() if not x509: return x509 if binary_form: return OpenSSL.crypto.dump_certificate( OpenSSL.crypto.FILETYPE_ASN1, x509) return { 'subject': ( (('commonName', x509.get_subject().CN),), ), 'subjectAltName': [ ('DNS', value) for value in get_subj_alt_name(x509) ] } def _verify_callback(cnx, x509, err_no, err_depth, return_code): return err_no == 0 def ssl_wrap_socket(sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None): ctx = OpenSSL.SSL.Context(_openssl_versions[ssl_version]) if certfile: ctx.use_certificate_file(certfile) if keyfile: ctx.use_privatekey_file(keyfile) if cert_reqs != ssl.CERT_NONE: ctx.set_verify(_openssl_verify[cert_reqs], _verify_callback) if ca_certs: try: ctx.load_verify_locations(ca_certs, None) except OpenSSL.SSL.Error as e: raise ssl.SSLError('bad ca_certs: %r' % ca_certs, e) cnx = OpenSSL.SSL.Connection(ctx, sock) cnx.set_tlsext_host_name(server_hostname) cnx.set_connect_state() while True: try: cnx.do_handshake() except OpenSSL.SSL.WantReadError: continue except OpenSSL.SSL.Error as e: raise ssl.SSLError('bad handshake', e) break return WrappedSocket(cnx, sock) transifex-client-0.10/txclib/packages/urllib3/contrib/__init__.py0000644000175000001440000000000012247321676025733 0ustar mpessasusers00000000000000transifex-client-0.10/txclib/packages/urllib3/connectionpool.py0000644000175000001440000006607412247321676025614 0ustar mpessasusers00000000000000# urllib3/connectionpool.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php import errno import logging from socket import error as SocketError, timeout as SocketTimeout import socket try: # Python 3 from http.client import HTTPConnection, HTTPException from http.client import HTTP_PORT, HTTPS_PORT except ImportError: from httplib import HTTPConnection, HTTPException from httplib import HTTP_PORT, HTTPS_PORT try: # Python 3 from queue import LifoQueue, Empty, Full except ImportError: from Queue import LifoQueue, Empty, Full import Queue as _ # Platform-specific: Windows try: # Compiled with SSL? HTTPSConnection = object class BaseSSLError(BaseException): pass ssl = None try: # Python 3 from http.client import HTTPSConnection except ImportError: from httplib import HTTPSConnection import ssl BaseSSLError = ssl.SSLError except (ImportError, AttributeError): # Platform-specific: No SSL. pass from .exceptions import ( ClosedPoolError, ConnectTimeoutError, EmptyPoolError, HostChangedError, MaxRetryError, SSLError, ReadTimeoutError, ProxyError, ) from .packages.ssl_match_hostname import CertificateError, match_hostname from .packages import six from .request import RequestMethods from .response import HTTPResponse from .util import ( assert_fingerprint, get_host, is_connection_dropped, resolve_cert_reqs, resolve_ssl_version, ssl_wrap_socket, Timeout, ) xrange = six.moves.xrange log = logging.getLogger(__name__) _Default = object() port_by_scheme = { 'http': HTTP_PORT, 'https': HTTPS_PORT, } ## Connection objects (extension of httplib) class VerifiedHTTPSConnection(HTTPSConnection): """ Based on httplib.HTTPSConnection but wraps the socket with SSL certification. """ cert_reqs = None ca_certs = None ssl_version = None def set_cert(self, key_file=None, cert_file=None, cert_reqs=None, ca_certs=None, assert_hostname=None, assert_fingerprint=None): self.key_file = key_file self.cert_file = cert_file self.cert_reqs = cert_reqs self.ca_certs = ca_certs self.assert_hostname = assert_hostname self.assert_fingerprint = assert_fingerprint def connect(self): # Add certificate verification try: sock = socket.create_connection( address=(self.host, self.port), timeout=self.timeout) except SocketTimeout: raise ConnectTimeoutError( self, "Connection to %s timed out. (connect timeout=%s)" % (self.host, self.timeout)) resolved_cert_reqs = resolve_cert_reqs(self.cert_reqs) resolved_ssl_version = resolve_ssl_version(self.ssl_version) if self._tunnel_host: self.sock = sock # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Wrap socket using verification with the root certs in # trusted_root_certs self.sock = ssl_wrap_socket(sock, self.key_file, self.cert_file, cert_reqs=resolved_cert_reqs, ca_certs=self.ca_certs, server_hostname=self.host, ssl_version=resolved_ssl_version) if resolved_cert_reqs != ssl.CERT_NONE: if self.assert_fingerprint: assert_fingerprint(self.sock.getpeercert(binary_form=True), self.assert_fingerprint) elif self.assert_hostname is not False: match_hostname(self.sock.getpeercert(), self.assert_hostname or self.host) ## Pool objects class ConnectionPool(object): """ Base class for all connection pools, such as :class:`.HTTPConnectionPool` and :class:`.HTTPSConnectionPool`. """ scheme = None QueueCls = LifoQueue def __init__(self, host, port=None): # httplib doesn't like it when we include brackets in ipv6 addresses host = host.strip('[]') self.host = host self.port = port def __str__(self): return '%s(host=%r, port=%r)' % (type(self).__name__, self.host, self.port) # This is taken from http://hg.python.org/cpython/file/7aaba721ebc0/Lib/socket.py#l252 _blocking_errnos = set([errno.EAGAIN, errno.EWOULDBLOCK]) class HTTPConnectionPool(ConnectionPool, RequestMethods): """ Thread-safe connection pool for one host. :param host: Host used for this HTTP Connection (e.g. "localhost"), passed into :class:`httplib.HTTPConnection`. :param port: Port used for this HTTP Connection (None is equivalent to 80), passed into :class:`httplib.HTTPConnection`. :param strict: Causes BadStatusLine to be raised if the status line can't be parsed as a valid HTTP/1.0 or 1.1 status line, passed into :class:`httplib.HTTPConnection`. .. note:: Only works in Python 2. This parameter is ignored in Python 3. :param timeout: Socket timeout in seconds for each individual connection. This can be a float or integer, which sets the timeout for the HTTP request, or an instance of :class:`urllib3.util.Timeout` which gives you more fine-grained control over request timeouts. After the constructor has been parsed, this is always a `urllib3.util.Timeout` object. :param maxsize: Number of connections to save that can be reused. More than 1 is useful in multithreaded situations. If ``block`` is set to false, more connections will be created but they will not be saved once they've been used. :param block: If set to True, no more than ``maxsize`` connections will be used at a time. When no free connections are available, the call will block until a connection has been released. This is a useful side effect for particular multithreaded situations where one does not want to use more than maxsize connections per host to prevent flooding. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param _proxy: Parsed proxy URL, should not be used directly, instead, see :class:`urllib3.connectionpool.ProxyManager`" :param _proxy_headers: A dictionary with proxy headers, should not be used directly, instead, see :class:`urllib3.connectionpool.ProxyManager`" """ scheme = 'http' def __init__(self, host, port=None, strict=False, timeout=Timeout.DEFAULT_TIMEOUT, maxsize=1, block=False, headers=None, _proxy=None, _proxy_headers=None): ConnectionPool.__init__(self, host, port) RequestMethods.__init__(self, headers) self.strict = strict # This is for backwards compatibility and can be removed once a timeout # can only be set to a Timeout object if not isinstance(timeout, Timeout): timeout = Timeout.from_float(timeout) self.timeout = timeout self.pool = self.QueueCls(maxsize) self.block = block self.proxy = _proxy self.proxy_headers = _proxy_headers or {} # Fill the queue up so that doing get() on it will block properly for _ in xrange(maxsize): self.pool.put(None) # These are mostly for testing and debugging purposes. self.num_connections = 0 self.num_requests = 0 def _new_conn(self): """ Return a fresh :class:`httplib.HTTPConnection`. """ self.num_connections += 1 log.info("Starting new HTTP connection (%d): %s" % (self.num_connections, self.host)) extra_params = {} if not six.PY3: # Python 2 extra_params['strict'] = self.strict return HTTPConnection(host=self.host, port=self.port, timeout=self.timeout.connect_timeout, **extra_params) def _get_conn(self, timeout=None): """ Get a connection. Will return a pooled connection if one is available. If no connections are available and :prop:`.block` is ``False``, then a fresh connection is returned. :param timeout: Seconds to wait before giving up and raising :class:`urllib3.exceptions.EmptyPoolError` if the pool is empty and :prop:`.block` is ``True``. """ conn = None try: conn = self.pool.get(block=self.block, timeout=timeout) except AttributeError: # self.pool is None raise ClosedPoolError(self, "Pool is closed.") except Empty: if self.block: raise EmptyPoolError(self, "Pool reached maximum size and no more " "connections are allowed.") pass # Oh well, we'll create a new connection then # If this is a persistent connection, check if it got disconnected if conn and is_connection_dropped(conn): log.info("Resetting dropped connection: %s" % self.host) conn.close() return conn or self._new_conn() def _put_conn(self, conn): """ Put a connection back into the pool. :param conn: Connection object for the current host and port as returned by :meth:`._new_conn` or :meth:`._get_conn`. If the pool is already full, the connection is closed and discarded because we exceeded maxsize. If connections are discarded frequently, then maxsize should be increased. If the pool is closed, then the connection will be closed and discarded. """ try: self.pool.put(conn, block=False) return # Everything is dandy, done. except AttributeError: # self.pool is None. pass except Full: # This should never happen if self.block == True log.warning("HttpConnectionPool is full, discarding connection: %s" % self.host) # Connection never got put back into the pool, close it. if conn: conn.close() def _get_timeout(self, timeout): """ Helper that always returns a :class:`urllib3.util.Timeout` """ if timeout is _Default: return self.timeout.clone() if isinstance(timeout, Timeout): return timeout.clone() else: # User passed us an int/float. This is for backwards compatibility, # can be removed later return Timeout.from_float(timeout) def _make_request(self, conn, method, url, timeout=_Default, **httplib_request_kw): """ Perform a request on a given httplib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) try: timeout_obj.start_connect() conn.timeout = timeout_obj.connect_timeout # conn.request() calls httplib.*.request, not the method in # request.py. It also calls makefile (recv) on the socket conn.request(method, url, **httplib_request_kw) except SocketTimeout: raise ConnectTimeoutError( self, "Connection to %s timed out. (connect timeout=%s)" % (self.host, timeout_obj.connect_timeout)) # Reset the timeout for the recv() on the socket read_timeout = timeout_obj.read_timeout log.debug("Setting read timeout to %s" % read_timeout) # App Engine doesn't have a sock attr if hasattr(conn, 'sock') and \ read_timeout is not None and \ read_timeout is not Timeout.DEFAULT_TIMEOUT: # In Python 3 socket.py will catch EAGAIN and return None when you # try and read into the file pointer created by http.client, which # instead raises a BadStatusLine exception. Instead of catching # the exception and assuming all BadStatusLine exceptions are read # timeouts, check for a zero timeout before making the request. if read_timeout == 0: raise ReadTimeoutError( self, url, "Read timed out. (read timeout=%s)" % read_timeout) conn.sock.settimeout(read_timeout) # Receive the response from the server try: try: # Python 2.7+, use buffering of HTTP responses httplib_response = conn.getresponse(buffering=True) except TypeError: # Python 2.6 and older httplib_response = conn.getresponse() except SocketTimeout: raise ReadTimeoutError( self, url, "Read timed out. (read timeout=%s)" % read_timeout) except SocketError as e: # Platform-specific: Python 2 # See the above comment about EAGAIN in Python 3. In Python 2 we # have to specifically catch it and throw the timeout error if e.errno in _blocking_errnos: raise ReadTimeoutError( self, url, "Read timed out. (read timeout=%s)" % read_timeout) raise # AppEngine doesn't have a version attr. http_version = getattr(conn, '_http_vsn_str', 'HTTP/?') log.debug("\"%s %s %s\" %s %s" % (method, url, http_version, httplib_response.status, httplib_response.length)) return httplib_response def close(self): """ Close all pooled connections and disable the pool. """ # Disable access to the pool old_pool, self.pool = self.pool, None try: while True: conn = old_pool.get(block=False) if conn: conn.close() except Empty: pass # Done. def is_same_host(self, url): """ Check if the given ``url`` is a member of the same host as this connection pool. """ if url.startswith('/'): return True # TODO: Add optional support for socket.gethostbyname checking. scheme, host, port = get_host(url) if self.port and not port: # Use explicit default port for comparison when none is given. port = port_by_scheme.get(scheme) return (scheme, host, port) == (self.scheme, self.host, self.port) def urlopen(self, method, url, body=None, headers=None, retries=3, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, **response_kw): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param body: Data to send in the request body (useful for creating POST requests, see HTTPConnectionPool.post_url for more convenience). :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Number of retries to allow before raising a MaxRetryError exception. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When False, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param \**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ if headers is None: headers = self.headers if retries < 0: raise MaxRetryError(self, url) if release_conn is None: release_conn = response_kw.get('preload_content', True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries - 1) conn = None try: # Request a connection from the queue conn = self._get_conn(timeout=pool_timeout) # Make the request on the httplib connection object httplib_response = self._make_request(conn, method, url, timeout=timeout, body=body, headers=headers) # If we're going to release the connection in ``finally:``, then # the request doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = not release_conn and conn # Import httplib's response into our own wrapper object response = HTTPResponse.from_httplib(httplib_response, pool=self, connection=response_conn, **response_kw) # else: # The connection will be put back into the pool when # ``response.release_conn()`` is called (implicitly by # ``response.read()``) except Empty: # Timed out by queue raise ReadTimeoutError( self, url, "Read timed out, no pool connections are available.") except SocketTimeout: # Timed out by socket raise ReadTimeoutError(self, url, "Read timed out.") except BaseSSLError as e: # SSL certificate error if 'timed out' in str(e) or \ 'did not complete (read)' in str(e): # Platform-specific: Python 2.6 raise ReadTimeoutError(self, url, "Read timed out.") raise SSLError(e) except CertificateError as e: # Name mismatch raise SSLError(e) except (HTTPException, SocketError) as e: if isinstance(e, SocketError) and self.proxy is not None: raise ProxyError('Cannot connect to proxy. ' 'Socket error: %s.' % e) # Connection broken, discard. It will be replaced next _get_conn(). conn = None # This is necessary so we can access e below err = e if retries == 0: raise MaxRetryError(self, url, e) finally: if release_conn: # Put the connection back to be reused. If the connection is # expired then it will be None, which will get replaced with a # fresh connection during _get_conn. self._put_conn(conn) if not conn: # Try again log.warn("Retrying (%d attempts remain) after connection " "broken by '%r': %s" % (retries, err, url)) return self.urlopen(method, url, body, headers, retries - 1, redirect, assert_same_host, timeout=timeout, pool_timeout=pool_timeout, release_conn=release_conn, **response_kw) # Handle redirect? redirect_location = redirect and response.get_redirect_location() if redirect_location: if response.status == 303: method = 'GET' log.info("Redirecting %s -> %s" % (url, redirect_location)) return self.urlopen(method, redirect_location, body, headers, retries - 1, redirect, assert_same_host, timeout=timeout, pool_timeout=pool_timeout, release_conn=release_conn, **response_kw) return response class HTTPSConnectionPool(HTTPConnectionPool): """ Same as :class:`.HTTPConnectionPool`, but HTTPS. When Python is compiled with the :mod:`ssl` module, then :class:`.VerifiedHTTPSConnection` is used, which *can* verify certificates, instead of :class:`httplib.HTTPSConnection`. :class:`.VerifiedHTTPSConnection` uses one of ``assert_fingerprint``, ``assert_hostname`` and ``host`` in this order to verify connections. If ``assert_hostname`` is False, no verification is done. The ``key_file``, ``cert_file``, ``cert_reqs``, ``ca_certs`` and ``ssl_version`` are only used if :mod:`ssl` is available and are fed into :meth:`urllib3.util.ssl_wrap_socket` to upgrade the connection socket into an SSL socket. """ scheme = 'https' def __init__(self, host, port=None, strict=False, timeout=None, maxsize=1, block=False, headers=None, _proxy=None, _proxy_headers=None, key_file=None, cert_file=None, cert_reqs=None, ca_certs=None, ssl_version=None, assert_hostname=None, assert_fingerprint=None): HTTPConnectionPool.__init__(self, host, port, strict, timeout, maxsize, block, headers, _proxy, _proxy_headers) self.key_file = key_file self.cert_file = cert_file self.cert_reqs = cert_reqs self.ca_certs = ca_certs self.ssl_version = ssl_version self.assert_hostname = assert_hostname self.assert_fingerprint = assert_fingerprint def _prepare_conn(self, connection): """ Prepare the ``connection`` for :meth:`urllib3.util.ssl_wrap_socket` and establish the tunnel if proxy is used. """ if isinstance(connection, VerifiedHTTPSConnection): connection.set_cert(key_file=self.key_file, cert_file=self.cert_file, cert_reqs=self.cert_reqs, ca_certs=self.ca_certs, assert_hostname=self.assert_hostname, assert_fingerprint=self.assert_fingerprint) connection.ssl_version = self.ssl_version if self.proxy is not None: # Python 2.7+ try: set_tunnel = connection.set_tunnel except AttributeError: # Platform-specific: Python 2.6 set_tunnel = connection._set_tunnel set_tunnel(self.host, self.port, self.proxy_headers) # Establish tunnel connection early, because otherwise httplib # would improperly set Host: header to proxy's IP:port. connection.connect() return connection def _new_conn(self): """ Return a fresh :class:`httplib.HTTPSConnection`. """ self.num_connections += 1 log.info("Starting new HTTPS connection (%d): %s" % (self.num_connections, self.host)) actual_host = self.host actual_port = self.port if self.proxy is not None: actual_host = self.proxy.host actual_port = self.proxy.port if not ssl: # Platform-specific: Python compiled without +ssl if not HTTPSConnection or HTTPSConnection is object: raise SSLError("Can't connect to HTTPS URL because the SSL " "module is not available.") connection_class = HTTPSConnection else: connection_class = VerifiedHTTPSConnection extra_params = {} if not six.PY3: # Python 2 extra_params['strict'] = self.strict connection = connection_class(host=actual_host, port=actual_port, timeout=self.timeout.connect_timeout, **extra_params) return self._prepare_conn(connection) def connection_from_url(url, **kw): """ Given a url, return an :class:`.ConnectionPool` instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating an :class:`.ConnectionPool` instance. :param url: Absolute URL string that must include the scheme. Port is optional. :param \**kw: Passes additional parameters to the constructor of the appropriate :class:`.ConnectionPool`. Useful for specifying things like timeout, maxsize, headers, etc. Example: :: >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') """ scheme, host, port = get_host(url) if scheme == 'https': return HTTPSConnectionPool(host, port=port, **kw) else: return HTTPConnectionPool(host, port=port, **kw) transifex-client-0.10/txclib/packages/urllib3/fields.py0000644000175000001440000001353012247321676024016 0ustar mpessasusers00000000000000# urllib3/fields.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php import email.utils import mimetypes from .packages import six def guess_content_type(filename, default='application/octet-stream'): """ Guess the "Content-Type" of a file. :param filename: The filename to guess the "Content-Type" of using :mod:`mimetimes`. :param default: If no "Content-Type" can be guessed, default to `default`. """ if filename: return mimetypes.guess_type(filename)[0] or default return default def format_header_param(name, value): """ Helper function to format and quote a single header parameter. Particularly useful for header parameters which might contain non-ASCII values, like file names. This follows RFC 2231, as suggested by RFC 2388 Section 4.4. :param name: The name of the parameter, a string expected to be ASCII only. :param value: The value of the parameter, provided as a unicode string. """ if not any(ch in value for ch in '"\\\r\n'): result = '%s="%s"' % (name, value) try: result.encode('ascii') except UnicodeEncodeError: pass else: return result if not six.PY3: # Python 2: value = value.encode('utf-8') value = email.utils.encode_rfc2231(value, 'utf-8') value = '%s*=%s' % (name, value) return value class RequestField(object): """ A data container for request body parameters. :param name: The name of this request field. :param data: The data/value body. :param filename: An optional filename of the request field. :param headers: An optional dict-like object of headers to initially use for the field. """ def __init__(self, name, data, filename=None, headers=None): self._name = name self._filename = filename self.data = data self.headers = {} if headers: self.headers = dict(headers) @classmethod def from_tuples(cls, fieldname, value): """ A :class:`~urllib3.fields.RequestField` factory from old-style tuple parameters. Supports constructing :class:`~urllib3.fields.RequestField` from parameter of key/value strings AND key/filetuple. A filetuple is a (filename, data, MIME type) tuple where the MIME type is optional. For example: :: 'foo': 'bar', 'fakefile': ('foofile.txt', 'contents of foofile'), 'realfile': ('barfile.txt', open('realfile').read()), 'typedfile': ('bazfile.bin', open('bazfile').read(), 'image/jpeg'), 'nonamefile': 'contents of nonamefile field', Field names and filenames must be unicode. """ if isinstance(value, tuple): if len(value) == 3: filename, data, content_type = value else: filename, data = value content_type = guess_content_type(filename) else: filename = None content_type = None data = value request_param = cls(fieldname, data, filename=filename) request_param.make_multipart(content_type=content_type) return request_param def _render_part(self, name, value): """ Overridable helper function to format a single header parameter. :param name: The name of the parameter, a string expected to be ASCII only. :param value: The value of the parameter, provided as a unicode string. """ return format_header_param(name, value) def _render_parts(self, header_parts): """ Helper function to format and quote a single header. Useful for single headers that are composed of multiple items. E.g., 'Content-Disposition' fields. :param header_parts: A sequence of (k, v) typles or a :class:`dict` of (k, v) to format as `k1="v1"; k2="v2"; ...`. """ parts = [] iterable = header_parts if isinstance(header_parts, dict): iterable = header_parts.items() for name, value in iterable: if value: parts.append(self._render_part(name, value)) return '; '.join(parts) def render_headers(self): """ Renders the headers for this request field. """ lines = [] sort_keys = ['Content-Disposition', 'Content-Type', 'Content-Location'] for sort_key in sort_keys: if self.headers.get(sort_key, False): lines.append('%s: %s' % (sort_key, self.headers[sort_key])) for header_name, header_value in self.headers.items(): if header_name not in sort_keys: if header_value: lines.append('%s: %s' % (header_name, header_value)) lines.append('\r\n') return '\r\n'.join(lines) def make_multipart(self, content_disposition=None, content_type=None, content_location=None): """ Makes this request field into a multipart request field. This method overrides "Content-Disposition", "Content-Type" and "Content-Location" headers to the request parameter. :param content_type: The 'Content-Type' of the request body. :param content_location: The 'Content-Location' of the request body. """ self.headers['Content-Disposition'] = content_disposition or 'form-data' self.headers['Content-Disposition'] += '; '.join(['', self._render_parts((('name', self._name), ('filename', self._filename)))]) self.headers['Content-Type'] = content_type self.headers['Content-Location'] = content_location transifex-client-0.10/txclib/packages/urllib3/poolmanager.py0000644000175000001440000002146112247321676025056 0ustar mpessasusers00000000000000# urllib3/poolmanager.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php import logging try: # Python 3 from urllib.parse import urljoin except ImportError: from urlparse import urljoin from ._collections import RecentlyUsedContainer from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool from .connectionpool import port_by_scheme from .request import RequestMethods from .util import parse_url __all__ = ['PoolManager', 'ProxyManager', 'proxy_from_url'] pool_classes_by_scheme = { 'http': HTTPConnectionPool, 'https': HTTPSConnectionPool, } log = logging.getLogger(__name__) SSL_KEYWORDS = ('key_file', 'cert_file', 'cert_reqs', 'ca_certs', 'ssl_version') class PoolManager(RequestMethods): """ Allows for arbitrary requests while transparently keeping track of necessary connection pools for you. :param num_pools: Number of connection pools to cache before discarding the least recently used pool. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param \**connection_pool_kw: Additional parameters are used to create fresh :class:`urllib3.connectionpool.ConnectionPool` instances. Example: :: >>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 """ proxy = None def __init__(self, num_pools=10, headers=None, **connection_pool_kw): RequestMethods.__init__(self, headers) self.connection_pool_kw = connection_pool_kw self.pools = RecentlyUsedContainer(num_pools, dispose_func=lambda p: p.close()) def _new_pool(self, scheme, host, port): """ Create a new :class:`ConnectionPool` based on host, port and scheme. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. """ pool_cls = pool_classes_by_scheme[scheme] kwargs = self.connection_pool_kw if scheme == 'http': kwargs = self.connection_pool_kw.copy() for kw in SSL_KEYWORDS: kwargs.pop(kw, None) return pool_cls(host, port, **kwargs) def clear(self): """ Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. """ self.pools.clear() def connection_from_host(self, host, port=None, scheme='http'): """ Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. """ scheme = scheme or 'http' port = port or port_by_scheme.get(scheme, 80) pool_key = (scheme, host, port) with self.pools.lock: # If the scheme, host, or port doesn't match existing open # connections, open a new ConnectionPool. pool = self.pools.get(pool_key) if pool: return pool # Make a fresh ConnectionPool of the desired type pool = self._new_pool(scheme, host, port) self.pools[pool_key] = pool return pool def connection_from_url(self, url): """ Similar to :func:`urllib3.connectionpool.connection_from_url` but doesn't pass any additional parameters to the :class:`urllib3.connectionpool.ConnectionPool` constructor. Additional parameters are taken from the :class:`.PoolManager` constructor. """ u = parse_url(url) return self.connection_from_host(u.host, port=u.port, scheme=u.scheme) def urlopen(self, method, url, redirect=True, **kw): """ Same as :meth:`urllib3.connectionpool.HTTPConnectionPool.urlopen` with custom cross-host redirect logic and only sends the request-uri portion of the ``url``. The given ``url`` parameter must be absolute, such that an appropriate :class:`urllib3.connectionpool.ConnectionPool` can be chosen for it. """ u = parse_url(url) conn = self.connection_from_host(u.host, port=u.port, scheme=u.scheme) kw['assert_same_host'] = False kw['redirect'] = False if 'headers' not in kw: kw['headers'] = self.headers if self.proxy is not None and u.scheme == "http": response = conn.urlopen(method, url, **kw) else: response = conn.urlopen(method, u.request_uri, **kw) redirect_location = redirect and response.get_redirect_location() if not redirect_location: return response # Support relative URLs for redirecting. redirect_location = urljoin(url, redirect_location) # RFC 2616, Section 10.3.4 if response.status == 303: method = 'GET' log.info("Redirecting %s -> %s" % (url, redirect_location)) kw['retries'] = kw.get('retries', 3) - 1 # Persist retries countdown kw['redirect'] = redirect return self.urlopen(method, redirect_location, **kw) class ProxyManager(PoolManager): """ Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param poxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary contaning headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 """ def __init__(self, proxy_url, num_pools=10, headers=None, proxy_headers=None, **connection_pool_kw): if isinstance(proxy_url, HTTPConnectionPool): proxy_url = '%s://%s:%i' % (proxy_url.scheme, proxy_url.host, proxy_url.port) proxy = parse_url(proxy_url) if not proxy.port: port = port_by_scheme.get(proxy.scheme, 80) proxy = proxy._replace(port=port) self.proxy = proxy self.proxy_headers = proxy_headers or {} assert self.proxy.scheme in ("http", "https"), \ 'Not supported proxy scheme %s' % self.proxy.scheme connection_pool_kw['_proxy'] = self.proxy connection_pool_kw['_proxy_headers'] = self.proxy_headers super(ProxyManager, self).__init__( num_pools, headers, **connection_pool_kw) def connection_from_host(self, host, port=None, scheme='http'): if scheme == "https": return super(ProxyManager, self).connection_from_host( host, port, scheme) return super(ProxyManager, self).connection_from_host( self.proxy.host, self.proxy.port, self.proxy.scheme) def _set_proxy_headers(self, url, headers=None): """ Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. """ headers_ = {'Accept': '*/*'} netloc = parse_url(url).netloc if netloc: headers_['Host'] = netloc if headers: headers_.update(headers) return headers_ def urlopen(self, method, url, redirect=True, **kw): "Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute." u = parse_url(url) if u.scheme == "http": # It's too late to set proxy headers on per-request basis for # tunnelled HTTPS connections, should use # constructor's proxy_headers instead. kw['headers'] = self._set_proxy_headers(url, kw.get('headers', self.headers)) kw['headers'].update(self.proxy_headers) return super(ProxyManager, self).urlopen(method, url, redirect, **kw) def proxy_from_url(url, **kw): return ProxyManager(proxy_url=url, **kw) transifex-client-0.10/txclib/packages/urllib3/response.py0000644000175000001440000002367112247321676024415 0ustar mpessasusers00000000000000# urllib3/response.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php import logging import zlib import io from .exceptions import DecodeError from .packages.six import string_types as basestring, binary_type from .util import is_fp_closed log = logging.getLogger(__name__) class DeflateDecoder(object): def __init__(self): self._first_try = True self._data = binary_type() self._obj = zlib.decompressobj() def __getattr__(self, name): return getattr(self._obj, name) def decompress(self, data): if not self._first_try: return self._obj.decompress(data) self._data += data try: return self._obj.decompress(data) except zlib.error: self._first_try = False self._obj = zlib.decompressobj(-zlib.MAX_WBITS) try: return self.decompress(self._data) finally: self._data = None def _get_decoder(mode): if mode == 'gzip': return zlib.decompressobj(16 + zlib.MAX_WBITS) return DeflateDecoder() class HTTPResponse(io.IOBase): """ HTTP Response container. Backwards-compatible to httplib's HTTPResponse but the response ``body`` is loaded and decoded on-demand when the ``data`` property is accessed. Extra parameters for behaviour not present in httplib.HTTPResponse: :param preload_content: If True, the response's body will be preloaded during construction. :param decode_content: If True, attempts to decode specific content-encoding's based on headers (like 'gzip' and 'deflate') will be skipped and raw data will be used instead. :param original_response: When this HTTPResponse wrapper is generated from an httplib.HTTPResponse object, it's convenient to include the original for debug purposes. It's otherwise unused. """ CONTENT_DECODERS = ['gzip', 'deflate'] REDIRECT_STATUSES = [301, 302, 303, 307, 308] def __init__(self, body='', headers=None, status=0, version=0, reason=None, strict=0, preload_content=True, decode_content=True, original_response=None, pool=None, connection=None): self.headers = headers or {} self.status = status self.version = version self.reason = reason self.strict = strict self.decode_content = decode_content self._decoder = None self._body = body if body and isinstance(body, basestring) else None self._fp = None self._original_response = original_response self._pool = pool self._connection = connection if hasattr(body, 'read'): self._fp = body if preload_content and not self._body: self._body = self.read(decode_content=decode_content) def get_redirect_location(self): """ Should we redirect and where to? :returns: Truthy redirect location string if we got a redirect status code and valid location. ``None`` if redirect status and no location. ``False`` if not a redirect status code. """ if self.status in self.REDIRECT_STATUSES: return self.headers.get('location') return False def release_conn(self): if not self._pool or not self._connection: return self._pool._put_conn(self._connection) self._connection = None @property def data(self): # For backwords-compat with earlier urllib3 0.4 and earlier. if self._body: return self._body if self._fp: return self.read(cache_content=True) def read(self, amt=None, decode_content=None, cache_content=False): """ Similar to :meth:`httplib.HTTPResponse.read`, but with two additional parameters: ``decode_content`` and ``cache_content``. :param amt: How much of the content to read. If specified, caching is skipped because it doesn't make sense to cache partial content as the full response. :param decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. :param cache_content: If True, will save the returned data such that the same result is returned despite of the state of the underlying file object. This is useful if you want the ``.data`` property to continue working after having ``.read()`` the file object. (Overridden if ``amt`` is set.) """ # Note: content-encoding value should be case-insensitive, per RFC 2616 # Section 3.5 content_encoding = self.headers.get('content-encoding', '').lower() if self._decoder is None: if content_encoding in self.CONTENT_DECODERS: self._decoder = _get_decoder(content_encoding) if decode_content is None: decode_content = self.decode_content if self._fp is None: return flush_decoder = False try: if amt is None: # cStringIO doesn't like amt=None data = self._fp.read() flush_decoder = True else: cache_content = False data = self._fp.read(amt) if amt != 0 and not data: # Platform-specific: Buggy versions of Python. # Close the connection when no data is returned # # This is redundant to what httplib/http.client _should_ # already do. However, versions of python released before # December 15, 2012 (http://bugs.python.org/issue16298) do not # properly close the connection in all cases. There is no harm # in redundantly calling close. self._fp.close() flush_decoder = True try: if decode_content and self._decoder: data = self._decoder.decompress(data) except (IOError, zlib.error) as e: raise DecodeError( "Received response with content-encoding: %s, but " "failed to decode it." % content_encoding, e) if flush_decoder and decode_content and self._decoder: buf = self._decoder.decompress(binary_type()) data += buf + self._decoder.flush() if cache_content: self._body = data return data finally: if self._original_response and self._original_response.isclosed(): self.release_conn() def stream(self, amt=2**16, decode_content=None): """ A generator wrapper for the read() method. A call will block until ``amt`` bytes have been read from the connection or until the connection is closed. :param amt: How much of the content to read. The generator will return up to much data per iteration, but may return less. This is particularly likely when using compressed data. However, the empty string will never be returned. :param decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. """ while not is_fp_closed(self._fp): data = self.read(amt=amt, decode_content=decode_content) if data: yield data @classmethod def from_httplib(ResponseCls, r, **response_kw): """ Given an :class:`httplib.HTTPResponse` instance ``r``, return a corresponding :class:`urllib3.response.HTTPResponse` object. Remaining parameters are passed to the HTTPResponse constructor, along with ``original_response=r``. """ # Normalize headers between different versions of Python headers = {} for k, v in r.getheaders(): # Python 3: Header keys are returned capitalised k = k.lower() has_value = headers.get(k) if has_value: # Python 3: Repeating header keys are unmerged. v = ', '.join([has_value, v]) headers[k] = v # HTTPResponse objects in Python 3 don't have a .strict attribute strict = getattr(r, 'strict', 0) return ResponseCls(body=r, headers=headers, status=r.status, version=r.version, reason=r.reason, strict=strict, original_response=r, **response_kw) # Backwards-compatibility methods for httplib.HTTPResponse def getheaders(self): return self.headers def getheader(self, name, default=None): return self.headers.get(name, default) # Overrides from io.IOBase def close(self): if not self.closed: self._fp.close() @property def closed(self): if self._fp is None: return True elif hasattr(self._fp, 'closed'): return self._fp.closed elif hasattr(self._fp, 'isclosed'): # Python 2 return self._fp.isclosed() else: return True def fileno(self): if self._fp is None: raise IOError("HTTPResponse has no file to get a fileno from") elif hasattr(self._fp, "fileno"): return self._fp.fileno() else: raise IOError("The file-like object this HTTPResponse is wrapped " "around has no file descriptor") def flush(self): if self._fp is not None and hasattr(self._fp, 'flush'): return self._fp.flush() def readable(self): return True transifex-client-0.10/txclib/packages/urllib3/filepost.py0000644000175000001440000000471712247321676024404 0ustar mpessasusers00000000000000# urllib3/filepost.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php import codecs import mimetypes from uuid import uuid4 from io import BytesIO from .packages import six from .packages.six import b from .fields import RequestField writer = codecs.lookup('utf-8')[3] def choose_boundary(): """ Our embarassingly-simple replacement for mimetools.choose_boundary. """ return uuid4().hex def iter_field_objects(fields): """ Iterate over fields. Supports list of (k, v) tuples and dicts, and lists of :class:`~urllib3.fields.RequestField`. """ if isinstance(fields, dict): i = six.iteritems(fields) else: i = iter(fields) for field in i: if isinstance(field, RequestField): yield field else: yield RequestField.from_tuples(*field) def iter_fields(fields): """ Iterate over fields. .. deprecated :: The addition of `~urllib3.fields.RequestField` makes this function obsolete. Instead, use :func:`iter_field_objects`, which returns `~urllib3.fields.RequestField` objects, instead. Supports list of (k, v) tuples and dicts. """ if isinstance(fields, dict): return ((k, v) for k, v in six.iteritems(fields)) return ((k, v) for k, v in fields) def encode_multipart_formdata(fields, boundary=None): """ Encode a dictionary of ``fields`` using the multipart/form-data MIME format. :param fields: Dictionary of fields or list of (key, :class:`~urllib3.fields.RequestField`). :param boundary: If not specified, then a random boundary will be generated using :func:`mimetools.choose_boundary`. """ body = BytesIO() if boundary is None: boundary = choose_boundary() for field in iter_field_objects(fields): body.write(b('--%s\r\n' % (boundary))) writer(body).write(field.render_headers()) data = field.data if isinstance(data, int): data = str(data) # Backwards compatibility if isinstance(data, six.text_type): writer(body).write(data) else: body.write(data) body.write(b'\r\n') body.write(b('--%s--\r\n' % (boundary))) content_type = str('multipart/form-data; boundary=%s' % boundary) return body.getvalue(), content_type transifex-client-0.10/txclib/packages/urllib3/request.py0000644000175000001440000001336212247321676024243 0ustar mpessasusers00000000000000# urllib3/request.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php try: from urllib.parse import urlencode except ImportError: from urllib import urlencode from .filepost import encode_multipart_formdata __all__ = ['RequestMethods'] class RequestMethods(object): """ Convenience mixin for classes who implement a :meth:`urlopen` method, such as :class:`~urllib3.connectionpool.HTTPConnectionPool` and :class:`~urllib3.poolmanager.PoolManager`. Provides behavior for making common types of HTTP request methods and decides which type of request field encoding to use. Specifically, :meth:`.request_encode_url` is for sending requests whose fields are encoded in the URL (such as GET, HEAD, DELETE). :meth:`.request_encode_body` is for sending requests whose fields are encoded in the *body* of the request using multipart or www-form-urlencoded (such as for POST, PUT, PATCH). :meth:`.request` is for making any kind of request, it will look up the appropriate encoding format and use one of the above two methods to make the request. Initializer parameters: :param headers: Headers to include with all requests, unless other headers are given explicitly. """ _encode_url_methods = set(['DELETE', 'GET', 'HEAD', 'OPTIONS']) _encode_body_methods = set(['PATCH', 'POST', 'PUT', 'TRACE']) def __init__(self, headers=None): self.headers = headers or {} def urlopen(self, method, url, body=None, headers=None, encode_multipart=True, multipart_boundary=None, **kw): # Abstract raise NotImplemented("Classes extending RequestMethods must implement " "their own ``urlopen`` method.") def request(self, method, url, fields=None, headers=None, **urlopen_kw): """ Make a request using :meth:`urlopen` with the appropriate encoding of ``fields`` based on the ``method`` used. This is a convenience method that requires the least amount of manual effort. It can be used in most situations, while still having the option to drop down to more specific methods when necessary, such as :meth:`request_encode_url`, :meth:`request_encode_body`, or even the lowest level :meth:`urlopen`. """ method = method.upper() if method in self._encode_url_methods: return self.request_encode_url(method, url, fields=fields, headers=headers, **urlopen_kw) else: return self.request_encode_body(method, url, fields=fields, headers=headers, **urlopen_kw) def request_encode_url(self, method, url, fields=None, **urlopen_kw): """ Make a request using :meth:`urlopen` with the ``fields`` encoded in the url. This is useful for request methods like GET, HEAD, DELETE, etc. """ if fields: url += '?' + urlencode(fields) return self.urlopen(method, url, **urlopen_kw) def request_encode_body(self, method, url, fields=None, headers=None, encode_multipart=True, multipart_boundary=None, **urlopen_kw): """ Make a request using :meth:`urlopen` with the ``fields`` encoded in the body. This is useful for request methods like POST, PUT, PATCH, etc. When ``encode_multipart=True`` (default), then :meth:`urllib3.filepost.encode_multipart_formdata` is used to encode the payload with the appropriate content type. Otherwise :meth:`urllib.urlencode` is used with the 'application/x-www-form-urlencoded' content type. Multipart encoding must be used when posting files, and it's reasonably safe to use it in other times too. However, it may break request signing, such as with OAuth. Supports an optional ``fields`` parameter of key/value strings AND key/filetuple. A filetuple is a (filename, data, MIME type) tuple where the MIME type is optional. For example: :: fields = { 'foo': 'bar', 'fakefile': ('foofile.txt', 'contents of foofile'), 'realfile': ('barfile.txt', open('realfile').read()), 'typedfile': ('bazfile.bin', open('bazfile').read(), 'image/jpeg'), 'nonamefile': 'contents of nonamefile field', } When uploading a file, providing a filename (the first parameter of the tuple) is optional but recommended to best mimick behavior of browsers. Note that if ``headers`` are supplied, the 'Content-Type' header will be overwritten because it depends on the dynamic random boundary string which is used to compose the body of the request. The random boundary string can be explicitly set with the ``multipart_boundary`` parameter. """ if encode_multipart: body, content_type = encode_multipart_formdata(fields or {}, boundary=multipart_boundary) else: body, content_type = (urlencode(fields or {}), 'application/x-www-form-urlencoded') if headers is None: headers = self.headers headers_ = {'Content-Type': content_type} headers_.update(headers) return self.urlopen(method, url, body=body, headers=headers_, **urlopen_kw) transifex-client-0.10/txclib/packages/urllib3/__init__.py0000644000175000001440000000324712247321676024313 0ustar mpessasusers00000000000000# urllib3/__init__.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php """ urllib3 - Thread-safe connection pooling and re-using. """ __author__ = 'Andrey Petrov (andrey.petrov@shazow.net)' __license__ = 'MIT' __version__ = '1.7.1' from .connectionpool import ( HTTPConnectionPool, HTTPSConnectionPool, connection_from_url ) from . import exceptions from .filepost import encode_multipart_formdata from .poolmanager import PoolManager, ProxyManager, proxy_from_url from .response import HTTPResponse from .util import make_headers, get_host, Timeout # Set default logging handler to avoid "No handler found" warnings. import logging try: # Python 2.7+ from logging import NullHandler except ImportError: class NullHandler(logging.Handler): def emit(self, record): pass logging.getLogger(__name__).addHandler(NullHandler()) def add_stderr_logger(level=logging.DEBUG): """ Helper for quickly adding a StreamHandler to the logger. Useful for debugging. Returns the handler after adding it. """ # This method needs to be in this __init__.py to get the __name__ correct # even if urllib3 is vendored within another package. logger = logging.getLogger(__name__) handler = logging.StreamHandler() handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s %(message)s')) logger.addHandler(handler) logger.setLevel(level) logger.debug('Added an stderr logging handler to logger: %s' % __name__) return handler # ... Clean up. del NullHandler transifex-client-0.10/txclib/packages/urllib3/_collections.py0000644000175000001440000000552212247321676025227 0ustar mpessasusers00000000000000# urllib3/_collections.py # Copyright 2008-2013 Andrey Petrov and contributors (see CONTRIBUTORS.txt) # # This module is part of urllib3 and is released under # the MIT License: http://www.opensource.org/licenses/mit-license.php from collections import MutableMapping from threading import RLock try: # Python 2.7+ from collections import OrderedDict except ImportError: from .packages.ordered_dict import OrderedDict __all__ = ['RecentlyUsedContainer'] _Null = object() class RecentlyUsedContainer(MutableMapping): """ Provides a thread-safe dict-like container which maintains up to ``maxsize`` keys while throwing away the least-recently-used keys beyond ``maxsize``. :param maxsize: Maximum number of recent elements to retain. :param dispose_func: Every time an item is evicted from the container, ``dispose_func(value)`` is called. Callback which will get called """ ContainerCls = OrderedDict def __init__(self, maxsize=10, dispose_func=None): self._maxsize = maxsize self.dispose_func = dispose_func self._container = self.ContainerCls() self.lock = RLock() def __getitem__(self, key): # Re-insert the item, moving it to the end of the eviction line. with self.lock: item = self._container.pop(key) self._container[key] = item return item def __setitem__(self, key, value): evicted_value = _Null with self.lock: # Possibly evict the existing value of 'key' evicted_value = self._container.get(key, _Null) self._container[key] = value # If we didn't evict an existing value, we might have to evict the # least recently used item from the beginning of the container. if len(self._container) > self._maxsize: _key, evicted_value = self._container.popitem(last=False) if self.dispose_func and evicted_value is not _Null: self.dispose_func(evicted_value) def __delitem__(self, key): with self.lock: value = self._container.pop(key) if self.dispose_func: self.dispose_func(value) def __len__(self): with self.lock: return len(self._container) def __iter__(self): raise NotImplementedError('Iteration over this class is unlikely to be threadsafe.') def clear(self): with self.lock: # Copy pointers to all values, then wipe the mapping # under Python 2, this copies the list of values twice :-| values = list(self._container.values()) self._container.clear() if self.dispose_func: for value in values: self.dispose_func(value) def keys(self): with self.lock: return self._container.keys() transifex-client-0.10/txclib/packages/__init__.py0000644000175000001440000000000012247321676022717 0ustar mpessasusers00000000000000transifex-client-0.10/txclib/exceptions.py0000644000175000001440000000040012234723375021565 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- """ Exception classes for the tx client. """ class UnInitializedError(Exception): """The project directory has not been initialized.""" class UnknownCommandError(Exception): """The provided command is not supported.""" transifex-client-0.10/txclib/web.py0000644000175000001440000000041712247321676020173 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- import platform from txclib import get_version def user_agent_identifier(): """Return the user agent for the client.""" client_info = (get_version(), platform.system(), platform.machine()) return "txclient/%s (%s %s)" % client_info transifex-client-0.10/txclib/project.py0000644000175000001440000013664212247614034021067 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- import getpass import os import re import fnmatch import datetime import time import ConfigParser import ssl from txclib.web import * from txclib.utils import * from txclib.packages import urllib3 from txclib.urls import API_URLS from txclib.config import OrderedRawConfigParser, Flipdict from txclib.log import logger from txclib.processors import visit_hostname from txclib.paths import posix_path, native_path, posix_sep class ProjectNotInit(Exception): pass class Project(object): """ Represents an association between the local and remote project instances. """ def __init__(self, path_to_tx=None, init=True): """ Initialize the Project attributes. """ if init: self._init(path_to_tx) def _init(self, path_to_tx=None): instructions = "Run 'tx init' to initialize your project first!" try: self.root = self._get_tx_dir_path(path_to_tx) self.config_file = self._get_config_file_path(self.root) self.config = self._read_config_file(self.config_file) self.txrc_file = self._get_transifex_file() local_txrc_file = self._get_transifex_file(os.getcwd()) self.txrc = self._get_transifex_config([self.txrc_file, local_txrc_file]) if os.path.exists(local_txrc_file): self.txrc_file = local_txrc_file except ProjectNotInit, e: logger.error('\n'.join([unicode(e), instructions])) raise host = self.config.get('main', 'host') self.conn = urllib3.connection_from_url(host) def _get_config_file_path(self, root_path): """Check the .tx/config file exists.""" config_file = os.path.join(root_path, ".tx", "config") logger.debug("Config file is %s" % config_file) if not os.path.exists(config_file): msg = "Cannot find the config file (.tx/config)!" raise ProjectNotInit(msg) return config_file def _get_tx_dir_path(self, path_to_tx): """Check the .tx directory exists.""" root_path = path_to_tx or find_dot_tx() logger.debug("Path to tx is %s." % root_path) if not root_path: msg = "Cannot find any .tx directory!" raise ProjectNotInit(msg) return root_path def _read_config_file(self, config_file): """Parse the config file and return its contents.""" config = OrderedRawConfigParser() try: config.read(config_file) except Exception, err: msg = "Cannot open/parse .tx/config file: %s" % err raise ProjectNotInit(msg) return config def _get_transifex_config(self, txrc_files): """Read the configuration from the .transifexrc files.""" txrc = OrderedRawConfigParser() try: txrc.read(txrc_files) except Exception, e: msg = "Cannot read configuration file: %s" % e raise ProjectNotInit(msg) self._migrate_txrc_file(txrc) return txrc def _migrate_txrc_file(self, txrc): """Migrate the txrc file, if needed.""" if not os.path.exists(self.txrc_file): return txrc for section in txrc.sections(): orig_hostname = txrc.get(section, 'hostname') hostname = visit_hostname(orig_hostname) if hostname != orig_hostname: msg = "Hostname %s should be changed to %s." logger.info(msg % (orig_hostname, hostname)) if (sys.stdin.isatty() and sys.stdout.isatty() and confirm('Change it now? ', default=True)): txrc.set(section, 'hostname', hostname) msg = 'Hostname changed' logger.info(msg) else: hostname = orig_hostname self._save_txrc_file(txrc) return txrc def _get_transifex_file(self, directory=None): """Fetch the path of the .transifexrc file. It is in the home directory of the user by default. """ if directory is not None: logger.debug(".transifexrc file is at %s" % directory) return os.path.join(directory, ".transifexrc") directory = os.path.expanduser('~') txrc_file = os.path.join(directory, ".transifexrc") logger.debug(".transifexrc file is at %s" % directory) if not os.path.exists(txrc_file): msg = "%s not found." % (txrc_file) logger.info(msg) mask = os.umask(077) open(txrc_file, 'w').close() os.umask(mask) return txrc_file def validate_config(self): """ To ensure the json structure is correctly formed. """ pass def getset_host_credentials(self, host, user=None, password=None): """ Read .transifexrc and report user,pass for a specific host else ask the user for input. """ try: username = self.txrc.get(host, 'username') passwd = self.txrc.get(host, 'password') except (ConfigParser.NoOptionError, ConfigParser.NoSectionError): logger.info("No entry found for host %s. Creating..." % host) username = user or raw_input("Please enter your transifex username: ") while (not username): username = raw_input("Please enter your transifex username: ") passwd = password while (not passwd): passwd = getpass.getpass() logger.info("Updating %s file..." % self.txrc_file) self.txrc.add_section(host) self.txrc.set(host, 'username', username) self.txrc.set(host, 'password', passwd) self.txrc.set(host, 'token', '') self.txrc.set(host, 'hostname', host) return username, passwd def set_remote_resource(self, resource, source_lang, i18n_type, host, file_filter="translations%(proj)s.%(res)s.%(extension)s"): """Method to handle the add/conf of a remote resource.""" if not self.config.has_section(resource): self.config.add_section(resource) p_slug, r_slug = resource.split('.', 1) file_filter = file_filter.replace("", r"%s" % posix_sep) self.url_info = { 'host': host, 'project': p_slug, 'resource': r_slug } extension = self._extension_for(i18n_type)[1:] self.config.set(resource, 'source_lang', source_lang) self.config.set( resource, 'file_filter', file_filter % {'proj': p_slug, 'res': r_slug, 'extension': extension} ) self.config.set(resource, 'type', i18n_type) if host != self.config.get('main', 'host'): self.config.set(resource, 'host', host) def get_resource_host(self, resource): """ Returns the host that the resource is configured to use. If there is no such option we return the default one """ return self.config.get('main', 'host') def get_resource_lang_mapping(self, resource): """Get language mappings for a specific resource.""" lang_map = Flipdict() try: args = self.config.get("main", "lang_map") for arg in args.replace(' ', '').split(','): k,v = arg.split(":") lang_map.update({k:v}) except ConfigParser.NoOptionError: pass except (ValueError, KeyError): raise Exception("Your lang map configuration is not correct.") if self.config.has_section(resource): res_lang_map = Flipdict() try: args = self.config.get(resource, "lang_map") for arg in args.replace(' ', '').split(','): k,v = arg.split(":") res_lang_map.update({k:v}) except ConfigParser.NoOptionError: pass except (ValueError, KeyError): raise Exception("Your lang map configuration is not correct.") # merge the lang maps and return result lang_map.update(res_lang_map) return lang_map def get_source_file(self, resource): """ Get source file for a resource. """ if self.config.has_section(resource): source_lang = self.config.get(resource, "source_lang") source_file = self.get_resource_option(resource, 'source_file') or None if source_file is None: try: file_filter = self.config.get(resource, "file_filter") filename = file_filter.replace('', source_lang) if os.path.exists(filename): return native_path(filename) except ConfigParser.NoOptionError: pass else: return native_path(source_file) def get_resource_files(self, resource): """ Get a dict for all files assigned to a resource. First we calculate the files matching the file expression and then we apply all translation excpetions. The resulting dict will be in this format: { 'en': 'path/foo/en/bar.po', 'de': 'path/foo/de/bar.po', 'es': 'path/exceptions/es.po'} NOTE: All paths are relative to the root of the project """ tr_files = {} if self.config.has_section(resource): try: file_filter = self.config.get(resource, "file_filter") except ConfigParser.NoOptionError: file_filter = "$^" source_lang = self.config.get(resource, "source_lang") source_file = self.get_source_file(resource) expr_re = regex_from_filefilter(file_filter, self.root) expr_rec = re.compile(expr_re) for f_path in files_in_project(self.root): match = expr_rec.match(posix_path(f_path)) if match: lang = match.group(1) if lang != source_lang: f_path = os.path.relpath(f_path, self.root) if f_path != source_file: tr_files.update({lang: f_path}) for (name, value) in self.config.items(resource): if name.startswith("trans."): value = native_path(value) lang = name.split('.')[1] # delete language which has same file if value in tr_files.values(): keys = [] for k, v in tr_files.iteritems(): if v == value: keys.append(k) if len(keys) == 1: del tr_files[keys[0]] else: raise Exception("Your configuration seems wrong."\ " You have multiple languages pointing to"\ " the same file.") # Add language with correct file tr_files.update({lang:value}) return tr_files return None def get_resource_option(self, resource, option): """ Return the requested option for a specific resource If there is no such option, we return None """ if self.config.has_section(resource): if self.config.has_option(resource, option): return self.config.get(resource, option) return None def get_resource_list(self, project=None): """ Parse config file and return tuples with the following format [ (project_slug, resource_slug), (..., ...)] """ resource_list= [] for r in self.config.sections(): if r == 'main': continue p_slug, r_slug = r.split('.', 1) if project and p_slug != project: continue resource_list.append(r) return resource_list def save(self): """ Store the config dictionary in the .tx/config file of the project. """ self._save_tx_config() self._save_txrc_file() def _save_tx_config(self, config=None): """Save the local config file.""" if config is None: config = self.config fh = open(self.config_file,"w") config.write(fh) fh.close() def _save_txrc_file(self, txrc=None): """Save the .transifexrc file.""" if txrc is None: txrc = self.txrc mask = os.umask(077) fh = open(self.txrc_file, 'w') txrc.write(fh) fh.close() os.umask(mask) def get_full_path(self, relpath): if relpath[0] == os.path.sep: return relpath else: return os.path.join(self.root, relpath) def _get_pseudo_file(self, slang, resource, file_filter): pseudo_file = file_filter.replace('', '%s_pseudo' % slang) return native_path(pseudo_file) def pull(self, languages=[], resources=[], overwrite=True, fetchall=False, fetchsource=False, force=False, skip=False, minimum_perc=0, mode=None, pseudo=False): """Pull all translations file from transifex server.""" self.minimum_perc = minimum_perc resource_list = self.get_chosen_resources(resources) if mode == 'reviewed': url = 'pull_reviewed_file' elif mode == 'translator': url = 'pull_translator_file' elif mode == 'developer': url = 'pull_developer_file' else: url = 'pull_file' for resource in resource_list: logger.debug("Handling resource %s" % resource) self.resource = resource project_slug, resource_slug = resource.split('.', 1) files = self.get_resource_files(resource) slang = self.get_resource_option(resource, 'source_lang') sfile = self.get_source_file(resource) lang_map = self.get_resource_lang_mapping(resource) host = self.get_resource_host(resource) logger.debug("Language mapping is: %s" % lang_map) if mode is None: mode = self._get_option(resource, 'mode') self.url_info = { 'host': host, 'project': project_slug, 'resource': resource_slug } logger.debug("URL data are: %s" % self.url_info) stats = self._get_stats_for_resource() try: file_filter = self.config.get(resource, 'file_filter') except ConfigParser.NoOptionError: file_filter = None # Pull source file pull_languages = set([]) new_translations = set([]) if pseudo: pseudo_file = self._get_pseudo_file( slang, resource, file_filter ) if self._should_download(slang, stats, local_file=pseudo_file): msg = "Pulling pseudo file for resource %s." % resource logger.info(msg) self._download_pseudo( project_slug, resource_slug, pseudo_file ) if fetchall: new_translations = self._new_translations_to_add( files, slang, lang_map, stats, force ) if new_translations: msg = "New translations found for the following languages: %s" logger.info(msg % ', '.join(new_translations)) existing, new = self._languages_to_pull( languages, files, lang_map, stats, force ) pull_languages |= existing new_translations |= new logger.debug("Adding to new translations: %s" % new) if fetchsource: if sfile and slang not in pull_languages: pull_languages.add(slang) elif slang not in new_translations: new_translations.add(slang) if pull_languages: logger.debug("Pulling languages for: %s" % pull_languages) msg = "Pulling translations for resource %s (source: %s)" logger.info(msg % (resource, sfile)) for lang in pull_languages: local_lang = lang if lang in lang_map.values(): remote_lang = lang_map.flip[lang] else: remote_lang = lang if languages and lang not in pull_languages: logger.debug("Skipping language %s" % lang) continue if lang != slang: local_file = files.get(lang, None) or files[lang_map[lang]] else: local_file = sfile logger.debug("Using file %s" % local_file) kwargs = { 'lang': remote_lang, 'stats': stats, 'local_file': local_file, 'force': force, 'mode': mode, } if not self._should_update_translation(**kwargs): msg = "Skipping '%s' translation (file: %s)." logger.info( msg % (color_text(remote_lang, "RED"), local_file) ) continue if not overwrite: local_file = ("%s.new" % local_file) logger.warning( " -> %s: %s" % (color_text(remote_lang, "RED"), local_file) ) try: r = self.do_url_request(url, language=remote_lang) except Exception, e: if not skip: raise e else: logger.error(e) continue base_dir = os.path.split(local_file)[0] mkdir_p(base_dir) fd = open(local_file, 'wb') fd.write(r) fd.close() if new_translations: msg = "Pulling new translations for resource %s (source: %s)" logger.info(msg % (resource, sfile)) for lang in new_translations: if lang in lang_map.keys(): local_lang = lang_map[lang] else: local_lang = lang remote_lang = lang if file_filter: local_file = os.path.relpath( os.path.join( self.root, native_path( file_filter.replace('', local_lang) ) ), os.curdir ) else: trans_dir = os.path.join(self.root, ".tx", resource) if not os.path.exists(trans_dir): os.mkdir(trans_dir) local_file = os.path.relpath(os.path.join(trans_dir, '%s_translation' % local_lang, os.curdir)) if lang != slang: satisfies_min = self._satisfies_min_translated( stats[remote_lang], mode ) if not satisfies_min: msg = "Skipping language %s due to used options." logger.info(msg % lang) continue logger.warning( " -> %s: %s" % (color_text(remote_lang, "RED"), local_file) ) r = self.do_url_request(url, language=remote_lang) base_dir = os.path.split(local_file)[0] mkdir_p(base_dir) fd = open(local_file, 'wb') fd.write(r) fd.close() def push(self, source=False, translations=False, force=False, resources=[], languages=[], skip=False, no_interactive=False): """ Push all the resources """ resource_list = self.get_chosen_resources(resources) self.skip = skip self.force = force for resource in resource_list: push_languages = [] project_slug, resource_slug = resource.split('.', 1) files = self.get_resource_files(resource) slang = self.get_resource_option(resource, 'source_lang') sfile = self.get_source_file(resource) lang_map = self.get_resource_lang_mapping(resource) host = self.get_resource_host(resource) logger.debug("Language mapping is: %s" % lang_map) logger.debug("Using host %s" % host) self.url_info = { 'host': host, 'project': project_slug, 'resource': resource_slug } logger.info("Pushing translations for resource %s:" % resource) stats = self._get_stats_for_resource() if force and not no_interactive: answer = raw_input("Warning: By using --force, the uploaded" " files will overwrite remote translations, even if they" " are newer than your uploaded files.\nAre you sure you" " want to continue? [y/N] ") if not answer in ["", 'Y', 'y', "yes", 'YES']: return if source: if sfile is None: logger.error("You don't seem to have a proper source file" " mapping for resource %s. Try without the --source" " option or set a source file first and then try again." % resource) continue # Push source file try: logger.warning("Pushing source file (%s)" % sfile) if not self._resource_exists(stats): logger.info("Resource does not exist. Creating...") fileinfo = "%s;%s" % (resource_slug, slang) filename = self.get_full_path(sfile) self._create_resource(resource, project_slug, fileinfo, filename) self.do_url_request( 'push_source', multipart=True, method="PUT", files=[( "%s;%s" % (resource_slug, slang) , self.get_full_path(sfile) )], ) except Exception, e: if not skip: raise else: logger.error(e) else: try: self.do_url_request('resource_details') except Exception, e: code = getattr(e, 'code', None) if code == 404: msg = "Resource %s doesn't exist on the server." logger.error(msg % resource) continue if translations: # Check if given language codes exist if not languages: push_languages = files.keys() else: push_languages = [] f_langs = files.keys() for l in languages: if l in lang_map.keys(): l = lang_map[l] push_languages.append(l) if l not in f_langs: msg = "Warning: No mapping found for language code '%s'." logger.error(msg % color_text(l,"RED")) logger.debug("Languages to push are %s" % push_languages) # Push translation files one by one for lang in push_languages: local_lang = lang if lang in lang_map.values(): remote_lang = lang_map.flip[lang] else: remote_lang = lang local_file = files[local_lang] kwargs = { 'lang': remote_lang, 'stats': stats, 'local_file': local_file, 'force': force, } if not self._should_push_translation(**kwargs): msg = "Skipping '%s' translation (file: %s)." logger.info(msg % (color_text(lang, "RED"), local_file)) continue msg = "Pushing '%s' translations (file: %s)" logger.warning( msg % (color_text(remote_lang, "RED"), local_file) ) try: self.do_url_request( 'push_translation', multipart=True, method='PUT', files=[( "%s;%s" % (resource_slug, remote_lang), self.get_full_path(local_file) )], language=remote_lang ) logger.debug("Translation %s pushed." % remote_lang) except Exception, e: if not skip: raise e else: logger.error(e) def delete(self, resources=[], languages=[], skip=False, force=False): """Delete translations.""" resource_list = self.get_chosen_resources(resources) self.skip = skip self.force = force if not languages: delete_func = self._delete_resource else: delete_func = self._delete_translations for resource in resource_list: project_slug, resource_slug = resource.split('.', 1) host = self.get_resource_host(resource) self.url_info = { 'host': host, 'project': project_slug, 'resource': resource_slug } logger.debug("URL data are: %s" % self.url_info) project_details = parse_json( self.do_url_request('project_details', project=self) ) teams = project_details['teams'] stats = self._get_stats_for_resource() delete_func(project_details, resource, stats, languages) def _delete_resource(self, project_details, resource, stats, *args): """Delete a resource from Transifex.""" project_slug, resource_slug = resource.split('.', 1) project_resource_slugs = [ r['slug'] for r in project_details['resources'] ] logger.info("Deleting resource %s:" % resource) if resource_slug not in project_resource_slugs: if not self.skip: msg = "Skipping: %s : Resource does not exist." logger.info(msg % resource) return if not self.force: slang = self.get_resource_option(resource, 'source_lang') for language in stats: if language == slang: continue if int(stats[language]['translated_entities']) > 0: msg = ( "Skipping: %s : Unable to delete resource because it " "has a not empty %s translation.\nPlease use -f or " "--force option to delete this resource." ) logger.info(msg % (resource, language)) return try: self.do_url_request('delete_resource', method="DELETE") self.config.remove_section(resource) self.save() msg = "Deleted resource %s of project %s." logger.info(msg % (resource_slug, project_slug)) except Exception, e: msg = "Unable to delete resource %s of project %s." logger.error(msg % (resource_slug, project_slug)) if not self.skip: raise def _delete_translations(self, project_details, resource, stats, languages): """Delete the specified translations for the specified resource.""" logger.info("Deleting translations from resource %s:" % resource) for language in languages: self._delete_translation(project_details, resource, stats, language) def _delete_translation(self, project_details, resource, stats, language): """Delete a specific translation from the specified resource.""" project_slug, resource_slug = resource.split('.', 1) if language not in stats: if not self.skip: msg = "Skipping %s: Translation does not exist." logger.warning(msg % (language)) return if not self.force: teams = project_details['teams'] if language in teams: msg = ( "Skipping %s: Unable to delete translation because it is " "associated with a team.\nPlease use -f or --force option " "to delete this translation." ) logger.warning(msg % language) return if int(stats[language]['translated_entities']) > 0: msg = ( "Skipping %s: Unable to delete translation because it " "is not empty.\nPlease use -f or --force option to delete " "this translation." ) logger.warning(msg % language) return try: self.do_url_request( 'delete_translation', language=language, method="DELETE" ) msg = "Deleted language %s from resource %s of project %s." logger.info(msg % (language, resource_slug, project_slug)) except Exception, e: msg = "Unable to delete translation %s" logger.error(msg % language) if not self.skip: raise def do_url_request(self, api_call, multipart=False, data=None, files=[], method="GET", **kwargs): """ Issues a url request. """ # Read the credentials from the config file (.transifexrc) host = self.url_info['host'] try: username = self.txrc.get(host, 'username') passwd = self.txrc.get(host, 'password') token = self.txrc.get(host, 'token') hostname = self.txrc.get(host, 'hostname') except ConfigParser.NoSectionError: raise Exception("No user credentials found for host %s. Edit" " ~/.transifexrc and add the appropriate info in there." % host) # Create the Url kwargs['hostname'] = hostname kwargs.update(self.url_info) url = (API_URLS[api_call] % kwargs).encode('UTF-8') logger.debug(url) if multipart: for info, filename in files: name = os.path.basename(filename) data = { "resource": info.split(';')[0], "language": info.split(';')[1], "uploaded_file": (name, open(filename, 'rb').read()) } headers = urllib3.util.make_headers( basic_auth='{0}:{1}'.format(username, passwd), accept_encoding=True, user_agent=user_agent_identifier(), keep_alive=True ) r = self.conn.request( method, url, fields=data, headers=headers ) else: headers = urllib3.util.make_headers( basic_auth='{0}:{1}'.format(username, passwd), accept_encoding=True, user_agent=user_agent_identifier(), keep_alive=True ) r = self.conn.request( method, url, fields=data, headers=headers ) r.close() if r.status < 200 or r.status >= 400: raise Exception(r.data) return r.data def _should_update_translation(self, lang, stats, local_file, force=False, mode=None): """Whether a translation should be udpated from Transifex. We use the following criteria for that: - If user requested to force the download. - If language exists in Transifex. - If the local file is older than the Transifex's file. - If the user requested a x% completion. Args: lang: The language code to check. stats: The (global) statistics object. local_file: The local translation file. force: A boolean flag. mode: The mode for the translation. Returns: True or False. """ return self._should_download(lang, stats, local_file, force) def _should_add_translation(self, lang, stats, force=False, mode=None): """Whether a translation should be added from Transifex. We use the following criteria for that: - If user requested to force the download. - If language exists in Transifex. - If the user requested a x% completion. Args: lang: The language code to check. stats: The (global) statistics object. force: A boolean flag. mode: The mode for the translation. Returns: True or False. """ return self._should_download(lang, stats, None, force) def _should_download(self, lang, stats, local_file=None, force=False, mode=None): """Return whether a translation should be downloaded. If local_file is None, skip the timestamps check (the file does not exist locally). """ try: lang_stats = stats[lang] except KeyError, e: logger.debug("No lang %s in statistics" % lang) return False satisfies_min = self._satisfies_min_translated(lang_stats, mode) if not satisfies_min: return False if force: logger.debug("Downloading translation due to -f") return True if local_file is not None: remote_update = self._extract_updated(lang_stats) if not self._remote_is_newer(remote_update, local_file): logger.debug("Local is newer than remote for lang %s" % lang) return False return True def _should_push_translation(self, lang, stats, local_file, force=False): """Return whether a local translation file should be pushed to Trasnifex. We use the following criteria for that: - If user requested to force the upload. - If language exists in Transifex. - If local file is younger than the remote file. Args: lang: The language code to check. stats: The (global) statistics object. local_file: The local translation file. force: A boolean flag. Returns: True or False. """ if force: logger.debug("Push translation due to -f.") return True try: lang_stats = stats[lang] except KeyError, e: logger.debug("Language %s does not exist in Transifex." % lang) return True if local_file is not None: remote_update = self._extract_updated(lang_stats) if self._remote_is_newer(remote_update, local_file): msg = "Remote translation is newer than local file for lang %s" logger.debug(msg % lang) return False return True def _generate_timestamp(self, update_datetime): """Generate a UNIX timestamp from the argument. Args: update_datetime: The datetime in the format used by Transifex. Returns: A float, representing the timestamp that corresponds to the argument. """ time_format = "%Y-%m-%d %H:%M:%S" return time.mktime( datetime.datetime( *time.strptime(update_datetime, time_format)[0:5] ).utctimetuple() ) def _get_time_of_local_file(self, path): """Get the modified time of the path_. Args: path: The path we want the mtime for. Returns: The time as a timestamp or None, if the file does not exist """ if not os.path.exists(path): return None return time.mktime(time.gmtime(os.path.getmtime(path))) def _satisfies_min_translated(self, stats, mode=None): """Check whether a translation fulfills the filter used for minimum translated percentage. Args: perc: The current translation percentage. Returns: True or False """ cur = self._extract_completed(stats, mode) option_name = 'minimum_perc' if self.minimum_perc is not None: minimum_percent = self.minimum_perc else: global_minimum = int( self.get_resource_option('main', option_name) or 0 ) resource_minimum = int( self.get_resource_option( self.resource, option_name ) or global_minimum ) minimum_percent = resource_minimum return cur >= minimum_percent def _remote_is_newer(self, remote_updated, local_file): """Check whether the remote translation is newer that the local file. Args: remote_updated: The date and time the translation was last updated remotely. local_file: The local file. Returns: True or False. """ if remote_updated is None: logger.debug("No remote time") return False remote_time = self._generate_timestamp(remote_updated) local_time = self._get_time_of_local_file( self.get_full_path(local_file) ) logger.debug( "Remote time is %s and local %s" % (remote_time, local_time) ) if local_time is not None and remote_time < local_time: return False return True @classmethod def _extract_completed(cls, stats, mode=None): """Extract the information for the translated percentage from the stats. Args: stats: The stats object for a language as returned by Transifex. mode: The mode of translations requested. Returns: The percentage of translation as integer. """ if mode == 'reviewed': key = 'reviewed_percentage' else: key = 'completed' try: return int(stats[key][:-1]) except KeyError, e: return 0 @classmethod def _extract_updated(cls, stats): """Extract the information for the last update of a translation. Args: stats: The stats object for a language as returned by Transifex. Returns: The last update field. """ try: return stats['last_update'] except KeyError, e: return None def _download_pseudo(self, project_slug, resource_slug, pseudo_file): response = self.do_url_request( 'pull_pseudo_file', resource_slug=resource_slug, project_slug=project_slug ) response = parse_json(response) base_dir = os.path.split(pseudo_file)[0] mkdir_p(base_dir) with open(pseudo_file, "wb") as fd: fd.write(response['content']) def _new_translations_to_add(self, files, slang, lang_map, stats, force=False): """Return a list of translations which are new to the local installation. """ new_translations = [] timestamp = time.time() langs = stats.keys() logger.debug("Available languages are: %s" % langs) for lang in langs: lang_exists = lang in files.keys() lang_is_source = lang == slang mapped_lang_exists = ( lang in lang_map and lang_map[lang] in files.keys() ) if lang_exists or lang_is_source or mapped_lang_exists: continue if self._should_add_translation(lang, stats, force): new_translations.append(lang) return set(new_translations) def _get_stats_for_resource(self): """Get the statistics information for a resource.""" try: r = self.do_url_request('resource_stats') logger.debug("Statistics response is %s" % r) stats = parse_json(r) except ssl.SSLError: logger.error("Invalid SSL certificate") raise except Exception, e: logger.debug(unicode(e)) raise return stats def get_chosen_resources(self, resources): """Get the resources the user selected. Support wildcards in the resources specified by the user. Args: resources: A list of resources as specified in command-line or an empty list. Returns: A list of resources. """ configured_resources = self.get_resource_list() if not resources: return configured_resources selected_resources = [] for resource in resources: found = False for full_name in configured_resources: if fnmatch.fnmatch(full_name, resource): selected_resources.append(full_name) found = True if not found: msg = "Specified resource '%s' does not exist." raise Exception(msg % resource) logger.debug("Operating on resources: %s" % selected_resources) return selected_resources def _languages_to_pull(self, languages, files, lang_map, stats, force): """Get a set of langauges to pull. Args: languages: A list of languages the user selected in cmd. files: A dictionary of current local translation files. Returns: A tuple of a set of existing languages and new translations. """ if not languages: pull_languages = set([]) pull_languages |= set(files.keys()) mapped_files = [] for lang in pull_languages: if lang in lang_map.flip: mapped_files.append(lang_map.flip[lang]) pull_languages -= set(lang_map.flip.keys()) pull_languages |= set(mapped_files) return (pull_languages, set([])) else: pull_languages = [] new_translations = [] f_langs = files.keys() for l in languages: if l not in f_langs and not (l in lang_map and lang_map[l] in f_langs): if self._should_add_translation(l, stats, force): new_translations.append(l) else: if l in lang_map.keys(): l = lang_map[l] pull_languages.append(l) return (set(pull_languages), set(new_translations)) def _extension_for(self, i18n_type): """Return the extension used for the specified type.""" try: res = parse_json(self.do_url_request('formats')) return res[i18n_type]['file-extensions'].split(',')[0] except Exception, e: logger.error(e) return '' def _resource_exists(self, stats): """Check if resource exists. Args: stats: The statistics dict as returned by Tx. Returns: True, if the resource exists in the server. """ return bool(stats) def _create_resource(self, resource, pslug, fileinfo, filename, **kwargs): """Create a resource. Args: resource: The full resource name. pslug: The slug of the project. fileinfo: The information of the resource. filename: The name of the file. Raises: URLError, in case of a problem. """ multipart = True method = "POST" api_call = 'create_resource' host = self.url_info['host'] try: username = self.txrc.get(host, 'username') passwd = self.txrc.get(host, 'password') token = self.txrc.get(host, 'token') hostname = self.txrc.get(host, 'hostname') except ConfigParser.NoSectionError: raise Exception("No user credentials found for host %s. Edit" " ~/.transifexrc and add the appropriate info in there." % host) # Create the Url kwargs['hostname'] = hostname kwargs.update(self.url_info) kwargs['project'] = pslug url = (API_URLS[api_call] % kwargs).encode('UTF-8') headers = None i18n_type = self._get_option(resource, 'type') if i18n_type is None: logger.error( "Please define the resource type in .tx/config (eg. type = PO)." " More info: http://bit.ly/txcl-rt" ) headers = urllib3.util.make_headers( basic_auth='{0}:{1}'.format(username, passwd), accept_encoding=True, user_agent=user_agent_identifier(), keep_alive=True ) data = { "slug": fileinfo.split(';')[0], "name": fileinfo.split(';')[0], "uploaded_file": (filename, open(filename, 'rb').read()), "i18n_type": i18n_type } try: r = self.conn.request(method, url, fields=data, headers=headers) except ssl.SSLError: logger.error("Invalid SSL certificate") r.close() return r.data def _get_option(self, resource, option): """Get the value for the option in the config file. If the option is not in the resource section, look for it in the project. Args: resource: The resource name. option: The option the value of which we are interested in. Returns: The option value or None, if it does not exist. """ value = self.get_resource_option(resource, option) if value is None: if self.config.has_option('main', option): return self.config.get('main', option) return value def set_i18n_type(self, resources, i18n_type): """Set the type for the specified resources.""" self._set_resource_option(resources, key='type', value=i18n_type) def set_min_perc(self, resources, perc): """Set the minimum percentage for the resources.""" self._set_resource_option(resources, key='minimum_perc', value=perc) def set_default_mode(self, resources, mode): """Set the default mode for the specified resources.""" self._set_resource_option(resources, key='mode', value=mode) def _set_resource_option(self, resources, key, value): """Set options in the config file. If resources is empty. set the option globally. """ if not resources: self.config.set('main', key, value) return for r in resources: self.config.set(r, key, value) transifex-client-0.10/txclib/paths.py0000644000175000001440000000141112247321676020530 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- """ Path handling. We need to take into account the differences between UNIX systems and Windows. """ import os posix_sep = os.sep if os.altsep is None else os.altsep def posix_path(fpath): """Convert a filesystem path to a posix path. Always use the forward slash as a separator. For instance, in windows the separator is the backslash. Args: fpath: The path to convert. """ return fpath if os.altsep is None else fpath.replace(os.sep, os.altsep) def native_path(fpath): """Convert a filesystem path to a native path. Use whatever separator is defined by the platform. Args: fpath: The path to convert. """ return fpath if os.altsep is None else fpath.replace(os.altsep, os.sep) transifex-client-0.10/txclib/parsers.py0000644000175000001440000002711212247365732021077 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- from optparse import OptionParser, OptionGroup class EpilogParser(OptionParser): def format_epilog(self, formatter): return self.epilog def delete_parser(): """Return the command-line parser for the delete command.""" usage = "usage: %prog [tx_options] delete OPTION [OPTIONS]" description = ( "This command deletes translations for a resource in the remote server." ) epilog = ( "\nExamples:\n" " To delete a translation:\n " "$ tx delete -r project.resource -l \n\n" " To delete a resource:\n $ tx delete -r project.resource\n" ) parser = EpilogParser(usage=usage, description=description, epilog=epilog) parser.add_option( "-r", "--resource", action="store", dest="resources", default=None, help="Specify the resource you want to delete (defaults to all)" ) parser.add_option( "-l","--language", action="store", dest="languages", default=None, help="Specify the translation you want to delete" ) parser.add_option( "--skip", action="store_true", dest="skip_errors", default=False, help="Don't stop on errors." ) parser.add_option( "-f","--force", action="store_true", dest="force_delete", default=False, help="Delete an entity forcefully." ) return parser def help_parser(): """Return the command-line parser for the help command.""" usage="usage: %prog help command" description="Lists all available commands in the transifex command"\ " client. If a command is specified, the help page of the specific"\ " command is displayed instead." parser = OptionParser(usage=usage, description=description) return parser def init_parser(): """Return the command-line parser for the init command.""" usage="usage: %prog [tx_options] init " description="This command initializes a new project for use with"\ " transifex. It is recommended to execute this command in the"\ " top level directory of your project so that you can include"\ " all files under it in transifex. If no path is provided, the"\ " current working dir will be used." parser = OptionParser(usage=usage, description=description) parser.add_option("--host", action="store", dest="host", default=None, help="Specify a default Transifex host.") parser.add_option("--user", action="store", dest="user", default=None, help="Specify username for Transifex server.") parser.add_option("--pass", action="store", dest="password", default=None, help="Specify password for Transifex server.") return parser def pull_parser(): """Return the command-line parser for the pull command.""" usage="usage: %prog [tx_options] pull [options]" description="This command pulls all outstanding changes from the remote"\ " Transifex server to the local repository. By default, only the"\ " files that are watched by Transifex will be updated but if you"\ " want to fetch the translations for new languages as well, use the"\ " -a|--all option. (Note: new translations are saved in the .tx folder"\ " and require the user to manually rename them and add then in "\ " transifex using the set_translation command)." parser = OptionParser(usage=usage,description=description) parser.add_option("-l","--language", action="store", dest="languages", default=[], help="Specify which translations you want to pull" " (defaults to all)") parser.add_option("-r","--resource", action="store", dest="resources", default=[], help="Specify the resource for which you want to pull" " the translations (defaults to all)") parser.add_option("-a","--all", action="store_true", dest="fetchall", default=False, help="Fetch all translation files from server (even new" " ones)") parser.add_option("-s","--source", action="store_true", dest="fetchsource", default=False, help="Force the fetching of the source file (default:" " False)") parser.add_option("-f","--force", action="store_true", dest="force", default=False, help="Force download of translations files.") parser.add_option("--skip", action="store_true", dest="skip_errors", default=False, help="Don't stop on errors. Useful when pushing many" " files concurrently.") parser.add_option("--disable-overwrite", action="store_false", dest="overwrite", default=True, help="By default transifex will fetch new translations files and"\ " replace existing ones. Use this flag if you want to disable"\ " this feature") parser.add_option("--minimum-perc", action="store", type="int", dest="minimum_perc", default=0, help="Specify the minimum acceptable percentage of a translation " "in order to download it.") parser.add_option("--pseudo", action="store_true", dest="pseudo", default=False, help="Apply this option to download a pseudo file.") parser.add_option( "--mode", action="store", dest="mode", help=( "Specify the mode of the translation file to pull (e.g. " "'reviewed'). See http://bit.ly/txcmod1 for available values." ) ) return parser def push_parser(): """Return the command-line parser for the push command.""" usage="usage: %prog [tx_options] push [options]" description="This command pushes all local files that have been added to"\ " Transifex to the remote server. All new translations are merged"\ " with existing ones and if a language doesn't exists then it gets"\ " created. If you want to push the source file as well (either"\ " because this is your first time running the client or because"\ " you just have updated with new entries), use the -f|--force option."\ " By default, this command will push all files which are watched by"\ " Transifex but you can filter this per resource or/and language." parser = OptionParser(usage=usage, description=description) parser.add_option("-l","--language", action="store", dest="languages", default=None, help="Specify which translations you want to push" " (defaults to all)") parser.add_option("-r","--resource", action="store", dest="resources", default=None, help="Specify the resource for which you want to push" " the translations (defaults to all)") parser.add_option("-f","--force", action="store_true", dest="force_creation", default=False, help="Push source files without checking modification" " times.") parser.add_option("--skip", action="store_true", dest="skip_errors", default=False, help="Don't stop on errors. Useful when pushing many" " files concurrently.") parser.add_option("-s", "--source", action="store_true", dest="push_source", default=False, help="Push the source file to the server.") parser.add_option("-t", "--translations", action="store_true", dest="push_translations", default=False, help="Push the translation files to the server") parser.add_option("--no-interactive", action="store_true", dest="no_interactive", default=False, help="Don't require user input when forcing a push.") return parser def set_parser(): """Return the command-line parser for the set command.""" usage="usage: %prog [tx_options] set [options] [args]" description="This command can be used to create a mapping between files"\ " and projects either using local files or using files from a remote"\ " Transifex server." epilog="\nExamples:\n"\ " To set the source file:\n $ tx set -r project.resource --source -l en \n\n"\ " To set a single translation file:\n $ tx set -r project.resource -l de \n\n"\ " To automatically detect and assign the source files and translations:\n"\ " $ tx set --auto-local -r project.resource 'expr' --source-lang en\n\n"\ " To set a specific file as a source and auto detect translations:\n"\ " $ tx set --auto-local -r project.resource 'expr' --source-lang en"\ " --source-file \n\n"\ " To set a remote release/resource/project:\n"\ " $ tx set --auto-remote \n" parser = EpilogParser(usage=usage, description=description, epilog=epilog) parser.add_option("--auto-local", action="store_true", dest="local", default=False, help="Used when auto configuring local project.") parser.add_option("--auto-remote", action="store_true", dest="remote", default=False, help="Used when adding remote files from Transifex" " server.") parser.add_option("-r","--resource", action="store", dest="resource", default=None, help="Specify the slug of the resource that you're" " setting up (This must be in the following format:" " `project_slug.resource_slug`).") parser.add_option( "--source", action="store_true", dest="is_source", default=False, help=( "Specify that the given file is a source file " "[doesn't work with the --auto-* commands]." ) ) parser.add_option("-l","--language", action="store", dest="language", default=None, help="Specify which translations you want to pull" " [doesn't work with the --auto-* commands].") parser.add_option("-t", "--type", action="store", dest="i18n_type", help=( "Specify the i18n type of the resource(s). This is only needed, if " "the resource(s) does not exist yet in Transifex. For a list of " "available i18n types, see " "http://help.transifex.com/features/formats.html" ) ) parser.add_option("--minimum-perc", action="store", dest="minimum_perc", help=( "Specify the minimum acceptable percentage of a translation " "in order to download it." ) ) parser.add_option( "--mode", action="store", dest="mode", help=( "Specify the mode of the translation file to pull (e.g. " "'reviewed'). See http://help.transifex.com/features/client/" "index.html#defining-the-mode-of-the-translated-file for the" "available values." ) ) group = OptionGroup(parser, "Extended options", "These options can only be" " used with the --auto-local command.") group.add_option("-s","--source-language", action="store", dest="source_language", default=None, help="Specify the source language of a resource" " [requires --auto-local].") group.add_option("-f","--source-file", action="store", dest="source_file", default=None, help="Specify the source file of a resource [requires" " --auto-local].") group.add_option("--execute", action="store_true", dest="execute", default=False, help="Execute commands [requires --auto-local].") parser.add_option_group(group) return parser def status_parser(): """Return the command-line parser for the status command.""" usage="usage: %prog [tx_options] status [options]" description="Prints the status of the current project by reading the"\ " data in the configuration file." parser = OptionParser(usage=usage,description=description) parser.add_option("-r","--resource", action="store", dest="resources", default=[], help="Specify resources") return parser def parse_csv_option(option): """Return a list out of the comma-separated option or an empty list.""" if option: return option.split(',') else: return [] transifex-client-0.10/txclib/urls.py0000644000175000001440000000314212247365732020402 0ustar mpessasusers00000000000000# These are the Transifex API urls API_URLS = { 'get_resources': '%(hostname)s/api/2/project/%(project)s/resources/', 'project_details': '%(hostname)s/api/2/project/%(project)s/?details', 'resource_details': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/', 'release_details': '%(hostname)s/api/2/project/%(project)s/release/%(release)s/', 'pull_file': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/?file', 'pull_pseudo_file': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/pseudo/?pseudo_type=MIXED', 'pull_reviewed_file': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/?file&mode=reviewed', 'pull_translator_file': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/?file&mode=translator', 'pull_developer_file': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/?file&mode=default', 'resource_stats': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/stats/', 'create_resource': '%(hostname)s/api/2/project/%(project)s/resources/', 'push_source': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/content/', 'push_translation': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/', 'delete_translation': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/translation/%(language)s/', 'formats': '%(hostname)s/api/2/formats/', 'delete_resource': '%(hostname)s/api/2/project/%(project)s/resource/%(resource)s/', } transifex-client-0.10/txclib/config.py0000644000175000001440000000666312234723375020672 0ustar mpessasusers00000000000000import ConfigParser class OrderedRawConfigParser( ConfigParser.RawConfigParser ): """ Overload standard Class ConfigParser.RawConfigParser """ def write(self, fp): """Write an .ini-format representation of the configuration state.""" if self._defaults: fp.write("[%s]\n" % DEFAULTSECT) for key in sorted( self._defaults ): fp.write( "%s = %s\n" % (key, str( self._defaults[ key ] ).replace('\n', '\n\t')) ) fp.write("\n") for section in self._sections: fp.write("[%s]\n" % section) for key in sorted( self._sections[section] ): if key != "__name__": fp.write("%s = %s\n" % (key, str( self._sections[section][ key ] ).replace('\n', '\n\t'))) fp.write("\n") optionxform = str _NOTFOUND = object() class Flipdict(dict): """An injective (one-to-one) python dict. Ensures that each key maps to a unique value, and each value maps back to that same key. Code mostly taken from here: http://code.activestate.com/recipes/576968-flipdict-python-dict-that-also-maintains-a-one-to-/ """ def __init__(self, *args, **kw): self._flip = dict.__new__(self.__class__) setattr(self._flip, "_flip", self) for key, val in dict(*args, **kw).iteritems(): self[key] = val @property def flip(self): """The inverse mapping.""" return self._flip def __repr__(self): return "%s(%r)" % (self.__class__.__name__, dict(self)) __str__ = __repr__ def copy(self): return self.__class__(self) @classmethod def fromkeys(cls, keys, value=None): return cls(dict.fromkeys(keys, value)) def __setitem__(self, key, val): k = self._flip.get(val, _NOTFOUND) if not (k is _NOTFOUND or k==key): raise KeyError('(key,val) would erase mapping for value %r' % val) v = self.get(key, _NOTFOUND) if v is not _NOTFOUND: dict.__delitem__(self._flip, v) dict.__setitem__(self, key, val) dict.__setitem__(self._flip, val, key) def setdefault(self, key, default = None): # Copied from python's UserDict.DictMixin code. try: return self[key] except KeyError: self[key] = default return default def update(self, other = None, **kwargs): # Copied from python's UserDict.DictMixin code. # Make progressively weaker assumptions about "other" if other is None: pass elif hasattr(other, 'iteritems'): # iteritems saves memory and lookups for k, v in other.iteritems(): self[k] = v elif hasattr(other, 'keys'): for k in other.keys(): self[k] = other[k] else: for k, v in other: self[k] = v if kwargs: self.update(kwargs) def __delitem__(self, key): val = dict.pop(self, key) dict.__delitem__(self._flip, val) def pop(self, key, *args): val = dict.pop(self, key, *args) dict.__delitem__(self._flip, val) return val def popitem(self): key, val = dict.popitem(self) dict.__delitem__(self._flip, val) return key, val def clear(self): dict.clear(self) dict.clear(self._flip) transifex-client-0.10/txclib/__init__.py0000644000175000001440000000044512247615003021144 0ustar mpessasusers00000000000000# -*- coding: utf-8 -*- VERSION = (0, 10, 0, 'final') def get_version(): version = '%s.%s' % (VERSION[0], VERSION[1]) if VERSION[2]: version = '%s.%s' % (version, VERSION[2]) if VERSION[3] != 'final': version = '%s %s' % (version, VERSION[3]) return version transifex-client-0.10/transifex_client.egg-info/0000755000175000001440000000000012247615466022613 5ustar mpessasusers00000000000000transifex-client-0.10/transifex_client.egg-info/SOURCES.txt0000644000175000001440000000223012247615466024474 0ustar mpessasusers00000000000000LICENSE MANIFEST.in README.rst setup.py tx transifex_client.egg-info/PKG-INFO transifex_client.egg-info/SOURCES.txt transifex_client.egg-info/dependency_links.txt transifex_client.egg-info/not-zip-safe transifex_client.egg-info/top_level.txt txclib/__init__.py txclib/commands.py txclib/config.py txclib/exceptions.py txclib/log.py txclib/parsers.py txclib/paths.py txclib/processors.py txclib/project.py txclib/urls.py txclib/utils.py txclib/web.py txclib/packages/__init__.py txclib/packages/urllib3/__init__.py txclib/packages/urllib3/_collections.py txclib/packages/urllib3/connectionpool.py txclib/packages/urllib3/exceptions.py txclib/packages/urllib3/fields.py txclib/packages/urllib3/filepost.py txclib/packages/urllib3/poolmanager.py txclib/packages/urllib3/request.py txclib/packages/urllib3/response.py txclib/packages/urllib3/util.py txclib/packages/urllib3/contrib/__init__.py txclib/packages/urllib3/contrib/ntlmpool.py txclib/packages/urllib3/contrib/pyopenssl.py txclib/packages/urllib3/packages/__init__.py txclib/packages/urllib3/packages/ordered_dict.py txclib/packages/urllib3/packages/six.py txclib/packages/urllib3/packages/ssl_match_hostname/__init__.pytransifex-client-0.10/transifex_client.egg-info/not-zip-safe0000644000175000001440000000000112144417710025025 0ustar mpessasusers00000000000000 transifex-client-0.10/transifex_client.egg-info/top_level.txt0000644000175000001440000000000712247615466025342 0ustar mpessasusers00000000000000txclib transifex-client-0.10/transifex_client.egg-info/PKG-INFO0000644000175000001440000000346612247615466023721 0ustar mpessasusers00000000000000Metadata-Version: 1.1 Name: transifex-client Version: 0.10 Summary: A command line interface for Transifex Home-page: https://www.transifex.com Author: Transifex Author-email: admin@transifex.com License: GPLv2 Description: ============================= Transifex Command-Line Tool ============================= The Transifex Command-line Client is a command line tool that enables you to easily manage your translations within a project without the need of an elaborate UI system. You can use the command line client to easily create new resources, map locale files to translations and synchronize your Transifex project with your local repository and vice verca. Translators and localization managers can also use it to handle large volumes of translation files easily and without much hassle. Check the full documentation at http://support.transifex.com/customer/portal/topics/440187-transifex-client/articles. Installing ========== You can install the latest version of transifex-client running ``pip install transifex-client`` or ``easy_install transifex-client`` You can also install the `in-development version`_ of transifex-client with ``pip install transifex-client==dev`` or ``easy_install transifex-client==dev``. .. _in-development version: http://github.com/transifex/transifex-client/tarball/master#egg=transifex-client-dev Keywords: translation,localization,internationalization Platform: UNKNOWN Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 transifex-client-0.10/transifex_client.egg-info/dependency_links.txt0000644000175000001440000000000112247615466026661 0ustar mpessasusers00000000000000 transifex-client-0.10/MANIFEST.in0000644000175000001440000000014212247321676017310 0ustar mpessasusers00000000000000include tx include txclib/cacert.pem # Docs include LICENSE README.rst recursive-include docs *