Google
Edit File: refactor.pyo
� ۜSec @� s� d Z d d l m Z d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z m Z d d l m Z d d l m Z m Z d d l m Z d d l m Z e d � Z d e f d � � YZ d � Z d � Z d � Z d � Z e j! d d f k r[d d l" Z" e" j# Z$ d � Z% d � Z& n e# Z$ e Z% e Z& d � Z' d e f d � � YZ( d e) f d � � YZ* d e f d � � YZ+ d e* f d � � YZ, d S( s� Refactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. i����( t with_statements# Guido van Rossum <guido@python.org>N( t chaini ( t drivert tokenizet token( t find_root( t pytreet pygram( t btm_utils( t btm_matcherc C� s� t | g g d g � } t j j | j � } g } xb t t j | � � D]K } | j d � rI | j d � rI | r� | d } n | j | d � qI qI W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_s .pyi i����( t __import__t ost patht dirnamet __file__t sortedt listdirt startswitht endswitht append( t fixer_pkgt remove_prefixt pkgt fixer_dirt fix_namest name( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt get_all_fix_names! s t _EveryNodec B� s e Z RS( ( t __name__t __module__( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR . s c C� s� t | t j t j f � rC | j d k r3 t � n t | j g � St | t j � rt | j rk t | j � St � n t | t j � r� t � } x5 | j D]* } x! | D] } | j t | � � q� Wq� W| St d | � � d S( sf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR( 2 s c C� s t j t � } g } x� | D]� } | j r� y t | j � } Wn t k r^ | j | � q� XxU | D] } | | j | � qf Wq | j d k r� | | j j | � q | j | � q Wx: t t j j j � t j j � D] } | | j | � q� Wt | � S( s^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N( t collectionst defaultdictt listt patternR( R R t _accept_typeR$ R R t python_grammart symbol2numbert itervaluest tokenst extendt dict( t fixer_listt head_nodest everyt fixert headst node_type( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _get_headnode_dictN s" c C� s( g t | t � D] } | d | ^ q S( sN Return the fully qualified names for fixers in the package pkg_name. t .( R t False( t pkg_namet fix_name( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt get_fixers_from_packageg s c C� s | S( N( ( t obj( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _identityn s i i c C� s | j d d � S( Nu u ( t replace( t input( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _from_system_newlinesu s c C� s* t j d k r" | j d t j � S| Sd S( Ns u ( R t linesepRI ( RJ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _to_system_newlinesw s c � s� t } t j t j | � j � � � f d � } t t j t j t j f � } t � } ykxdt r�| � \ } } | | k r� q] q] | t j k r� | r� Pn t } q] | t j k r�| d k r�| � \ } } | t j k s� | d k r� Pn | � \ } } | t j k s| d k r Pn | � \ } } | t j k rY| d k rY| � \ } } n xa | t j k r�| j | � | � \ } } | t j k s�| d k r�Pn | � \ } } q\Wq] Pq] WWn t k r�n Xt | � S( Nc � s � j � } | d | d f S( Ni i ( t next( t tok( t gen( s( /usr/lib64/python2.7/lib2to3/refactor.pyt advance� s u fromu __future__u importu (u ,( RC R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR% t Truet STRINGt NAMEt OPt addt StopIteration( t sourcet have_docstringRQ t ignoret featurest tpt value( ( RP s( /usr/lib64/python2.7/lib2to3/refactor.pyt _detect_future_features� sD t FixerErrorc B� s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyRf � s t RefactoringToolc B� s! e Z i e d 6e d 6Z d Z d Z d d d � Z d � Z d � Z d � Z d � Z d � Z e e d � Z e e d � Z d � Z e e d � Z d � Z e d � Z d � Z d � Z d e d d � Z d d � Z d Z d Z d � Z d � Z d � Z d � Z d � Z d � Z RS( t print_functiont write_unchanged_filest FixR c C� s� | | _ | p g | _ | j j � | _ | d k rI | j j | � n | j d re t j | _ n t j | _ | j j d � | _ g | _ t j d � | _ g | _ t | _ t j | j d t j d | j �| _ | j � \ | _ | _ g | _ t j � | _ g | _ g | _ x} t! | j | j � D]f } | j" rT| j j# | � q2| | j k rv| j j$ | � q2| | j k r2| j j$ | � q2q2Wt% | j � | _& t% | j � | _' d S( s� Initializer. Args: fixer_names: a list of fixers to import options: an dict with configuration. explicit: a list of fixers to run even if they are explicit. Ri Rj Rh t convertt loggerN(( t fixerst explicitt _default_optionst copyt optionsR$ R* R t! python_grammar_no_print_statementt grammarR5 t getRj t errorst loggingt getLoggerRm t fixer_logRC t wroteR t DriverR Rl t get_fixerst pre_ordert post_ordert filest bmt BottomMatchert BMt bmi_pre_ordert bmi_post_orderR t BM_compatiblet add_fixerR RA t bmi_pre_order_headst bmi_post_order_heads( t selft fixer_namesRr Ro R> ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt __init__� s< c C� s� g } g } x�| j D]�} t | i i d g � } | j d d � d } | j | j � rr | t | j � } n | j d � } | j d j g | D] } | j � ^ q� � } y t | | � } Wn' t k r� t d | | f � � n X| | j | j � } | j r?| j t k r?| | j k r?| j d | � q n | j d | � | j d k rn| j | � q | j d k r�| j | � q t d | j � � q Wt j d � } | j d | � | j d | � | | f S( s Inspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. R RB i i����t _t s Can't find %s.%ss Skipping implicit fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rn R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRf Rr Ry Ro RY t log_messaget log_debugt orderR t operatort attrgettert sort( R� t pre_order_fixerst post_order_fixerst fix_mod_patht modRE t partsR. t class_namet fix_classR> t key_func( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR| � s8 / c O� s � d S( s Called when an error occurs.N( ( R� t msgt argst kwds( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt log_error s c G� s' | r | | } n | j j | � d S( s Hook to log a message.N( Rm t info( R� R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� s c G� s' | r | | } n | j j | � d S( N( Rm t debug( R� R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� s c C� s d S( sT Called with the old version, new version, and filename of a refactored file.N( ( R� t old_textt new_textt filenamet equal( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt print_output! s c C� sP xI | D]A } t j j | � r5 | j | | | � q | j | | | � q Wd S( s) Refactor a list of files and directories.N( R R t isdirt refactor_dirt refactor_file( R� t itemst writet doctests_onlyt dir_or_file( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt refactor& s c C� s� t j d } x� t j | � D]� \ } } } | j d | � | j � | j � xe | D]] } | j d � rW t j j | � d | k rW t j j | | � } | j | | | � qW qW Wg | D] } | j d � s� | ^ q� | (q Wd S( s� Descends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. t pys Descending into %sRB i N( R t extsept walkR� R� R R t splitextR� R� ( R� t dir_nameR� R� t py_extt dirpatht dirnamest filenamesR t fullnamet dn( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� / s c C� s� y t | d � } Wn' t k r< } | j d | | � d SXz t j | j � d } Wd | j � Xt | d d | �� } t | j � � | f SWd QXd S( sG Do our best to decode a Python source file correctly. t rbs Can't open %s: %si NR- t encoding( NN( t opent IOErrorR� R$ R t detect_encodingRT t closet _open_with_encodingRK t read( R� R� t ft errR� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _read_python_sourceC s c C� s | j | � \ } } | d k r% d S| d 7} | r� | j d | � | j | | � } | j sl | | k r� | j | | | | | � q� | j d | � nc | j | | � } | j s� | r� | j r� | j t | � d | d | d | �n | j d | � d S( s Refactors a file.Nu s Refactoring doctests in %ss No doctest changes in %si����R� R� s No changes in %s( R� R$ R� t refactor_docstringRj t processed_filet refactor_stringt was_changedt unicode( R� R� R� R� RJ R� t outputt tree( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� S s c C� s� t | � } d | k r* t j | j _ n zM y | j j | � } Wn0 t k ru } | j d | | j j | � d SXWd | j | j _ X| | _ | j d | � | j | | � | S( sF Refactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. Ri s Can't parse %s: %s: %sNs Refactoring %s( Re R Rs R Rt t parse_stringR+ R� t __class__R t future_featuresR� t refactor_tree( R� t dataR Rb R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� j s c C� s� t j j � } | ro | j d � | j | d � } | j sI | | k r_ | j | d | � q� | j d � nS | j | d � } | j s� | r� | j r� | j t | � d | � n | j d � d S( Ns Refactoring doctests in stdins <stdin>s No doctest changes in stdins No changes in stdin( t syst stdinR� R� R� Rj R� R� R� R� ( R� R� RJ R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt refactor_stdin� s c C� s� x- t | j | j � D] } | j | | � q W| j | j | j � � | j | j | j � � | j j | j � � } x�t | j � � rcx�| j j D]�} | | k r� | | r� | | j d t j j d t � | j r� | | j d t j j � n x[t | | � D]F} | | | k r9| | j | � n y t | � Wn t k r]qn X| j r|| | j k r|qn | j | � } | r| j | | � } | d k rU| j | � x9 | j � D]+ } | j s�g | _ n | j j | � q�W| j j | j � � } x? | D]4 } | | k r6g | | <n | | j | | � qWqUqqWq� q� Wq} Wx- t | j | j � D] } | j | | � qzW| j S( s� Refactors a parse tree (modifying the tree in place). For compatible patterns the bottom matcher module is used. Otherwise the tree is traversed node-to-node for matches. Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. R� t reverseN(! R R} R~ t start_treet traverse_byR� R� R� t runt leavest anyt valuesRn R� R t Baset depthRY t keep_line_ordert get_linenoR2 t removeR t ValueErrort fixers_appliedt matcht transformR$ RI R R9 t finish_treeR� ( R� R� R R> t match_sett nodet resultst newt new_matchest fxr( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � sJ . c C� s� | s d Sxr | D]j } xa | | j D]R } | j | � } | r% | j | | � } | d k rw | j | � | } qw q% q% Wq Wd S( s Traverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N( R# R� R� R$ RI ( R� Rn t traversalR� R> R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � s c C� s� | j j | � | d k rB | j | � d } | d k rB d Sn | | k } | j | | | | � | r� | j d | � | j s� d Sn | r� | j | | | | � n | j d | � d S( sR Called when a file has been refactored and there may be changes. i Ns No changes to %ss Not writing changes to %s( R R R$ R� R� R� Rj t write_file( R� R� R� R� R� R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � s c C� s� y t | d d | �} Wn* t j k rE } | j d | | � d SXzG y | j t | � � Wn) t j k r� } | j d | | � n XWd | j � X| j d | � t | _ d S( s� Writes a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. t wR� s Can't create %s: %sNs Can't write %s: %ss Wrote changes to %s( R� R t errorR� R� RM R� R� RY Rz ( R� R� R� R� R� R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� s s >>> s ... c C� s� g } d } d } d } d } x+| j t � D]} | d 7} | j � j | j � r� | d k r� | j | j | | | | � � n | } | g } | j | j � } | | } q. | d k r| j | | j � s� | | | j j � d k r| j | � q. | d k r/| j | j | | | | � � n d } d } | j | � q. W| d k rz| j | j | | | | � � n d j | � S( s� Refactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) i i u u N( R$ t splitlinesRY t lstripR t PS1R9 t refactor_doctestt findt PS2t rstripR R� ( R� RJ R� t resultt blockt block_linenot indentt linenot linet i( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� ( s: c C� sP y | j | | | � } Wnu t k r� } | j j t j � rm x* | D] } | j d | j d � � qG Wn | j d | | | j j | � | SX| j | | � rLt | � j t � } | | d | | d } } | d j d � s� | d c d 7<n | | j | j d � g } | rL| g | D] } | | j | ^ q(7} qLn | S( s� Refactors one doctest. A doctest is given as a block of lines, the first of which starts with ">>>" (possibly indented), while the remaining lines start with "..." (identically indented). s Source: %su s+ Can't parse docstring in %s line %s: %s: %si i����i ( t parse_blockR+ Rm t isEnabledForRw t DEBUGR� R R� R� R R� R� R� RY R R� t popR ( R� R R R R� R� R� R R� t clipped( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� S s$ .c C� s | j r d } n d } | j s4 | j d | � n1 | j d | � x | j D] } | j | � qN W| j r� | j d � x! | j D] } | j | � q� Wn | j rt | j � d k r� | j d � n | j d t | j � � x0 | j D]" \ } } } | j | | | � q� Wn d S( Nt weres need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rz R R� Ry Rv R� ( R� R t filet messageR� R� R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt summarizep s$ c C� s1 | j j | j | | | � � } t � | _ | S( s� Parses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. ( R t parse_tokenst wrap_toksRU R� ( R� R R R R� ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR � s ! c c� s� t j | j | | � j � } xe | D]] \ } } \ } } \ } } } | | d 7} | | d 7} | | | | f | | f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RR t gen_linesRN ( R� R R R R8 R# Rd t line0t col0t line1t col1t line_text( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR � s (c c� s� | | j } | | j } | } xi | D]a } | j | � rN | t | � Vn4 | | j � d k rl d Vn t d | | f � � | } q' Wx t r� d Vq� Wd S( s� Generates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. u s line=%r, prefix=%rR� N( R� R R R� R t AssertionErrorRY ( R� R R t prefix1t prefix2t prefixR ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR � s N( R R RC Rp R� R� R$ R� R| R� R� R� R� R� R� R� R� R� R� R� R� R� R� R� R R� R� R R R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyRh � s: 4 ( O + t MultiprocessingUnsupportedc B� s e Z RS( ( R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR � s t MultiprocessRefactoringToolc B� s5 e Z d � Z e e d d � Z d � Z d � Z RS( c O� s/ t t | � j | | � d | _ d | _ d S( N( t superR R� R$ t queuet output_lock( R� R� t kwargs( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � s i c C� sf | d k r( t t | � j | | | � Sy d d l } Wn t k rQ t � n X| j d k rp t d � � n | j � | _ | j � | _ g t | � D] } | j d | j � ^ q� } z; x | D] } | j � q� Wt t | � j | | | � Wd | j j � x$ t | � D] } | j j d � qWx' | D] } | j � r5| j � q5q5Wd | _ Xd S( Ni i����s already doing multiple processest target( R! R R� t multiprocessingt ImportErrorR R" R$ t RuntimeErrort JoinableQueuet LockR# t xranget Processt _childt startR� t putt is_alive( R� R� R� R� t num_processesR& R t processesR. ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � s2 + c C� so | j j � } xY | d k rj | \ } } z t t | � j | | � Wd | j j � X| j j � } q Wd S( N( R" Ru R$ R! R R� t task_done( R� t taskR� R$ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR- � s c O� sE | j d k r( | j j | | f � n t t | � j | | � Sd S( N( R" R$ R/ R! R R� ( R� R� R$ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR� � s ( R R R� RC R� R- R� ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR � s (- Rg t __future__R t __author__R R� Rw R� R0 RS t itertoolsR t pgen2R R R t fixer_utilR R� R R R t buR R� RY R R+ R R( RA RF RH t version_infot codecsR� R� RK RM Re Rf t objectRh R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt <module> sF (� �