ZL iP DdZddlZddlmZmZmZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZddlmZmZdeed eeeeffd Zd eeefd efd Zd eeeefd ed eeeeffdZd eeeefdeded eeeeffdZded eeeefd dfdZ de d efdZ!GddedZ"dede#ed dfdZ$de d dfdZ%dee de d dfdZ&de d efdZ'd e e ee fd e#efd!Z(eGd"d#Z)deded ee)fd$Z*d%ed&ee)d'ed efd(Z+y))z?Functions related to Black's formatting by line ranges feature.N) CollectionIteratorSequence) dataclass)Union) LNSTANDALONE_COMMENTLeafNodeVisitor first_leaf furthest_ancestor_with_last_leaf last_leafsyms)ASYNCNEWLINE line_rangesreturnc g}|D]_}|jd}t|dk7rtd| t|d}t|d}|j ||fa|S#t$rtd|dwxYw)N-z:Incorrect --line-ranges format, expect 'START-END', found rz|S)aReturns the valid line ranges for the given source. This removes ranges that are entirely outside the valid lines. Other ranges are normalized so that the start values are at least 1 and the end values are at most the (1-based) index of the last source line.  r)countendswithmaxminr)rr) good_linessrc_line_countr!r"s r#sanitized_linesr21s  J!''-N   &!( s > ! E1  ; #~&5#,'( r(original_sourcemodified_sourcect||}g}d}t|D]\}}t|||}t|||} |}|t|k\s| t|k\r?||} || } | jr | j } n|| j z | j z} | jr | j} n|| j z | j z} | | f}t|s|j||S)aReturns the adjusted line ranges based on edits from the original code. This computes the new line ranges by diffing original_source and modified_source, and adjust each range based on how the range overlaps with the diffs. Note the diff can contain lines outside of the original line ranges. This can happen when the formatting has to be done in adjacent to maintain consistent local results. For example: 1. def my_func(arg1, arg2, 2. arg3,): 3. pass If it restricts to line 2-2, it can't simply reformat line 2, it also has to reformat line 1: 1. def my_func( 2. arg1, 3. arg2, 4. arg3, 5. ): 6. pass In this case, we will expand the line ranges to also include the whole diff block. Args: lines: a collection of line ranges. original_source: the original source. modified_source: the modified source. r) _calculate_lines_mappingssorted_find_lines_mapping_indexris_changed_blockmodified_startoriginal_start modified_endr'r)rr3r4lines_mappings new_linescurrent_mapping_indexr!r"start_mapping_indexend_mapping_index start_mapping end_mapping new_startnew_end new_ranges r#adjusted_linesrGMs1J/PNIUm!( s7   !  6     !4 #n"5 59Jc O :  &':; $%67  ) )%44I 444}7S7SS   ' '!..GK6669S9SSG( y )   Y 'C!(D r(src_nodect}|D]#\}}|jt||dz%t|}t |j |}t ||y)aConverts unchanged lines to STANDALONE_COMMENT. The idea is similar to how `# fmt: on/off` is implemented. It also converts the nodes between those markers as a single `STANDALONE_COMMENT` leaf node with the unformatted code as its value. `STANDALONE_COMMENT` is a "fake" token that will be formatted as-is with its prefix normalized. Here we perform two passes: 1. Visit the top-level statements, and convert them to a single `STANDALONE_COMMENT` when unchanged. This speeds up formatting when some of the top-level statements aren't changed. 2. Convert unchanged "unwrapped lines" to `STANDALONE_COMMENT` nodes line by line. "unwrapped lines" are divided by the `NEWLINE` token. e.g. a multi-line statement is *one* "unwrapped line" that ends with `NEWLINE`, even though this statement itself can span multiple lines, and the tokenizer only sees the last '\n' as the `NEWLINE` token. NOTE: During pass (2), comment prefixes and indentations are ALWAYS normalized even when the lines aren't changed. This is fixable by moving more formatting to pass (1). However, it's hard to get it correct when incorrect indentations are used. So we defer this to future optimizations. rN)setupdaterange_TopLevelStatementsVisitorlistvisit_convert_unchanged_line_by_line)rHr lines_setr!r"visitor_s r#convert_unchanged_linesrTs^0%I0 sucAg./0(3G W]]8 $%A#Hi8r(nodect|tr|jtk(S|jD]}t |syy)NTF) isinstancer typer children_contains_standalone_comment)rUchilds r#rZrZs?$yy...]] E+E2 r(cNeZdZdZdeefdZdededfdZ dededfdZ y) rMa A node visitor that converts unchanged top-level statements to STANDALONE_COMMENT. This is used in addition to _convert_unchanged_line_by_line, to speed up formatting when there are unchanged top-level classes/functions/statements. rQc||_yN) _lines_set)selfrQs r#__init__z#_TopLevelStatementsVisitor.__init__s #r(rUrNc#KgEd{t|}|sy|jtk(sJd|jt|}t |j |j s t|yy7uw)NzUnexpectedly found leaf.type=)rrXrr_get_line_range intersectionr_#_convert_node_to_standalone_comment)r`rU newline_leafancestors r#visit_simple_stmtz,_TopLevelStatementsVisitor.visit_simple_stmts        ( ? *<+<+<*= > ? ( 4LAx(55dooF / 9G sBA>A6Bc#(KgEd{t|ry|j}|5|j)|jjtk(r |j}|1t |j |js t|yyy7wr^) rZparent prev_siblingrXrrcrdr_re)r`rUsemantic_parents r# visit_suitez&_TopLevelStatementsVisitor.visit_suites ( - ++  &,,8#0055>"1"8"8  & 0 ,t '0( 0 @0( &# sBBBB) __name__ __module__ __qualname____doc__rJrrar rrhrmr&r(r#rMrMsH$#c($:d:x~:"AA$Ar(rMrQc|jD]c}|jtk7r|jr|jjtj k(rZg}|j }|r!|jd||j }|r!t|j|rt|||jr$|jjtjk(r|jj }g}|r[|jtjk7r>|jd||j }|r|jtjk7r>|jj}|E|j 9|j jtk(r|jd|j t|j|rt||t|}|jtjk(r?|jr3|jjtjk(r |j}t|j|rYt!|fy)z6Converts unchanged to STANDALONE_COMMENT line by line.r)newlineN)leavesrXrrjr match_stmtrkinsertrcrd$_convert_nodes_to_standalone_commentsuiterr decorator decoratorsre)rUrQleafnodes_to_ignorerkparent_sibling grandparentrgs r#rPrPs 1> 99   ;;4;;++t> )+O,,L&&q,7+88 #?3@@K4_dS [[T[[--;"[[55N O ^%8%8DJJ%F&&q.9!/!r(c |j}|syt|}t|}|r|sy||ury|j}d|_|j }||j t jk(r2t|tsJ|jdttdt|dd}|j|tt|||yy)z@Convert node to STANDALONE_COMMENT by modifying the tree inline.Nrr+prefixfmt_pass_converted_first_leaf)rjr rrremoverXr decoratedrWr insert_childr rstrr )rUrjfirstlastrindexvalues r#rere3s [[F  t E T?D  }  \\FEL KKME  99 &dD) ))   agt!4 5D #2  ".3   #r(nodesrsc |sy|dj}t|d}|r|sy|j}d|_djd|D}|jr||jz }d|_|dj }|ddD]}|j |$|j |t t|||yy)zAConvert nodes to STANDALONE_COMMENT by modifying the tree inline.Nrrc32K|]}t|ywr^)r).0rUs r# z7_convert_nodes_to_standalone_comment..rs0$CI0srr)rjr rjoinrrr r )rrsrjrrrrrUs r#rwrwhs  1X__F uQx E  \\FEL GG0%0 0E~~  !HOO Eab     ".3   r(r{c|jtk(r |jS|jt|j dzS)z5Returns the line number of the leaf node's last line.r+)rXrlinenorr,)r{s r#_leaf_line_endrs6 yyG{{{{SY__T222r( node_or_nodesct|trg|}|s tSt|d}t |d}|r1|r/|j }t |}tt||dzStS|}t|tr+tt|j t |dzSt|}t |}|r-|r+tt|j t |dzStS)z5Returns the line range of this node or list of nodes.rrr) rWrNrJr rrrrLr )rrrr line_startline_endrUs r#rcrcs-&5L58$r# TJ%d+HuZA67 75L dD !uT[[.*>*BCD Dt$ET?D5~d/Ca/GHIIu r(cDeZdZUdZeed<eed<eed<eed<eed<y) _LinesMappingz1-based lines mapping from original source to modified source. Lines [original_start, original_end] from original source are mapped to [modified_start, modified_end]. The ranges are inclusive on both ends. r; original_endr:r<r9N)rnrorprqr__annotations__boolr&r(r#rrs'r(rc @tjd|jd|jd}|j}g}t |D]E\}}|dk(rR|j dk7s|j dk7r|jtd|j d|j dno||dz }|jt|j |jzdz|j |j |jzdz|j d|t|dz ks|jt|j dz|j |jz|j dz|j |jzdH|S)aReturns a sequence of _LinesMapping by diffing the sources. For example, given the following diff: import re - def func(arg1, - arg2, arg3): + def func(arg1, arg2, arg3): pass It returns the following mappings: original -> modified (1, 1) -> (1, 1), is_changed_block=False (the "import re" line) (2, 3) -> (2, 2), is_changed_block=True (the diff) (4, 4) -> (3, 3), is_changed_block=False (the "pass" line) You can think of this visually as if it brings up a side-by-side diff, and tries to map the line ranges from the left side to the right side: (1, 1)->(1, 1) 1. import re 1. import re (2, 3)->(2, 2) 2. def func(arg1, 2. def func(arg1, arg2, arg3): 3. arg2, arg3): (4, 4)->(3, 3) 4. pass 3. pass Args: original_source: the original source. modified_source: the modified source. NT)keependsrrF)r;rr:r<r9) difflibSequenceMatcher splitlinesget_matching_blocks enumerateabrrsizer)r3r4matchermatching_blocksr=iblockprevious_blocks r#r6r6s<%% ""D"1""D"1G 113O*,No. 5 6ww!|uww!|%%!'(%*WW'(%*WW). -QU3N  ! !#1#3#3n6I6I#IA#M!&#1#3#3n6I6I#IA#M!&%)   s?#a' '  ! !#(77Q;!&5::!5#(77Q;!&5::!5%*  1 B r( original_liner= start_indexc|}|t|kr;||}|j|cxkr|jkr|S|dz }|t|kr;|S)zGReturns the original index of the lines mappings for the original line.r)rr;r)rr=rrmappings r#r8r8 sc E #n% % '  ! !] Jg6J6J JL K   #n% % Lr(),rqrcollections.abcrrr dataclassesrtypingr black.nodesrr r r r r rrrblib2to3.pgen2.tokenrrrrNtuplerr$rr'r2rGrTrZrMrJrPrerwrrcrr6r8r&r(r#rs2E::!   08C=T%S/5J,-uS#X-4-  eCHo &69c3h 8M eCHo &MMM %S/ M`9d9:eCHo3N9SW9@rd3A3Al3>$3>3s83>3>l2 b2 T2 j   $ SW <33#35T"X#63s86  "LLLmL^  ]+   r(