dni1|dZddlZddlZddlZddlmZddlmZmZm Z m Z m Z ddl m Z ddl mZGddZdS) z Directory Sync Engine - Unidirectional Mirror Synchronization Mirrors source directory to destination using mtime+size comparison N)Path)OptionalCallableDictAnyList)datetime)loggerc eZdZdZ ddededeedefdZdd ee d e ee ffd Z d e d efd Zdde ded e e e ffdZde de d efdZde de fdZde e e fd efdZde fdZdS) DirectorySynca! Unidirectional directory synchronization engine Mirrors source -> destination: - Copies new/modified files (based on mtime + size) - Deletes files from destination that don't exist in source - Preserves directory structure - Handles network paths (e.g., Z:\) NFsource destinationexclude_patternsprotect_excludedc t||_t||_|pg|_||_|jst d||jst d||jdd|jr2tj d|d|dt|jdd Stj d|d|d S) a Initialize sync engine Args: source: Source directory path (can be network path) destination: Destination directory path (can be network path) exclude_patterns: List of patterns to exclude (e.g., ['venv/', '*.log', '__pycache__/']) protect_excluded: If True, don't delete excluded files in destination (for deploy scenarios) If False, delete excluded files from destination (for clean mirror) zSource path does not exist: z Source path is not a directory: Tparentsexist_okzDirectorySync initialized: z -> z (excluding z patterns)N) rr rrrexists ValueErroris_dirmkdirr infolen)selfr rrrs /C:\PYTHON\Scheduler\modules\sync\sync_engine.py__init__zDirectorySync.__init__sK6ll  ,, 0 6B 0{!!## FDFDDEE E{!!## JHHHII I td;;;   Q KBfBB+BB[^_c_t[u[uBBB C C C C C KOfOO+OO P P P P Pprogress_callbackreturnc |ddddgd} |r|ddddd||j}t||d<tjdt|d|r|d dt|ddt |D]\}\}}|j|z } |||rM| |||d xxd z cc<|d xx| j z cc<|r-|d |d zt|t|d#t$rA}d |d|} tj| |d| Yd}~d}~wwxYw|r|ddddd||} | |d<tjd|d d|ddnG#t$r:}d|} tj| |d| d}~wwxYw|S)a Perform synchronization Args: progress_callback: Optional callback for progress updates Called with dict: {'phase': str, 'current': int, 'total': int, 'file': str} Returns: Dict with statistics: { 'files_scanned': int, 'files_copied': int, 'files_deleted': int, 'bytes_total': int, 'errors': List[str] } r) files_scanned files_copied files_deleted bytes_totalerrorsscanning)phasecurrenttotalfiler"zScanned z files in sourcesyncingr#r%zError syncing : r&Ncleaningr$zSync completed: z copied, z deletedzCritical sync error: )_scan_directoryr rr r enumerateitemsr _needs_sync _copy_filestatst_sizestr Exceptionerrorappend_cleanup_destination exception) rrstats source_filesidxrel_path source_file dest_filee error_msg deleted_counts rsynczDirectorySync.sync9s&   1   _!!J1qZ\"]"]^^^// <#>,%0D0DX0N0N%*a/N$-1**#_5!!!'C'C'C'C'CDDD ! ! !    LEYEE!EE F F F   A   KVNVVV W W WsL+B$1A*#B$$A*)B$*B;BB$BB$$ C.C  CrBrCcR|sdS |}|}|j|jkrtjd|jdSdS#t tf$r%}tjd|d|Yd}~dSd}~wwxYw)ax Check if file needs to be synced using size comparison only Note: mtime comparison disabled because WebDAV/network drives don't preserve timestamps correctly with shutil.copy2() Args: source_file: Source file path dest_file: Destination file path Returns: True if file needs to be copied TzSize mismatch: FzCannot compare r/N) rr6r7r debugrSrarbrc)rrBrC source_stat dest_statrDs rr4zDirectorySync._needs_syncs!! 4 %**,,K!((I"i&777 A{/?AABBBt5)    N?[??A?? @ @ @44444 sAA00B&B!!B&c & |jddtj||t jd|jdS#tttj f$r#}t j d|d|d|d}~wwxYw)z Copy file from source to destination, preserving metadata Args: source_file: Source file path dest_file: Destination file path TrzCopied: zFailed to copy z to r/N) parentrshutilcopy2r rhrSrarbErrorr:)rrBrCrDs rr5zDirectorySync._copy_files    " "4$ " ? ? ? Li 0 0 0 L6K$466 7 7 7 7 7&,7    LL;LLILLLL M M M  sA AB-B  Br?c bd}d} ||jd}|D]"}||vr|jr||r|dz }*|j|z } |t jd||dz }f#t$r tj |d|t jd||dz }n8#ttf$r$}t j d|d |Yd }~nd }~wwxYwYt$r%}t j d|d |Yd }~d }~wwxYw$|dkrt j d |d | |jn.#t$r!}t j d |Yd }~nd }~wwxYw|S)a Delete files from destination that don't exist in source Behavior depends on protect_excluded flag: - protect_excluded=True: Don't delete files matching exclusion patterns (for deploy) - protect_excluded=False: Delete all files not in source, including excluded (clean mirror) Args: source_files: Dict of relative paths from source Returns: Number of files deleted rF)r[r.z Deleted: izDeleted (forced): zFailed to delete r/NzSkipped deletion of z6 excluded files in destination (protect_excluded=True)zError during cleanup: )r1rrrYunlinkr rhrboschmodrar:r_cleanup_empty_dirsr9)rr?rFskipped_excluded dest_filesrArCrDs rr<z"DirectorySync._cleanup_destination sq % 7--d.>QV-WWJ' K K<//,!1E1Eh1O1O!(A-( $ 08 ;IK!((*** %;%;%;<<<%* *OOOOHY666%,,..."L)Hh)H)HIII)Q.MM '9OOO"L)MY)M)M!)M)MNNNNNNNNO"KKK %I%I%Ia%I%IJJJJJJJJK+00 !## {3C{{{|||  $ $T%5 6 6 6 6 7 7 7 L5!55 6 6 6 6 6 6 6 6 7syAF0B  F EACED,D E DEF ED>8F>E=F F, F''F,c tjt|dD]\}}}t|}||kr t |s+|tjd|j#ttf$rY}wxYwdS#t$r"}tj d|Yd}~dSd}~wwxYw)zy Remove empty directories from destination Args: directory: Root directory to clean F)topdownzRemoved empty directory: z"Error cleaning empty directories: N) rrwalkr8ranyiterdirrmdirr rhrarbr9rc)rrZdirpathdirnames filenamesdir_pathrDs rrtz!DirectorySync._cleanup_empty_dirsFs E02IPU0V0V0V  ,9==y((x//1122M ((( %K%K%KLLL1D   E E E NCCC D D D D D D D D D Es<?B(A BB(B# B("B##B(( C2CC)NF)N)T)__name__ __module__ __qualname____doc__r8rboolrrrrrrGrrYr1r4r5intr<rtrrr r sUY*/QQsQQS Q#'QQQQ>MMhx&8MDcNMMMM^$$$$$$L$$$$QUVZ\`V`Qa$$$$L!t!!!!!!Fdt*8dDj1A8c8888tETEEEEEErr )rrrrmrRpathlibrtypingrrrrrr logurur r rrrrs  66666666666666PEPEPEPEPEPEPEPEPEPEr