;ς ¦,£Cc@sΘdkZdkZdkZdklZdklZlZlZdk Tdk Td„Z d„Z d„Z d„Zd„Zd „Zd „Zd „Zd „Zd „Zd„Zd„Zd„ZdS(N(surlparse(s urlretrieves urlcleanupsurlopen(s*cCs^t|ƒdjo|ddjodSnt|ddƒ\}}}}}}|djSdS(sBReturn non-zero when "name" is a URL, zero when it's a local file.iis:isN( slensnamesurlparsesschemesmachspathsparmsquerysfrag(snamesfragsparmsqueryspathsschemesmach((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysis_urls $$cCsΌt|ƒdjo|ddjodd|fSnt|ddƒ\}}}}}}||||}|djo |djoVt i ||ƒt|ƒ}|t|ƒjo||djo||d}qΫn|djo|djo|d djoO|d}d}t i |dƒ}|djo||d}|| }qUn|d jo d}n|d jo!t|ƒdjo|ddjo|d}n|||fSd S( s*Split a URL into scheme, machine and path.iis:sisrsyncs/s//sfilesftpN( slensnamesurlparsesschemesmachspathsparmsquerysfragsstringsindexsisfind(snamesfragsisparmsqueryspathsschemesmach((s+/mnt/gmirror/ports/devel/aap/work/Remote.pys url_split3s*$$$+    1cCs“t|ƒoFdkl}|||ƒ}|o|iƒ}q‹t||ƒ}n9yt i i |ƒ}Wnt t fj o d}nX|SdS(s‹Obtain the timestamp in seconds (in GMT if possible) for the URL "name". Returns zero (very old) if the timestamp can't be obtained.(s cache_lookupiN(sis_urlsnamesCaches cache_lookupsrecdictscs timestampsts remote_timesosspathsgetmtimesIOErrorsOSError(srecdictsnamescs cache_lookupst((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysurl_time@s   cCsšykt|tdƒ|ƒt|ƒ}t|iƒƒ}|i ƒ|djot|tdƒ|ƒnWn$t|tdƒ|ƒd}nX|SdS(s#Get the timestamp of a remote file.sgetting timestamp for "%s"is&"%s" can be found but has no timestamps#Could not obtain timestamp for "%s"iN( smsg_infosrecdicts_snamesurlopensupsget_header_datesinfostsclose(srecdictsnamesupst((s+/mnt/gmirror/ports/devel/aap/work/Remote.pys remote_timeUs    cCskdkl}|idƒoti||dƒƒSn|idƒoti||dƒƒSndSdS(sBGet the date from a MIME header. Returns zero when not available.(s parsedates Last-ModifiedsDateiN(srfc822s parsedatesheadersshas_keystimesmktime(sheaderss parsedate((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysget_header_dateds cCst|dddƒSdS(sQ Use $RSYNC if defined, otherwise use "rsync -p --rsh=ssh --copy-links". sRSYNCsrsyncs -p --rsh=ssh --copy-linksN(s get_prognamesrecdict(srecdict((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysget_progname_rsyncqscCst|dddƒSdS(s9 Use $SCP if defined, otherwise use "scp -C -p". sSCPsscps -C -pN(s get_prognamesrecdict(srecdict((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysget_progname_scpwscCst|dddƒSdS(s6 Use $RCP if defined, otherwise use "rcp -p". sRCPsrcps -pN(s get_prognamesrecdict(srecdict((s+/mnt/gmirror/ports/devel/aap/work/Remote.pysget_progname_rcp}scCsυt|td|ƒƒd}t|ƒ\}}}|i dƒ}|t j o|i d|ƒ}nt }|p1|djp$|djp|djp |djo1|d jod kl}|ƒ} qΨ|} n|oDt||||| fƒ}| ottd ƒ||f‚q’n€|djo]d kl}d kl} || |ƒ|dƒdt|ƒ||| f}t||ƒn|djo{d kl}d kl} || |ƒ|dƒdt|ƒ||| f}tidjod|}nt||ƒnŽ|djo{d kl}d kl} || |ƒ|dƒdt|ƒ||| f}tidjod|}nt||ƒn|djo dkl }d } yt"| dƒ}Wn+t$j o}tdƒ| |f} nX| o™||ƒ\}} | oYdk'} |i(dƒy|i)d||i*dƒWqy| i+j o}|} qyXn|i,ƒ| oti-| ƒqŸn| o t| ‚q’nμ|d jot.|ƒ\} }nDt.||ƒ\} }| |jot0i1| |ƒ|} nt2ƒt"| ƒ} | i4dƒ}| i,ƒdk6}|i7d||i8ƒo't9| ƒttdƒ||f‚n|ot:|ƒ}n|d jot|td|ƒƒnt|td||fƒƒ| |fSdS(seAttempt downloading file "url" to file "fname". Overwrite "fname" if it already exists. When "fname" is empty, use a temporary file. The caller has to use "url_cleanup()" when done with it. Returns a tuple of the filename and the timestamp of the remote file when possible. Throws an IOError if downloading failed.sAttempting download of "%s"is_nosfetch_srcpsscpsrsyncsftps(s tempfnames fetch_%s() could not download %s(s assert_pkg(s getrpstacks %s %s:%s %ssposixs{interactive} (s ftpConnectswbs Cannot open "%s" for writing: %sNisRETR i iθs\s*404\s*not\s*founds)���fetch_%s() encountered a 404 error for %ss���Downloaded "%s"s���Downloaded "%s" to "%s"(;���s���msg_infos���recdicts���_s���urls���rtimes ���url_split3s���fschemes���fmachs���fpaths���gets���scopes���Nones ���user_funcs���fnames ���RecPythons ���tempfnames���resfiles���applys���ress���IOErrors ���DoInstalls ���assert_pkgs���Works ���getrpstacks���get_progname_rcps���cmds ���logged_systems���get_progname_scps���oss���names���get_progname_rsyncs���CopyMoves ���ftpConnects���msgs���opens���ftpfiles ���StandardErrors���es���ftps���ftplibs���set_pasvs ���retrbinarys���writes ���all_errorss���closes���removes ���urlretrieves���hs���shutils���copy2s ���urlcleanups���fs���reads���txts���res���searchs ���IGNORECASEs ���try_deletes���get_header_date(���s���recdicts���urls���fnames ���ftpConnects���ftps ���tempfnames���ress���fschemes���res���msgs ���getrpstacks���ftplibs���fs���resfiles���txts���scopes���rtimes���es���ftpfiles���hs���cmds���fpaths ���user_funcs���fmachs ���assert_pkg(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys ���url_download„���s �����;    !                      c���������C���s#���|��d�d�g�j� o �t�ƒ��n�d�S(���s6���Cleanup after using url_download with scheme "scheme".s���scps���rsyncN(���s���schemes ���urlcleanup(���s���scheme(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys ���url_cleanup��s�����c���������C���s2��d�k��l�} �d�k�l�}�|�d�} �|�| �ƒ�\�}�}�|�d�j�p �t�| �ƒ� o �d�}�n�|� ož�t �ƒ��o%�t �|��t �d�ƒ�|�i�ƒ��ƒ�d�Sn�y�t�|��| �|�i�ƒ�\�} �} �WnE�t�j �o9�} �t�|��t �d�ƒ�| �|�i�ƒ��t�| �ƒ�f�ƒ�d�Sn�Xd�Sn�|�i�d �ƒ�o�|�d �}�n�t�}�| �|��| �|�ƒ�\�}�}�|�o�t�i�i�|�ƒ�oΣ�t �ƒ��o!�t �|��t �d �ƒ�|�i�ƒ��ƒ�n‘�y�t�i �|�|�i�ƒ�Wn;�t!�j �o/�} �t"�t �d �ƒ�|�|�i#�f�t�| �ƒ�‚�n�X|�o!�t �|��t �d �ƒ�|�i�ƒ��ƒ�n$�t �|��t �d �ƒ�|�|�i�ƒ��f�ƒ�d�Sn�d�Sd�S(���sψ���Download a file according to "url_dl" and copy it over "node.name". Use the cache when "use_cache" is non-zero, otherwise obtain a fresh copy. Can also be used for a local file, it is copied. Return non-zero for success.(���s ���local_name(���s���separate_schemes���names���filei���s���Skip download for "%s"s ���Cannot download "%s" to "%s": %si����s ���cache_updates���Skip copy file: "%s"s���Cannot copy "%s" to "%s": s���Copied file from cache: "%s"s���Copied file "%s" to "%s"N($���s���Caches ���local_names���VersConts���separate_schemes���url_dls���urls���schemes���fnames���is_urls ���use_caches ���skip_commandss���msg_infos���recdicts���_s���nodes ���short_names ���url_downloads���absnames���fs���rtimes���EnvironmentErrors���es���msg_notes���strs���has_keys���cus���Nones ���used_caches���oss���paths���existss���shutils���copyfiles���IOErrors ���UserErrors���name(���s���recdicts���url_dls���nodes ���use_caches���cus���separate_schemes���fnames���schemes ���used_caches���rtimes���es���fs���urls ���local_name(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys ���download_file��sB�����     ,  !+!#c������ ���C���s+��d�k��l�} �d�k�l�}�g��} �|�}�xω|�oρd�} �g��}�xο�|�D]ζ�}�|�|�d�|�i �ƒ�} �t �i�i�|�i�ƒ�t �i�i�| �ƒ�j�om�t �i�i�| �ƒ�}�|� o �d�}�n�| � p �|�| �j�o0�|�i�h��d�|�i�<ƒ�|�i�|�ƒ�|�} �q+qE�| � o-�h��d�|�i�<g�}�|�i�|�ƒ�| �} �PqE�qE�Wt�|�ƒ�d�j�o �| �| �j�o'�t�| �t �i�i�|�d�d�ƒ�ƒ�} �n�t�|��d�t�t�d �„��t�|�ƒ�ƒ�ƒ�| �f�ƒ�| �g��|��d�|�h��d�| �<h��d �d�<d�d �d�ƒ} �x@�| �D]8�}�x/�|�D]'�}�|�i�|�j�o�| �i�|�ƒ�qοqοWqβWq*�W| �Sd �S( ���sZ���Upload nodes in "nodelist" according to "url_dl". Return list of nodes that failed.(���s���remote_copy_move(���s���repl_file_names����s���names���.i���i����s���Uploading %s to %sc���������C���s���|��d�S(���Ns���name(���s���x(���s���x(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys���<lambda>|��s����s���mkdirs���errmsgN(���s���CopyMoves���remote_copy_moves���VersConts���repl_file_names���faileds���nodelists���todolists���to_items���fromlists���nodes���url_dls���names���to_names���oss���paths���basenames���absnames���dirnames���ds���appends���removes���lens ���path_joins���msg_infos���recdicts���strs���maps���shorten_dictlists���flists���f(���s���recdicts���url_dls���nodelists���fromlists���ds���todolists���fs���repl_file_names���nodes���flists���faileds���to_items���to_names���remote_copy_move(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys ���upload_fileU��sL�����  ��(     ''��c���������C���s���t��|��d�|�d�ƒ�d�Sd�S(���s9���Delete remote file for node "node" according to "url_dl".s!���Removing "%s" NOT IMPLEMENTED YETs���namei���N(���s���msg_infos���recdicts���url_dl(���s���recdicts���url_dls���node(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys ���remote_remove‹��s�����(���s���os.paths���oss���shutils���times���urlparses���urllibs ���urlretrieves ���urlcleanups���urlopens���Utils���Messages���is_urls ���url_split3s���url_times ���remote_times���get_header_dates���get_progname_rsyncs���get_progname_scps���get_progname_rcps ���url_downloads ���url_cleanups ���download_files ���upload_files ���remote_remove(���s ���remote_removes���url_times ���url_downloads ���url_split3s���urlopens���is_urls���times���get_progname_rcps ���url_cleanups���urlparses���get_progname_rsyncs ���urlretrieves���get_progname_scps ���urlcleanups���get_header_dates ���download_files���shutils���oss ���remote_times ���upload_file(����(����s+���/mnt/gmirror/ports/devel/aap/work/Remote.pys���? ���s&���     !      ‘  : 6