;ò ~ä¿Dc@s˜dkZdkZdkZdkZdkZeZydklZWn"e j oZ dklZnXde fd„ƒYZ dei fd„ƒYZdefd„ƒYZdfd „ƒYZd eifd „ƒYZd klZlZlZlZlZlZlZdkZdkZdkZdkZdkZdkZd eifd„ƒYZ dei!fd„ƒYZ!ea"d„Z#d„Z$d„Z%dS(N(sStringIOs RangeErrorcBstZdZRS(s6Error raised when an unsatisfiable range is requested.(s__name__s __module__s__doc__(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys RangeError#s sHTTPRangeHandlercBs tZdZd„Zd„ZRS(sºHandler that enables HTTP Range headers. This was extremely simple. The Range header is a HTTP feature to begin with so all this class does is tell urllib2 that the "206 Partial Content" reponse from the HTTP server is what we expected. Example: import urllib2 import byterange range_handler = range.HTTPRangeHandler() opener = urllib2.build_opener(range_handler) # install it urllib2.install_opener(opener) # create Request and set Range header req = urllib2.Request('http://www.python.org/') req.header['Range'] = 'bytes=30-50' f = urllib2.urlopen(req) cCs5ti|||iƒƒ}||_||_|SdS(N( surllibs addinfourlsfpshdrssreqs get_full_urlsrscodesmsg(sselfsreqsfpscodesmsgshdrssr((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pyshttp_error_206?s  cCstdƒ‚dS(NsRequested Range Not Satisfiable(s RangeError(sselfsreqsfpscodesmsgshdrs((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pyshttp_error_416Fs(s__name__s __module__s__doc__shttp_error_206shttp_error_416(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysHTTPRangeHandler's  sHTTPSRangeHandlercBs tZdZd„Zd„ZRS(s! Range Header support for HTTPS. cCs|i|||||ƒSdS(N(sselfshttp_error_206sreqsfpscodesmsgshdrs(sselfsreqsfpscodesmsgshdrs((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pyshttps_error_206MscCs|i|||||ƒdS(N(sselfshttps_error_416sreqsfpscodesmsgshdrs(sselfsreqsfpscodesmsgshdrs((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pyshttps_error_416Ps(s__name__s __module__s__doc__shttps_error_206shttps_error_416(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysHTTPSRangeHandlerJs  sRangeableFileObjectcBshtZdZd„Zd„Zd„Zdd„Zdd„Zdd„Zd „Z d „Z d „Z RS( s"File object wrapper to enable raw range handling. This was implemented primarilary for handling range specifications for file:// urls. This object effectively makes a file object look like it consists only of a range of bytes in the stream. Examples: # expose 10 bytes, starting at byte position 20, from # /etc/aliases. >>> fo = RangeableFileObject(file('/etc/passwd', 'r'), (20,30)) # seek seeks within the range (to position 23 in this case) >>> fo.seek(3) # tell tells where your at _within the range_ (position 3 in # this case) >>> fo.tell() # read EOFs if an attempt is made to read past the last # byte in the range. the following will return only 7 bytes. >>> fo.read(30) cCs>||_t|ƒ\|_|_d|_|i|iƒdS(sCreate a RangeableFileObject. fo -- a file like object. only the read() method need be supported but supporting an optimized seek() is preferable. rangetup -- a (firstbyte,lastbyte) tuple specifying the range to work over. The file object provided is assumed to be at byte offset 0. iN(sfosselfsrange_tuple_normalizesrangetups firstbyteslastbytesrealposs_do_seek(sselfsfosrangetup((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys__init__hs   cCs4t|i|ƒot|i|ƒSnt|‚dS(s«This effectively allows us to wrap at the instance level. Any attribute not found in _this_ object will be searched for in self.fo. This includes methods.N(shasattrsselfsfosnamesgetattrsAttributeError(sselfsname((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys __getattr__vscCs|i|iSdS(s5Return the position within the range. This is different from fo.seek in that position 0 is the first byte position of the range tuple. For example, if this object was created with a range tuple of (500,899), tell() will return 0 when at byte position 500 of the file. N(sselfsrealposs firstbyte(sself((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pystell~sicCsµ|dddfjpt‚|djo|i|}n<|djo|i|}n|djotdƒ‚n|io ||ijo |i}n|i ||iƒdS(seSeek within the byte range. Positioning is identical to that described under tell(). iiis$seek from end of file not supported.N( swhencesAssertionErrorsselfs firstbytesoffsets realoffsetsrealpossIOErrorslastbytes_do_seek(sselfsoffsetswhences realoffset((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysseek‡s    iÿÿÿÿcCs>|i|ƒ}|ii|ƒ}|it|ƒ7_|SdS(s`Read within the range. This method will limit the size read based on the range. N(sselfs_calc_read_sizessizesfosreadsrsltsrealposslen(sselfssizesrslt((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysreadšs cCs>|i|ƒ}|ii|ƒ}|it|ƒ7_|SdS(sfRead lines within the range. This method will limit the size read based on the range. N(sselfs_calc_read_sizessizesfosreadlinesrsltsrealposslen(sselfssizesrslt((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysreadline£s cCsb|ioP|djo/|i||ijo|i|i}qVqZ|i|i}n|SdS(sSHandles calculating the amount of data to read based on the range. iÿÿÿÿN(sselfslastbytessizesrealpos(sselfssize((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys_calc_read_size¬s  cCsc|djpt‚t|idƒ o|i|ƒn|ii|i|ƒ|i|7_dS(sSeek based on whether wrapped object supports seek(). offset is relative to the current position (self.realpos). isseekN(soffsetsAssertionErrorshasattrsselfsfos_poor_mans_seeksseeksrealpos(sselfsoffset((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys_do_seek¸s cCsƒd}d}xp||job|||jo||}n|ii|ƒ}t|ƒ|jotdƒ‚n||7}qWdS(s¯Seek by calling the wrapped file objects read() method. This is used for file like objects that do not have native seek support. The wrapped objects read() method is called to manually seek to the desired position. offset -- read this number of bytes from the wrapped file object. raise RangeError if we encounter EOF before reaching the specified offset. iisRequested Range Not SatisfiableN( spossbufsizesoffsetsselfsfosreadsbufslens RangeError(sselfsoffsetspossbufsizesbuf((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys_poor_mans_seekÃs  ( s__name__s __module__s__doc__s__init__s __getattr__stellsseeksreadsreadlines_calc_read_sizes_do_seeks_poor_mans_seek(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysRangeableFileObjectSs     sFileRangeHandlercBstZdZd„ZRS(s~FileHandler subclass that adds Range support. This class handles Range headers exactly like an HTTP server would. cCsàdk} dk}|iƒ} |iƒ}ti|ƒ} t i | ƒ}|t i }ti|t iƒ}| i|ƒd}| oOti| ƒ\} }|pti| ƒ|iƒjotidƒ‚qÔnt| dƒ}|ii dt!ƒ}t#|ƒ}|fjpt$‚|o}|\} } | djo |} n| djp| |jp | |jot'dƒ‚n| | }t(|| | fƒ}n|i)t*d|pd||fƒƒ}ti+||d |ƒSdS( Nisfile not on local hostsrbsRangessRequested Range Not Satisfiables6Content-Type: %s Content-Length: %d Last-modified: %s s text/plainsfile:(,s mimetypess mimetoolssreqsget_hostshosts get_selectorsfilesurllibs url2pathnames localfilesossstatsstatssST_SIZEssizesrfc822s formatdatesST_MTIMEsmodifieds guess_typesmtypes splitportsportssockets gethostbynamesselfs get_namessurllib2sURLErrorsopensfosheaderssgetsNonesbrangesrange_header_to_tuplesAssertionErrorsfbslbs RangeErrorsRangeableFileObjectsMessagesStringIOs addinfourl(sselfsreqs mimetoolssbrangesmtypesfilesportssizesstatssfbs localfiles mimetypesshostslbsfosmodifiedsheaders((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysopen_local_fileÜs6     #   ' )(s__name__s __module__s__doc__sopen_local_file(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysFileRangeHandler×s (s splitports splitusers splitpasswds splitattrsunquotes addclosehooks addinfourlsFTPRangeHandlercBstZd„Zd„ZRS(NcCsì|iƒ}| otddf‚nt|ƒ\}}|tjo ti}nt |ƒ\}}|ot |ƒ\}} nt} t |ƒ}t |pdƒ}t | pdƒ} yti|ƒ}Wn'tij o}ti|ƒ‚nXt|iƒƒ\}}|idƒ} tt | ƒ} | d | df\} }| o | d o| d} nyI|i|| ||| ƒ} |odpd } xb|D]Z}t|ƒ\}}|i"ƒd jo|d d d ddd fjo|i#ƒ} q¡q¡Wt}t%|i&i'dtƒƒ}|fjpt)‚|o'|\}} |djo |}qbn| i,|| |ƒ\}}|o—|\}} | djo[|tjp |djot/dƒ‚n|} | |}|djot/dƒ‚qq| |}t0|d|fƒ}nd}t1i2|i3ƒƒd}|o|d|7}n|tj o |djo|d|7}nt5|ƒ}t7i8|ƒ}t9|||i3ƒƒSWn4ti:j o%}td|ft;i<ƒd‚nXdS(Ns ftp errors no host givenss/iÿÿÿÿiisIsDstypesasAsisdsRanges@Requested Range Not Satisfiable due to unobtainable file length.sRequested Range Not SatisfiablesContent-Type: %s sContent-Length: %d i(=sreqsget_hostshostsIOErrors splitportsportsNonesftplibsFTP_PORTs splitusersusers splitpasswdspasswdsunquotessockets gethostbynameserrorsmsgsurllib2sURLErrors splitattrs get_selectorspathsattrsssplitsdirssmapsfilesselfs connect_ftpsfwstypesattrsvalueslowersuppersrestsrange_header_to_tuplesheaderssgets range_tupsAssertionErrorsfbslbsretrfilesfpsretrlens RangeErrorsRangeableFileObjects mimetypess guess_types get_full_urlsmtypesStringIOssfs mimetoolssMessages addinfourls all_errorsssyssexc_info(sselfsreqsfps range_tupsrestsmtypesattrssfilesportslbstypesdirssfwspasswdsmsgshostsuserspathsretrlensattrsfbsvaluesheadersssf((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysftp_open sz    2        cCs t|||||ƒ}|SdS(N(s ftpwrappersuserspasswdshostsportsdirssfw(sselfsuserspasswdshostsportsdirssfw((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys connect_ftpXs(s__name__s __module__sftp_opens connect_ftp(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysFTPRangeHandler s Ls ftpwrappercBstZed„ZRS(Nc Cs<|iƒ|ddfjod}d}nd|}d}y|ii|ƒWn0tij o!|i ƒ|ii|ƒnXt } |o| o&y|ii |ƒWn4tij o%}td|ftiƒd‚nX|ii|ƒy#d |}|ii||ƒ} Wq¿tij o’}t|ƒd d jo;|i||ƒ\}}t||d fƒ}||fSq»t|ƒd d jo td|ftiƒd‚q»q¿Xn| oA|iidƒ|od|}nd}|ii|ƒ} nd|_t| didƒ|iƒ| dfSdS(NsdsDsTYPE AisTYPE is ftp errorisRETR is501ss550sLIST sLISTsrb(sselfs endtransferstypescmdsisdirsftpsvoidcmdsftplibs all_errorssinitsNonesconnsfilesnlsts error_permsreasonsIOErrorssyssexc_infos ntransfercmdsrestsstrsretrfilesfpsretrlensRangeableFileObjectsbusys addclosehooksmakefile( sselfsfilestypesrestsisdirscmdsreasonsfpsretrlensconn((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysretrfileasF  ! ) (s__name__s __module__sNonesretrfile(((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys ftpwrapper\scCsª|tjotSnttjodk}|idƒanti|ƒ}|oNt|iddƒƒ}|o|do|d|ddf}n|SnfSdS(sØGet a (firstbyte,lastbyte) tuple from a Range header value. Range headers have the form "bytes=-". This function pulls the firstbyte and lastbyte values and returns a (firstbyte,lastbyte) tuple. If lastbyte is not specified in the header value, it is returned as an empty string in the tuple. Return None if range_header is None Return () if range_header does not conform to the range spec pattern. Ns^bytes=(\d{1,})-(\d*)iii( s range_headersNones_rangeresrescompilesmatchsrange_tuple_normalizesgroupstup(s range_headerstupsresmatch((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysrange_header_to_tuple‘s    cCs_|tjotSnt|ƒ}|o3|do|d|ddf}nd|SndS(s•Convert a range tuple to a Range header value. Return a string of the form "bytes=-" or None if no range is needed. iis bytes=%s-%sN(s range_tupsNonesrange_tuple_normalize(s range_tup((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysrange_tuple_to_header¬s   cCsü|tjotSn|d}|tdfjo d}n t|ƒ}y|d}Wntj o d}n6X|tjo d}n|djot|ƒ}n||fddfjotSn||jotd||fƒ‚n||fSdS(s7Normalize a (first_byte,last_byte) range tuple. Return a tuple whose first element is guaranteed to be an int and whose second element will be '' (meaning: the last byte) or an int. Finally, return None if the normalized tuple == (0,'') as that is equivelant to retrieving the entire file. isisInvalid byte range: %s-%sN(s range_tupsNonesfbsintslbs IndexErrors RangeError(s range_tupsfbslb((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pysrange_tuple_normalize¸s(        (&sossstatsurllibsurllib2srfc822sNonesDEBUGs cStringIOsStringIOs ImportErrorsmsgsIOErrors RangeErrors BaseHandlersHTTPRangeHandlersHTTPSRangeHandlersRangeableFileObjects FileHandlersFileRangeHandlers splitports splitusers splitpasswds splitattrsunquotes addclosehooks addinfourlsftplibssocketssyss mimetypess mimetoolss FTPHandlersFTPRangeHandlers ftpwrappers_rangeresrange_header_to_tuplesrange_tuple_to_headersrange_tuple_normalize(sunquotes mimetoolssrange_tuple_normalizesrange_tuple_to_headers addinfourls splitports splitusersurllibs RangeErrorsFileRangeHandlers splitattrs mimetypessftplibsstats addclosehooksrfc822sRangeableFileObjectsurllib2ssyssmsgsDEBUGs ftpwrappersHTTPSRangeHandlerssocketsStringIOsrange_header_to_tuplesHTTPRangeHandlers splitpasswdsFTPRangeHandlersos((sR/mnt/gmirror/ports/www/py-urlgrabber/work/urlgrabber-3.1.0/urlgrabber/byterange.pys?s6     # „+1      Q4