Version 0.4.0 [2007-12-15] ================================================================= Focus: minor bugfixes Fix: history GUI; reset text entry fields on delete/clear Fix: import tkinter only if available (bug #10488) Chg: minor tweaks in configure GUI Version 0.3.5 (nomad) [2007-12-07] ================================================================= Focus: bugfixes New: short options for --disable-* New: --configure; 'clear' button New: --check-updates; mswin; prompt for mswin binaries download Chg: be more discreet about program options at startup Fix: set initial --configure, --history toplevel positions Fix: check if host is supported _before_ attempting to get & parse page html Fix: domain; stage6.divx.com -> stage6.com (bug #10425) Version 0.3.4 (nomad) [2007-12] ================================================================= Focus: Feature enhancements, minor tweaks and bugfixes New: ~/.clive/history.log; stores successfully visited and parsed URLs New: --history; gui (tkinter) for browsing history log New: --configure; gui (tkinter) for configuring clive (~/.clive/config.py) Fix: exception handling; traceback is your friend; fixes also the 'error: o' Fix: --emit; use basename for filename instead of $prefix/filename Fix: --quiet; force stdout for --emit Fix: --emit; ignore the resume/overwrite/rename behaviour altogether Chg: added and cleaned up the startup options display Chg: replaced 'note:' stdout messages with 'status:' etc. Chg: save ~/.clive/last.log only if queue > 0 and/or skipped > 0 Chg: urlgrabber errors are now displayed with URLs Chg: config.py; removed support for the `last' variable ----------------------------------------------------------------- NOTE: PATCH SUBMISSIONS ----------------------------------------------------------------- As of the release 0.3.3, the `clive' project, will only merge those user submitted patches into the development trunk that are: a) Bugfixes b) Feature enhancements b.1) If the patch adds support for a website, the website must: b.1.1) be free to use (with or without registration) b.1.2) allow users to submit new video content b.1.3) qualify otherwise as a "host for user-generated videos" Those patches that go clearly against the general architecture of `clive' and/or the notes above, may remain available in the patch tracking system. They will not be however merged into the trunk development code. If a user chooses to apply any of the user submitted patches available in the patch tracking system, he/she should bear in mind that the author of `clive' will not provide support for an unofficial, or otherwise modified, version of `clive'. One should contact the original submitter of the patch instead. Version 0.3.3 (nomad) [2007-11-15] ================================================================= Focus: Severe bugfix Fix: startup error; ~/.clive does not exist (bug #10332) [!dottedmag] Chg: .iss; clive.1.html -> clive.1.txt Version 0.3.2 (nomad) [2007-11-15] ================================================================= Focus: Minor tweaks, new features New: --last, --disable-log, --clear-logs New: console commands `l_clr', `q_r_last' New: accept gv embedded urls (sr #1804) [!sk] New: --check-updates Fix: improved stage6 title parsing Chg: renamed console variable `accept_gzip' to `gzip' Chg: renamed many of the console commands Chg: user-agent defaults now to a random string Chg: tweaked help strings Chg: --filename-chars -> --output-mask Chg: --filename-format -> --output-fmt Chg: --from-clipboard -> --from-clipb Chg: config.py and console; variable names changed accordingly Chg: fixed various typos in clive(1) Version 0.3.1 (nomad) [2007-11] ================================================================= Focus: Minor bugfixes, tweaks New: 'clear' console command New: accept multiple urls on command line New: guba (free) support New: stage6 support New: mswindows binary setup (mswin/setup_clive.iss) Fix: verify prefix path Fix: random output filename; insert prefix dash only if len(title) > 0 Chg: cleaned up stdout messages Chg: improved manpage documentation Version 0.3.0 (nomad) [2007-10] ================================================================= Focus: Complete rewrite/cleanup and new features Be sure to uninstall any previous versions first. NOTE: Some old features have been removed. Some may re-appear in future releases. The noticeable removals are listed below. New: Complete CLI overhaul New: urlgrabber module requirement New: Transfer rate throttle, `--throttle' (1) New: Auto-resume file transfers for dmotion (2) New: Clipboard support (unix/x: xclip, mswin: win32clipboard) New: ~/.clive/config.py support (3) New: Console interface (4) Chg: New `--emit' output format; see `--help' Chg: Removed ~/.cliverc support Chg: Removed newt module requirement and the newt UI Chg: Removed support Chg: Removed ytube log-in support Chg: Removed reading urls from files (5) (1) = Limits transfer rate to the specified ~KB/s (default=0, unlimited). (2) = ytube and vgoogle do not support resumed file transfers and only render the existing file useless. clive enforces --rename for ytube and vgoogle. NOTE: dmotion; --overwrite and --rename are ignored if local_file_len < remote_file_len. The fail-safe switches to use `--overwrite' if the server forbids (HTTP/403) the request. NOTE: vgoogle video extraction urls provide 'begin' and 'len' parameters which, in theory at least, could be utilized to allow resuming. Neither appear to take the values as bytes. (3) = See clive(1) manual page for more details. (4) = Entered if clive is started without the URL parameter. Type "help" while in stdin read mode to see the complete list of the supported console commands. (5) = Try: `% clive < list-of-urls.file`. Version 0.2.1 [2007-09] ================================================================= focus: minor feature enchancements and bug fixes fix: dmotion extraction (bug #9971) [!legatvs] fix: broken vgoogle support (bug #9964) [!legatvs] fix: removed use of hardcoded video.google.com domain (bug #9239) [!grok] fix: broken youtube support (bug #9873) [!alfie] chg: newt: main menu 'Continue' -> 'OK' (sr #1598) [!grok] chg: prompt only if number of videos exceed 1 (sr #1600) [!asjo] chg: check `--encoder=PATH' at start (sr #1601) [!asjo] NOTE: Bug #110880 in newt (Ubuntu): python-newt scripts crash on exit (https://bugs.launchpad.net/ubuntu/+source/newt/+bug/110880) Version 0.2.0 [late 2007-04] ================================================================= focus: major feature enchancements new: support for dailymotion.com (excl. embedded videos and login) (1) new: `--proxy-user', `--proxy-pass', `--no-proxy' (2) new: `--no-youtube-login' new: short option for `--play-format' new: `--no-encode', `--no-accept', `--no-play', `--no-skip' new: `--filename-chars' (3) new: `--filename-format' (4) chg: limit filename to 64 chars (excl. extension) chg: renamed `--output' as `--output-file' chg: program defaults to http_proxy env. variable chg: removed `--save' (5) chg: newt ui shows menu elements dynamically chg: less cryptic proxy info at startup chg: renamed `--nonewt' as `--no-newt' chg: renamed `--nostrip' as `--no-strip' chg: removed `--no-strip' short option `-n' chg: renamed `--report-bugs' as `--bugs' chg: removed `--login' chg: added `--youtube-user' and `--youtube-user' chg: renamed `--log-info' as `--youtube-status' chg: renamed `--logout' as `--youtube-logout' chg: youtube.com login is now done using `--youtube-user' and `--youtube-user' chg: `--play-format=FORMAT' now sets both `--play' and `--FORMAT' (6) chg: renamed `--yes,-y' as `--accept,-A' chg: renamed `-k' as `-s' (`--skip') fix: a potential div by 0 in clive/cli.py fix: `--emit', display ignored urls w/o prompting if scan found 0 videos fix: use newt ui if scan found 0 videos (and if newt is enabled) fix: display video info if newt is not used and `--yes' was not used fix: `--quiet', add a newline after scan fix: `--emit' acknowledges `--output-file' (1) login support may be added later. embedded video support is unlikely to be added. (2) Proxy authentication support is experimental and req. testing. Please report any bugs. The username and password are encoded using the 'basic' authentication scheme. NOTE: Proxy is on by default if the appropriate env. variable is defined. (3) `--filename-chars' (cliverc: filename_chars = STRING) Sets the accepted characters that are used from the video page title. clive uses regular expression (regexp) to strip the unneeded characters from the title. The default STRING is 'A-Za-z0-9' which translates roughly to 'accept all A-Z, a-z and 0-9 characters'. So for example, if you wanted to accept whitespaces and commas as well, you could use: # clive --filename-chars='A-Za-z0-9 ,' URL NOTE: Some punctuations may not be used for filenames NOTE: `--output-file' overrides this option (4) `--filename-format' (cliverc: filename_format = STRING) Specifies the format of a video filename. This is done using string identifiers such as %t or %h that are simply replaced by clive with the appropriate data. The default STRING is '%t.%e'. Accepted string identifiers: %t = title %i = video id %h = host (e.g. 'youtube' or 'vgoogle') %e = file extension (e.g. 'flv' or 'avi') Examples: # clive --filename-format='%t (%i) [%h].%e' URL NOTE: The output of `--emit' will change accordingly and `--output-file' overrides this option. (5) The location of cliverc is ~/.cliverc. The file is not created or installed by default. use_newt = on/off Enable/disable newt UI, enabled by default if newt is installed. `--no-newt' disables it'. user_agent = string Use string as user-agent for HTTP connections, the same as `--agent' prefix = string Save extracted videos to prefix, the same as `--prefix' http_proxy = string Use string as HTTP proxy, instead of the one specified in env. You can disable the use of HTTP proxy with `--no-proxy'. proxy_user = string Set PROXY authentication user name to string, the same as `--proxy-user' proxy_pass = string Set PROXY authentication password to string, the same as `--proxy-pass' youtube_user = string Set youtube.com user name to string, the same as `--youtube-user' youtube_pass = string Set youtube.com password to string, the same as `--youtube-pass' NOTE: Both youtube_user and youtube_pass must be used in order to log into youtube.com. If they are used, clive checks the login status at startup and logs in if required. You can bypass the login procedure with `--youtube-no-login' while the above two variables are being used. player = string Set player to string, the same as `--player' Example: player = vlc --fullscreen %%i NOTE: %%i is not a typo, be sure to use %% in cliverc encoder = string Set encoder to string, the same as `--encoder' Example: encoder = ffmpeg -y -b 1200 -ab 128 -i %%i %%o NOTE: Use %% instead of % in cliverc Be sure to use -y with ffmpeg, this tells ffmpeg to overwrite the existing output file without a prompt. always_accept = on/off Enable/disable extraction without prompting, the same as `--accept' `--no-accept' disables it always_play = on/off Enable/disable playing extracted video, the same as `--play' `--no-play' disables it play_format = string Set default play format as string, like `--play-format=FORMAT' `--no-play' and `--no-encode' disable it NOTE: Sets (and overrides) `--play' and `--FORMAT' encode_format = string(s) Re-encode videos to formats defined by the string, like `--mpeg/avi/flv' `--no-encode' disables it It is possible to pass more than one format, e.g.: encode_format = mpeg,avi always_skip = on/off Enable/disable skipping already extracted videos, the same as `--skip' `--no-skip' disables it always_strip = on/off Enable/disable stripping stdout messages, `--no-strip' disables it `--no-strip' disables it. filename_chars = STRING Set STRING as the accepted filename (title) characters (default: A-Za-z0-9) filename_format = STRING Set STRING as the filename format (default: %t.%e) NOTE: Use %% instead of %, e.g. %%t NOTES: (1) All previously existing options have been renamed and a few new ones have been added. Be sure to edit any previously existing cliverc file accordingly. (2) for on/off you may use also 0/1, true/false and yes/no (3) Example cliverc: [clive] player = vlc --fullscreen %%i encoder = /usr/bin/ffmpeg -y -b 1200 -ab 128 -i %%i %%o http_proxy = http://proxy:80 youtube_user = myusername youtube_pass = myuserpass always_accept = yes encode_format = mpeg always_play = on filename_chars = A-Za-z0-9 ,!? filename_format = %%t (%%i) [%%h].%%e (6) Before: # clive --play --mpeg --play-format=mpeg This does the same thing now: # clive --play-format=mpeg Version 0.1.9 [late 2007-03] ================================================================= focus: minor feature enchancements new: --output=FILE, write all extracted videos to FILE (1) new: accepts extraction URLs (2) new: login to youtube.com (experimental) (3) chg: increased --emit verbosity, new output format (4) chg: stdout/stderr lines are formatted slightly differently chg: improved pipe support (5) chg: --quiet shows scan messages chg: --emit lists ignored urls with reasons chg: use --nostrip if --emit is being used fix: display ignored urls if failed to find any videos (1) --output=FILE works best for single video extractions (2) URLs. Briefly. * Video page URL - e.g. http://youtube.com/watch?v=6IPN30Lzv7w * Video extraction URL - e.g. http://www.youtube.com/get_video?video_id= (...) - used to download the actual video file from the website - clive can list these URLs when --emit is used NOTE: When video extraction URLs are being used, be sure to use --output=FILE with YouTube URLs, for example: # clive --output=video.flv (vid-ext-url) Both of the above URL types can be used with clive. (3) Login to YouTube NOTE: Experimental, any feedback is highly valued Enables extracting videos that require logging in, including those that have been flagged as 'inpropriate content'. # clive --login=username:password Once logged in, any youtube.com extractions are done as the user that has been logged in. # clive --logout Logout from youtube.com, removes the ~/.clivecookies file. # clive --loginfo Displays info from the ~/.clivecookies file. Functionality with some notes: 1: youtube.com sets the 'LOGIN_INFO' cookie if login succeeds. 2: clive sets 'is_adult' cookie without connecting to youtube.com. This eliminates the step that would normally require that the user hits the confirm button to accept any content that has been flagged as 'inappropriate'. 3: Cookies are saved to ~/.clivecookies. The LOGIN_INFO cookie is valid for 10 years. youtube.com sets a selection of other cookies. If privacy is a concern, then avoid using --login. Many of these cookies are likely to be used to keep a track of the visitor's activities on the website. NOTE: Login is done over unencrypted HTTP transfers. Anyone intercepting the HTTP traffic could steal the login info. (4) --emit is now more verbose, each extraction URL takes a line like before, but the URL is now followed by 'filename' and 'size'. Each found URL line begins with '(f) : ', for example: (f) : http://... example.avi 8.0MB Each element (URL, filename, size) is separated by a single whitespace. Ignored URLs are printed after the found ones, each ignored URL line begins with '(e) : ' and is followed by the URL and the reason, for example: (e) : http://... reason NOTE: 'reason' may have whitespaces, for example: (e) : http://... terminated (reason: age-verification) NOTE: --nostrip is now used by default if --emit is being used (5) If an URL or a file is not specified on command line, clive will read stdin for input. This allows pipes to be used properly with clive, e.g.: # cat urls.txt | clive # clive < urls.txt Version 0.1.8 [mid 2007-03] ================================================================= focus: minor fixes and feature enchancements new: --emit a listing of video extraction urls to stdout [sr #1505], notes(1) new: short option (-n) for --nostrip chg: catch urllib2 exceptions inside the url checking loop fix: newt < 0.52 esc key binding incompatibility [#8697] (Nicola Vitale) fix: transfer timer is now reset correctly fix: batch loading; list order is no longer lost (1) For a no nonsense listing, try: # clive -enq URL|FILE NOTE: -e is a short option for --emit (2) New maintainers: FreeBSD: Nicola Vitale (http://www.freshports.org/multimedia/clive) Debian: Mikhail Gusarov (http://packages.debian.org/clive) Version 0.1.7 [early 2007-03] ================================================================= focus: major feature enhancements, partial rewrite new: all urls are now scanned for videos (incl. embedded) before extracting new: exit with return code (non-zero=error) new: use gzip encoding for http (page) requests, see notes(1) new: short option strings new: --prefix option new: prompt to extract a video new: 'video removed' detection (yt) new: newt user interface, see notes(2) new: --nonewt, disable newt ui new: transfer rate + time left new: --skip, skip already existing videos chg: urllib -> urllib2 chg: page parsing should now use slightly less system resources chg: --quiet extracts all videos without prompting chg: --quiet no longer overwrites, existing files are skipped chg: --quiet is now slightly more verbose chg: removed --saveas, --extract-all, --simulate, obsoleted by scan chg: removed --batch, obsoleted by new synopsis, see notes(3) chg: renamed --proxy=env as --proxy=http_proxy for clarity chg: replaced --ffmpeg by --encoder, see notes(4) chg: overwrite prompt is now more verbose chg: --encoder, --player are now more verbose of req. options chg: --player uses similar formatting with --encoder, see notes(4) chg: --play=FORMAT, where FORMAT can be mpeg/avi/flv, see notes(5) chg: --proxy=None now disables the use of a proxy fix: crash with --play, without path to a player fix: batch parser no longer accepts empty lines Important: This version changes the way --player=STRING is being used, please make sure to read the note(4) below (1) Some proxies (e.g. privoxy) block gzip-encoding header in http reqs. gzip can give a good compression rate on HTML pages. Typically bandwidth required to retrieve a video page can cut down to 1/3. (2) clive uses newt (snack module) for some parts of the new user interface (e.g. video selection). The newt user interface is used by default if the module is installed. Use --nonewt switch to disable it and use the 'classic' interface (prompt-for- video). Debian / *buntu: sudo apt-get install python-newt Redhat / Fedora: should have it installed Others: most Linux distros should have it installed (3) URL argument can now be a file containing URL(s). This was done previously with the --batch=FILE parameter. e.g. # clive -myo urls.txt NOTE: clive no longer prompts for URL, instead it terminates with an error if an URL was not specified. (4) --encoder="/usr/bin/ffmpeg -y -i %i %o" --encoder="/usr/bin/mencoder %i -ovc lavc -oac mp3lame -o %o" where %i = input file, %o = output file NOTE: --player is now used in the same manner, for example: --player="`which vlc` %i" (5) --play defaults to the extracted video if FORMAT is not specified NOTE: FORMAT can only be used with --mpeg/avi/flv, so for example: --mpeg --play=mpeg would work, whereas: --mpeg --play=avi would not because the video was not re-encoded to avi format after extraction. Version 0.1.6 [mid 2007-02] ================================================================= new: zero-length video detection fix: embedded videos: accept urls with a clear yt/gv fingerprint only fix: accept long gv urls (note: be sure to quote them on cmdline) fix: zero division in progress fix: total bytes read calculation Version 0.1.5 (Complete Rewrite) [early 2007-02] ================================================================= new: embedded video extraction new: --extract-all switch new: batch mode new: simulation mode new: proxy (non-auth) support new: user-agent default (Python-urllib/x.xx) new: custom exception hook fix: accept embedded video yt urls [#8390] (Adam Sjogren) fix: omit ctrl+c/ctrl+d backtrace [#8391] (Adam Sjogren / Klaus A. Seistrup) fix: manual page installation (slackware et al) chg: renamed --nowrap as --nostrip chg: errors printed to stderr Version 0.1.0 (Initial) [mid 2007-01] =================================================================