python-based downloader (youtube-dl) missing critical feature ...
python-based youtube-dl
~
http://rg3.github.com/youtube-dl/
~
is sorely missing a flag in order to indicate the maximum file length of the
data feed it would download (well, unless, for some reason, it is considered a
"feature").
~
I wonder what developers were thinking about when they came up this nice piece
of code. If you actually look in the code
~
...
data = urllib2.urlopen(basic_request)
content_length = data.info()['Content-Length']
...
~
you will see they get the content length of the actual data feed and they also
print the progress status based on the content length
~
Implementing an if statement a la:
~
max_indicated_content_length = self.params.get('max_content_length', None);
~
if( content_length > max_indicated_content_length ){ [do not download, just
report "data feed too large"] }
else{ [do] }
~
shouldn't be hard at all
~
youtube-dl is under the Creative Commons License copyrighted by 2006-2011
Ricardo Garcia Gonzalez and maintained by him and a group of python developers
~
They are the ones keeping a mental map of that project. It would be a plus if
they implement this feature, but anyother python developer can implemented
(please, let me know if/when you do)
~
lbrtchx
--
http://mail.python.org/mailman/listinfo/python-list
short reading materials about anthropological, universal themes for students with no reading habit ...
> A few things to note: > * Not all sites necessarily send the Content-Length header. > * RTMP URLs would have to be treated differently ~ No some don't, but at least for the ones that do (like youtube) it would be a useful feature. The Content-Length header is used in the code anyway ~ > * Sending a Range header might allow for a better implementation. ~ Yes, it would ~ > Why do you want to restrict the filesize of the download in the first > place? I can't see a use case, but that doesn't mean there isn't one. ~ OK, I see your point. Say, you have a list of youtube urls, which feeds you want to download using a script or a playlist someone put together, but you don't want to download files that are too large, which may not be of any use to you. For example, I teach and it doesn't make any sense to use a full movie as part of a class set. So I would like for youtube-dl to let me know which files are larger than a given size (and possibly save them in a file) for me to check the files first My languages are ANSI C, C++ and java. When I was young and silly would blab girls just because they crossed my way, now I don't like to even look into anything that I don't want to invest my time in on an ongoing basis. I would let people that code python and have a mental map of that code base do it themselves lbrtchx -- http://mail.python.org/mailman/listinfo/python-list
youtube-dl: way to deal with the size cap issue + new errors + issues ...
~ I did find my way (through a silly hack) to get all files within a size range without waiting for youtube-dl to be "enhanced". You could simply run youtube-dl in simulate mode and then parse that data to get the info ~ $ youtube-dl --help | grep simulate -s, --simulate do not download the video and do not write anything to disk -g, --get-urlsimulate, quiet but print URL -e, --get-title simulate, quiet but print title --get-thumbnail simulate, quiet but print thumbnail URL --get-descriptionsimulate, quiet but print video description --get-filename simulate, quiet but print output filename --get-format simulate, quiet but print output format ~ it turns out I needed the data anyway and %(uploader)s %(stitle)s and %(ext)s are helpful as well, for example, in case you decide to skip a certain uploader ~ I have also been getting errors reporting: ~ RTMP download detected but "rtmpdump" could not be run ~ What does it mean? Is it a youtube thing or a python/youtube-dl one (or both)? Could you fix that with some flag? ~ It would be very helpful if you could redirect youtube-dl errors to a separate file you would indicate via a flag ~ lbrtchx comp.lang.python: youtube-dl: way to deal with the size cap issue + new errors + issues ... ~ // __ ERROR: RTMP download detected but "rtmpdump" could not be run ~ downloading: http://www.youtube.com/watch?v=TD-66LHJF9E [youtube] Setting language [youtube] TD-66LHJF9E: Downloading video webpage [youtube] TD-66LHJF9E: Downloading video info webpage [youtube] TD-66LHJF9E: Extracting video information [youtube] RTMP download detected [download] Destination: ./LionsgateMovies-TD-66LHJF9E_Trading_Mom.flv ERROR: RTMP download detected but "rtmpdump" could not be run ~ downloading: http://www.youtube.com/watch?v=Ft5fFOktUno [youtube] Setting language [youtube] Ft5fFOktUno: Downloading video webpage [youtube] Ft5fFOktUno: Downloading video info webpage [youtube] Ft5fFOktUno: Extracting video information [youtube] RTMP download detected [download] Destination: ./LionsgateMovies-Ft5fFOktUno_Speed_Racer_The_Movie.flv ERROR: RTMP download detected but "rtmpdump" could not be run ~ downloading: http://www.youtube.com/watch?v=wRbAGrIjCr4 [youtube] Setting language [youtube] wRbAGrIjCr4: Downloading video webpage [youtube] wRbAGrIjCr4: Downloading video info webpage [youtube] wRbAGrIjCr4: Extracting video information [youtube] RTMP download detected [download] Destination: ./LionsgateMovies-wRbAGrIjCr4_Jonah_A_VeggieTales_Movie.flv ERROR: RTMP download detected but "rtmpdump" could not be run ~ downloading: http://www.youtube.com/watch?v=yU0KpRBkeMY [youtube] Setting language [youtube] yU0KpRBkeMY: Downloading video webpage [youtube] yU0KpRBkeMY: Downloading video info webpage [youtube] yU0KpRBkeMY: Extracting video information [youtube] RTMP download detected [download] Destination: ./LionsgateMovies-yU0KpRBkeMY_Hercules_In_New_York.flv ERROR: RTMP download detected but "rtmpdump" could not be run ~ -- http://mail.python.org/mailman/listinfo/python-list
