86296ad2cd
[utils] Add ability to control skipping false values in dict_get
2016-02-07 08:13:04 +06:00
cbecc9b903
[utils] Add dict_get convenience method
2016-02-07 06:12:53 +06:00
6b77d52b1f
[test_utils] Add tests for encode_compat_str
2015-12-20 07:07:14 +06:00
db2fe38b55
[utils] Support alternative timestamp format in TTML
...
Fixes #7608
2015-12-19 19:29:51 +08:00
d631d5f9f2
[utils] Fix TTML conversion
...
Tolerate invalid timestamps (closes #7909 )
2015-12-19 18:21:42 +08:00
31b2051e21
[utils] Add remove_quotes
2015-12-14 21:30:58 +06:00
9cb9a5df77
[utils] Check ext with trailing slash against the list of known extensions
2015-11-22 17:27:13 +06:00
5035536e3f
[test_utils] Add tests for determine_ext
2015-11-22 06:33:52 +06:00
7aefc49c40
[utils] Skip invalid/non HTML entities ( Closes #7518 )
2015-11-16 20:20:16 +06:00
6a75040278
[utils] unified_strdate: Return None if the date format can't be recognized ( fixes #7340 )
...
This issue was introduced with ae12bc3ebb
2015-11-02 14:08:38 +01:00
30eecc6a04
Merge pull request #7296 from jaimeMF/xml_attrib_unicode
...
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (…
2015-10-31 18:15:21 +00:00
578c074575
[utils] Support list of xpath in xpath_element
2015-10-31 22:39:44 +06:00
52c3a6e49d
[utils] Improve parse_iso8601
2015-10-28 21:40:22 +06:00
36e6f62cd0
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x ( #7178 )
...
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
2015-10-25 20:13:16 +01:00
d01949dc89
[utils:js_to_json] Fix bad escape in double quoted strings
2015-10-20 23:09:51 +06:00
f71264490c
[test_utils] Add tests for cli option converters
2015-09-05 03:07:19 +06:00
87f70ab39d
[test_utils] Add more tests for xpath
2015-09-05 00:36:16 +06:00
ee114368ad
[utils] Make value optional for find_xpath_attr
...
This allows selecting particular attributes by name but without specifying the value and similar to xpath syntax `[@attrib]`
2015-08-01 20:22:13 +06:00
9c29bc69f7
[utils] Improve parse_duration
...
Now dots are parsed. For example '87 Min.'
2015-07-22 23:15:22 +08:00
1b0427e6c4
[utils] Support TTML without default namespace
...
In a strict sense such TTML is invalid, but Yahoo uses it.
2015-05-19 00:45:01 +08:00
7dff03636a
[utils] Support 'dur' field in TTML
2015-05-12 12:47:37 +08:00
d39e0f05db
[utils] Remove sanitize_url_path_consecutive_slashes()
...
This function is used only in SohuIE, which is updated to use a new
extraction logic.
2015-05-09 17:37:39 +08:00
0fe2ff78e6
[NBC] Enhance embedURL extraction ( closes #2549 )
2015-05-04 21:55:04 +08:00
b3ed15b760
[utils] Add replace_extension
2015-05-02 23:23:06 +06:00
a4bcaad773
[test_utils] Add tests for prepend_extension
2015-05-02 23:10:48 +06:00
bf6427d2fb
[ffmpeg] Add dfxp (TTML) subtitles support ( #3432 , #5146 )
2015-04-25 23:18:27 +08:00
0a1603634b
[utils] Remove url_infer_protocol
2015-04-08 21:39:34 +08:00
418c5cc3fc
[udn] Add new extractor
2015-04-08 17:26:51 +08:00
8cf70de428
[test_utils] Add test for unified_strdate
2015-04-04 19:11:01 +06:00
ba9e68f402
[utils] Drop trailing comma before closing brace
2015-04-04 17:48:55 +06:00
91757b0f37
[utils] Escape all HTML entities written in hexadecimal form
2015-03-26 17:15:27 +02:00
5379a2d40d
[test/utils] Test xpath_text
2015-03-21 14:12:43 +01:00
92a4793b3c
[utils] Place sanitize url function near other sanitizing functions
2015-03-17 21:34:22 +06:00
dc03a42537
Merge branch 'sohu_fix' of https://github.com/yan12125/youtube-dl into yan12125-sohu_fix
2015-03-17 21:18:36 +06:00
2ebfeacabc
[utils] Keep dot and dotdot unmodified ( Closes #5171 )
2015-03-10 00:50:11 +06:00
f18ef2d144
[utils] Disallow trailing dot in sanitize_path for a path part
2015-03-08 22:08:48 +06:00
a2aaf4dbc6
[utils] Add sanitize_path
2015-03-08 20:55:22 +06:00
55969016e9
[utils] Add a function to sanitize consecutive slashes in URLs
2015-03-06 12:43:49 +08:00
a7440261c5
[utils] Streap leading dots
...
Fixes #2865 , closes #5087
2015-03-02 19:07:19 +01:00
3e675fabe0
[airmozilla] Be more tolerant when nonessential items are missing ( #5030 )
2015-02-26 01:25:00 +01:00
5a42414b9c
[utils] Prevent hyphen at beginning of filename ( Fixes #5035 )
2015-02-24 11:38:01 +01:00
d305dd73a3
[utils] Fix js_to_json
...
Previously, the runtime could be atrocious for longer inputs.
2015-02-18 23:59:51 +01:00
347de4931c
[YoutubeDL] Add generic video filtering ( Fixes #4916 )
...
This functionality is intended to eventually encompass the current format filtering.
2015-02-10 03:32:24 +01:00
9bb8e0a3f9
[wsj] Add new extractor ( Fixes #4854 )
2015-02-03 10:58:28 +01:00
8f4b58d70e
[ntvde] Add new extractor ( Fixes #4850 )
2015-02-02 21:48:54 +01:00
cfb56d1af3
Add --list-thumbnails
2015-01-25 02:43:19 +01:00
61ca9a80b3
[generic] Add support for BOMs ( Fixes #4753 )
2015-01-23 01:21:30 +01:00
a69801e2c6
[utils] Add additional format to unified_strdate
2015-01-14 00:16:34 +02:00
a5fb718c50
[test_utils] Add more tests for parse_duration
2015-01-12 21:39:58 +06:00
2aeb06d6dc
[utils] Improve colon handling ( Fixes #4683 )
2015-01-11 17:40:45 +01:00
0590062925
Respect age_limit when listing extractors ( Fixes #4653 )
2015-01-07 07:20:20 +01:00
cae97f6521
Improve and test ffmpeg version detection
2014-12-14 21:59:59 +01:00
42bdd9d051
[cinchcast] Add new extractor ( Fixes #4428 )
2014-12-12 02:57:36 +01:00
47d7c64274
[test_utils] Make test more realistically ( #4377 )
2014-12-06 12:36:23 +01:00
5f9b83944d
[ffmpeg] Improve version check and call it from hls ( Fixes #4377 )
2014-12-06 12:14:26 +01:00
e8df5cee12
[minhateca] Fix duration parsing
2014-12-04 17:35:40 +01:00
4349c07dd7
[minhateca] Add extractor ( Fixes #4094 )
2014-12-04 17:02:05 +01:00
e075a44afb
[tests] Remove useless u prefixes
2014-11-26 13:07:32 +01:00
be64b5b098
[xminus] Simplify and extend ( #4302 )
2014-11-25 09:54:54 +01:00
8bcc875676
PEP8: more applied
2014-11-23 21:20:46 +01:00
5f6a1245ff
PEP8 applied
2014-11-23 20:41:03 +01:00
7d4111ed14
Provide guidance when called with a YouTube ID starting with a dash.
...
Reported at https://news.ycombinator.com/item?id=8648121
2014-11-23 10:51:09 +01:00
6a68bb574a
[eporner] Fix duration ( Closes #4188 )
2014-11-16 14:55:22 +01:00
609a61e3e6
[npo] Improve npo.nl ( Fixes #4173 )
2014-11-13 16:28:05 +01:00
4c0924bb24
[utils] Fix intlist_to_bytes in Python 2 ( #4181 )
2014-11-13 15:28:42 +01:00
e4bdb37ec6
[spiegel] Add support for embeds
2014-11-13 15:02:31 +01:00
12ea2f30cf
[utils] Remove unused get_meta_content function
2014-11-04 23:20:39 +01:00
8c25f81bee
[util] Move compatibility functions out of util
...
utils is large enough without these compatibility functions.
Everything that is present in newer versions of Python (i.e. with dev Python it's just an import) goes into compat.py .
Everything else (i.e. youtube-dl-specific helpers) goes into utils.py .
2014-11-02 11:23:42 +01:00
7d11297f3f
Merge branch 'master' of github.com:rg3/youtube-dl
2014-10-29 20:10:07 +01:00
6ad4013d40
[drtv] Allow fractional timestamps ( Fixes #4059 )
2014-10-29 20:10:00 +01:00
8abec2c8bb
[test_utils] Fix compat_getenv and compat_expanduser tests on python 3.x
2014-10-29 11:13:34 +01:00
f889cea109
Merge branch 'compat-getenv-and-expanduser' of https://github.com/dstftw/youtube-dl into dstftw-compat-getenv-and-expanduser
...
Conflicts:
test/test_utils.py
youtube_dl/__init__.py
2014-10-26 19:56:52 +07:00
e7b6d12254
[utils] Improve and test js_to_json
2014-10-01 00:08:34 +02:00
410f3e73ab
[utils] Fix js_to_json
2014-10-01 00:08:28 +02:00
4644ac5527
[core] Decode environment variables with filesystem encoding ( Fixes #3854 , Fixes #3217 , Fixes #2918 )
...
Introduces compat versions of os.getenv and os.path.expanduser
2014-09-30 22:27:53 +07:00
99b67fecc5
[arte] Fix upload date extraction
2014-09-29 12:45:18 +02:00
9c44d2429b
[vimeo:likes] Support large like lists ( Fixes #3847 )
2014-09-29 00:36:06 +02:00
0003a5c416
Merge remote-tracking branch 'dstftw/escape-non-ascii-in-urls'
...
Conflicts:
test/test_utils.py
2014-09-15 15:40:10 +02:00
a020a0dc20
[facebook] Fix support for untitled videos ( Fixes #3757 )
2014-09-15 15:10:24 +02:00
d05cfe0600
[YoutubeDL/utils] Clarify rationale for URL escaping in comment, move escape routines to utils and add some tests
2014-09-13 20:59:16 +07:00
ca7b3246b6
[utils] Improve parse_duration
2014-08-31 06:41:30 +07:00
4e408e479b
[utils] Modernize tests
2014-08-27 19:11:45 +02:00
7adcbe7594
[rtlnl] Extract duration
2014-08-25 12:59:53 +02:00
173a7026d5
[test/test_utils] Fix typo in method name
2014-08-10 11:08:56 +02:00
676eb3f2dd
Fix unicode_escape ( Fixes #2695 )
2014-04-04 23:00:51 +02:00
fac55558ad
[washingtonpost] Add extractor ( Fixes #2622 )
2014-03-24 23:21:20 +01:00
912b38b428
[instagram] Fix info_dict key name
2014-03-24 01:40:09 +01:00
b74fa8cd2c
[facebook] Fix login process
...
It was broken and didn't work in python 3.
And use `_download_webpage` instead of `compat_urllib_request.urlopen`.
2014-03-07 15:25:33 +01:00
62e609ab77
Ignore BOM in batch files ( Fixes #2450 )
2014-02-25 01:43:17 +01:00
2db806b4aa
Improve parse_duration
2014-02-17 03:46:26 +07:00
b53466e168
Fix f4m downloading on Python 2.6
2014-02-15 16:24:43 +01:00
026fcc0495
Fix #2355 (date parsing with dashes)
2014-02-09 18:09:57 +01:00
65697b3bf3
Merge branch 'paged-lists'
...
Conflicts:
test/test_utils.py
youtube_dl/extractor/youtube.py
2014-01-22 20:00:16 +01:00
5aafe895fc
Correct XML ampersand fixup
2014-01-20 22:11:34 +01:00
b7ab059084
Add infrastructure for paged lists
...
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
2014-01-20 11:36:47 +01:00
608d11f515
[cnn] Add multiple formats, duration, and upload_date
2013-12-26 13:49:44 +01:00
d6c7a367e8
[utils] Fix url_basename
2013-12-17 12:32:58 +01:00
29eb517403
Add webpage_url_basename info_dict field ( Fixes #1938 )
2013-12-17 04:13:36 +01:00
f53c966a73
[dailymotion] Extract view count ( #1895 )
2013-12-06 13:36:36 +01:00
a6a173c2fd
utils.shell_quote: Convert the args to unicode strings
...
The youtube test video failed with `UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 34: ordinal not in range(128)`, the problem was with the filenames being encoded.
2013-11-21 14:09:28 +01:00
9d4660cab1
[generic] Support embedded vimeo videos ( #1602 )
2013-10-15 12:05:13 +02:00
44a5f1718a
Simplify tests
...
* Make them directly executable again
* Move common stuff (md5, parameters) to helper
* Never import *
* General clean up
2013-10-15 02:00:55 +02:00
d7e66d39a0
Add an extractor for internetvideoarchive.com videos
...
It's used by videodetective.com
2013-10-12 21:34:04 +02:00
a921f40799
[ustream] Simplify channel extraction
...
the ChannelParser has been moved to a new function in utils get_meta_content
Instead of the SocialStreamParser now it uses a regex
2013-09-13 22:05:29 +02:00
59ae56fad5
Add helper function find_path_attr
2013-07-11 16:12:08 +02:00
37254abc36
Allow to use relative dates in the format (now|today)[+-][0-9](day|week|month|year)(s)? ( Closes #137 )
...
Also fix DateRange not accepting ranges of one day.
2013-04-28 11:39:37 +02:00
bf50b0383e
Fix some IEs that didn't return the uploade_date in the YYYYMMDD format
...
Create a function unified_strdate in utils.py to fix these problems
2013-04-27 15:14:20 +02:00
bd55852517
Allow to select videos to download by their upload dates (related #137 )
...
Only absolute dates.
2013-04-27 14:01:55 +02:00
314d506b96
Do not use deprecated method
2012-12-20 13:26:37 +01:00
e387eb5aba
Let youtube IE handle IDs starting with PL ( Closes #572 )
2012-12-04 10:59:38 +01:00
796173d08b
Keep video IDs verbatim if possible ( Closes #571 )
2012-12-03 15:36:41 +01:00
627dcfff39
Restrict more characters ( Closes #566 )
2012-11-28 12:59:27 +01:00
59ae15a507
Convert all tabs to 4 spaces (PEP8)
2012-11-28 02:04:46 +01:00
8cd10ac4ef
Fix printing title etc.
2012-11-28 00:46:21 +01:00
dae7c920f6
Make test_utils.py run on Python 3
2012-11-27 23:20:29 +01:00
dffe658bac
Remove exclamation mark in --restrict-filenames mode
2012-11-27 17:15:33 +01:00
1a911e60a4
Add test for asian characters ( #551 )
2012-11-27 16:07:52 +01:00
46cbda0be4
Minor filename encoding improvement in a common case
2012-11-27 15:07:10 +01:00
c7032546f1
Clean up test
2012-11-27 12:46:27 +01:00
56781d3d2e
Switch back to underline for invalid characters, and make restricted ASCII-only
2012-11-27 12:46:09 +01:00
1c469a9480
New optoin --restrict-filenames
2012-11-26 23:58:46 +01:00
156a59e7a9
Additional tests in file name sanitation
2012-10-29 08:19:54 +01:00
42cb53fcfa
modified filename escaping to a "smarter" one
2012-10-28 22:47:02 +01:00
4279a0ca98
correcting test to be compatible with python2.6
2012-10-19 12:53:20 +02:00
137c5803c3
some changes to keep the same standard
2012-10-12 20:25:22 +02:00
80a846e119
correction on the test for the utils.py
2012-10-12 20:24:58 +02:00
44fb345437
adding TestCase class and corresponding test
2012-10-12 20:24:58 +02:00
9993976ae4
correction on the sanitize title method, change in title resulting
2012-10-12 20:24:58 +02:00
Sergey M․
Yen Chi Hsuan
Jaime Marquínez Ferrándiz
Naglis Jonaitis
Philipp Hagemeister
Jouke Waleson
Filippo Valsorda
gcmalloc