5a1fbbf8b7
[extractor/common] Fix inline HTML5 media tags processing and add test ( closes #27345 )
2020-12-09 00:05:21 +07:00
191286265d
[youtube:tab] Fix feeds extraction ( closes #25695 , closes #26452 )
2020-11-24 00:10:25 +07:00
71ddc222ad
Fix typos ( #27084 )
...
* spelling: authorization
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: brightcove
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: creation
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: exceeded
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: exception
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: extension
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: extracting
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: extraction
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: frontline
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: improve
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: length
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: listsubtitles
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: multimedia
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: obfuscated
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: partitioning
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: playlist
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: playlists
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: restriction
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: services
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: split
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: srmediathek
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: support
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: thumbnail
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: verification
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: whitespaces
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-11-21 22:00:05 +07:00
ab0eda99e1
[YoutubeDL] Fix --ignore-errors for playlists with generator-based entries of url_transparent ( closes #27064 )
2020-11-21 04:00:08 +07:00
2864179293
[youtube] Improve extraction
...
+ Add support for --no-playlist (closes #27009 )
* Improve playlist and mix extraction (closes #26390 , closes #26509 , closes #26534 , closes #27011 )
+ Extract playlist uploader data
* Update tests
2020-11-18 03:34:08 +07:00
fe07e788bf
[utils] Skip ! prefixed code in js_to_json
2020-11-17 01:30:43 +07:00
2de2ca6659
[youtube] Rework extractors
...
WIP
2020-11-12 06:16:37 +07:00
4eda10499e
[utils] Don't attempt to coerce JS strings to numbers in js_to_json ( #26851 )
...
The current logic in `js_to_json` tries to rewrite octal/hex numbers to
decimal. However, when the logic actually happens the `"` or `'` have
already been trimmed off. This causes what were originally strings, that
happen to look like octal/hex numbers, to get rewritten to decimal and
returned as a number rather than a string.
In practive something like:
```js
{
"0x40": "foo",
"040": "bar",
}
```
would get rewritten as:
```json
{
64: "foo",
32: "bar
}
```
This is problematic since this isn't valid JSON as you cannot have
non-string keys.
2020-10-18 00:10:41 +07:00
1d9bf655e6
[utils] Recognize wav mimetype ( closes #26463 )
2020-09-06 11:19:53 +07:00
84213ea8d4
[youtube] Extract chapters from JSON ( closes #24819 )
2020-06-06 04:22:10 +07:00
c380cc28c4
[utils] Improve cookie files support
...
+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)
2020-05-05 04:21:25 +07:00
e40c758c2a
[youtube] Improve player id extraction and add tests
2020-05-02 07:18:08 +07:00
042b664933
Revert "[utils] Add support for cookies with spaces used instead of tabs"
...
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.
1. https://curl.haxx.se/docs/http-cookies.html
This reverts commit cff99c91d1
2020-03-10 04:53:51 +07:00
cff99c91d1
[utils] Add support for cookies with spaces used instead of tabs
2020-03-08 18:01:32 +07:00
ea17979d83
[test_subtitles] Remove obsolete test
2020-02-29 22:08:43 +07:00
4e9e1e240d
[test_YoutubeDL] Add tests for #10591 ( closes #23873 )
2020-02-15 03:37:31 +07:00
e0abaab293
[test_YoutubeDL] Fix get_ids
2020-02-15 03:37:25 +07:00
42db58ec73
[utils] Improve str_to_int
2019-12-15 23:15:24 +07:00
348c6bf1c1
[utils] handle int values passed to str_to_int
2019-11-29 17:39:18 +01:00
1ced222120
[utils] Add generic caesar cipher and rot47
2019-11-27 02:26:42 +07:00
9d30c2132a
[utils] Handle rd-suffixed day parts in unified_strdate ( #23199 )
2019-11-27 00:08:37 +07:00
237513e801
[yahoo] restore support for cbs suffixed URLs
2019-10-31 07:38:53 +01:00
824fa51165
[utils] Improve subtitles_filename ( closes #22753 )
2019-10-18 04:03:53 +07:00
28cc2241e4
[utils] Restrict parse_codecs and add theora as known vcodec ( #21381 )
2019-06-14 01:56:17 +07:00
53cd37bac5
[utils] Improve strip_or_none
2019-05-24 00:03:01 +07:00
3089bc748c
Fix W504 and disable W503 ( closes #20863 )
2019-05-11 03:57:40 +07:00
fd35d8cdfd
[utils] Transliterate "þ" as "th" ( #20897 )
...
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":
$ echo þ-Þ | iconv -t ASCII//TRANSLIT
th-TH
2019-05-11 01:42:31 +07:00
5e1271c56d
[utils] Improve int_or_none and float_or_none ( #20403 )
2019-03-23 01:08:54 +07:00
d493f15c11
[extractor/common] Improve HTML5 entries extraction and add some realworld tests
2019-03-17 09:09:32 +07:00
0dc41787af
[utils] Introduce parse_bitrate
2019-03-17 09:07:47 +07:00
2e27421c70
[test_InfoExtractor] Add test for #20346
2019-03-15 01:20:24 +07:00
067aa17edf
Start moving to ytdl-org
2019-03-11 04:00:54 +07:00
fca9baf0da
[test] Fix test_compat_etree_Element
2019-03-06 02:46:26 +07:00
399f76870d
[compat] Introduce compat_etree_Element
2019-03-06 01:18:52 +07:00
e7e62441cd
[utils] strip #HttpOnly_ prefix from cookies files ( #20219 )
2019-03-03 19:23:59 +07:00
22f5f5c6fc
[malltv] Add extractor ( closes #18058 )
2019-02-08 00:43:26 +07:00
e118a8794f
[YoutubeDL] Fix typo in string negation implementation and add more tests ( closes #18961 )
2019-01-24 01:39:39 +07:00
fad4ceb534
[utils] Fix urljoin for paths with non-http(s) schemes
2019-01-20 20:22:19 +07:00
fc746c3fdd
[test/test_InfoExtractor] add test for #18923
2019-01-20 09:05:12 +01:00
2cc779f497
[YoutubeDL] Add negation support for string comparisons in format selection expressions ( closes #18600 , closes #18805 )
2019-01-20 13:48:49 +07:00
a16c7c033a
[test/helper] Add support for maxcount and count collection len test checkers
2019-01-16 02:17:49 +07:00
6e29458f24
[test/testdata/cookies/session_cookies.txt] Fix empty expires test data
2018-12-10 04:30:00 +07:00
9e02c2c704
[YoutubeDLCookieJar] Add test for keeping session cookies
2018-12-09 22:57:00 +07:00
6864855eb1
[tests] Fix invalid escape sequences
2018-11-23 00:43:42 +07:00
95e42d7336
[extractor/common] Ensure response handle is not prematurely closed before it can be read if it matches expected_status ( resolves #17195 , closes #17846 , resolves #17447 )
2018-11-03 01:18:20 +07:00
25d110be30
[utils] Properly recognize AV1 codec ( closes #17506 )
2018-09-10 02:37:22 +07:00
af03000ad5
[utils] Introduce url_or_none
2018-07-21 18:03:58 +07:00
e9c671d5e8
[utils] Allow JSONP with empty func name ( closes #17028 )
2018-07-21 12:30:18 +07:00
85750f8972
[openload] Improve ext extraction
2018-06-02 00:16:22 +07:00
3bb3ff38a1
[test_utils] add tests for b836118724
2018-05-23 12:20:05 +01:00
6cc622327f
[utils] Introduce merge_dicts
2018-04-28 02:47:17 +07:00
1cc47c6674
[utils] Fix match_str for boolean meta fields
2018-04-24 23:54:49 +07:00
518d5ba519
Fix some tests
2018-04-18 00:10:02 +07:00
47a5cb7734
Generalize XML manifest processing code and improve XSPF parsing ( closes #15794 )
2018-03-18 02:52:17 +07:00
96b8b9abae
[extractor/generic] Support relative URIs in _parse_xspf
...
<location> can have relative URIs, not just absolute.
2018-03-18 02:48:44 +07:00
f226880c6d
[tennistv] Add support for tennistv.com
2018-03-14 09:55:21 +01:00
b871d7e954
[utils] Add parse_resolution
2018-03-02 23:39:04 +07:00
befa4708fd
[utils] Fixup some common URL's typos in sanitize_url ( closes #15649 )
2018-02-19 22:50:23 +07:00
6fcc053947
[test_downloader_http] Use try_rm
2018-02-03 23:24:27 +07:00
f19eae429a
[test_http] Use 127.0.0.1 instead of localhost
2018-02-03 23:09:25 +07:00
ba515388b8
Introduce --http-chunk-size
2018-02-03 23:08:58 +07:00
655c410063
[test_download] Fix download tests for lazy extractors ( closes #13554 , closes #13757 )
2018-01-21 18:15:11 +07:00
c707b1d828
[test_utils] Add tests for malformed JSON handling in js_to_json
2018-01-20 23:00:09 +07:00
c384d537f8
[util] Improve scientific notation handling in js_to_json ( closes #14789 )
2018-01-20 22:54:21 +07:00
9d6ac71c27
[extractor/common] Fix extraction of DASH formats with the same representation id ( closes #15111 )
2017-12-29 23:14:56 +07:00
b555ae9bf1
[utils] Add another date format pattern ( #14999 )
2017-12-16 21:56:16 +07:00
78593e294c
Add references for #14844
2017-12-02 21:22:43 +07:00
41bf647e89
[extractor/common] Add support for DASH manifests with SegmentLists with bare SegmentURLs
2017-12-02 21:16:36 +07:00
f58a506044
[test_InfoExtractor] Fix flake8
2017-11-27 21:30:47 +08:00
d08dcd2dbd
[test_YoutubeDL] Fix typo ( closes #14856 )
2017-11-26 21:06:14 +07:00
181e381fda
[test_InfoExtractor] Add test for #14660
2017-11-04 22:15:58 +07:00
056653bbb1
[utils] Add support for zero years and months in parse_duration
2017-10-29 07:04:48 +07:00
af0f74288d
[YoutubeDL] Improve _default_format_spec ( closes #14461 )
2017-10-11 23:48:05 +07:00
cd6fc19ed7
[YoutubeDL] Ignore duplicates in --playlist-items
...
E.g. '--playlist-items 2-4,3-4,3' should result in '[2,3,4]', not '[2,3,4,3,4,3]'
2017-10-06 23:50:34 +07:00
86a15ed64b
[test_YoutubeDL] Add test for #14425
2017-10-06 23:41:28 +07:00
3869028ffb
[utils] Use bytes-like objects in dfxp2srt
...
This fixes handling of non-UTF8 TTML subtitles
Closes #14191
2017-09-16 12:18:38 +08:00
95f3f7c20a
[utils] Fix unescapeHTML for misformed string like "&a"" ( #13935 )
2017-08-19 21:40:53 +08:00
4850478543
[extractor/common] Add support for float durations in _parse_mpd_formats ( closes #13919 )
2017-08-15 23:58:00 +07:00
5b232f46dc
[utils] Skip missing params in cli_bool_option ( closes #13865 )
2017-08-09 22:28:19 +07:00
8cda78ef72
[test_YoutubeDL] Add a test for #10083
2017-08-02 23:12:34 +07:00
73095e013f
[options] Typo
2017-07-23 16:24:18 +08:00
905d18a7aa
[options] Correctly hide login info from debug outputs ( #13696 )
...
Iterate over opts instead of PRIVATE_OPTS for both performance and
correctness
2017-07-23 16:22:14 +08:00
0017d9ad6d
[YoutubeDL] Improve default format specification ( closes #13704 )
2017-07-23 00:12:01 +07:00
0396806f67
[YoutubeDL] Do not override id, extractor and extractor_key in url_transparent
...
All these meta fields must be borrowed from final extractor that actually performs extraction.
This commit fixes extractor id in download archives for url_transparent downloads. Previously, 'transparent' extractor was erroneously
used for extractor archive id, e.g. 'eggheadlesson 4n8ugwwj5t' instead of 'wistia 4n8ugwwj5t'.
2017-07-21 00:13:32 +07:00
15da37c7dc
[YoutubeDL] Don't expand env variables in meta fields ( closes #13637 )
2017-07-14 00:42:12 +07:00
dee2ff1d81
[test_utils] Fix tests under Windows
2017-07-06 00:25:37 +07:00
609ff8ca19
[utils] Support attributes with no values in get_elements_by_attribute()
2017-07-05 23:27:12 +08:00
b4a3d461e4
[utils] Handle HTMLParseError in extract_attributes ( closes #13349 )
2017-06-12 01:52:24 +07:00
2ae2ffda5e
[utils] Improve unified_timestamp
2017-06-11 21:27:22 +07:00
39d4c1be4d
[youtube] Improve chapters extraction ( closes #13247 )
2017-06-01 23:29:45 +07:00
5552c9eb0f
[utils] Recognize more patterns in strip_jsonp()
...
Used in Youku Show pages
2017-05-26 21:58:18 +08:00
e4a75d7932
[test_youtube_chapters] PEP 8
2017-05-07 00:00:11 +07:00
a57a8e9918
[test_youtube_chapters] Add coding cookie
2017-05-06 05:30:56 +07:00
9cafc3fd8b
[youtube] Extract chapters
2017-05-06 02:27:06 +07:00
0c26548601
[cda] Implement birthday verification ( closes #12789 )
2017-05-04 16:26:17 +08:00
deef31955b
[utils] Improve unified_timestamp
...
Seen at http://zaq1.pl/video/xev0e
2017-04-30 21:45:53 +07:00
9222d94510
[test_utils] Add one more clean_html test
2017-04-28 18:05:14 +02:00
30bb6ce1a4
[test_InfoExtractor] Fix test_parse_m3u8_formats
2017-04-28 03:01:43 +07:00
374560f018
[test_download] Fix order when testing file's md5
2017-04-27 22:27:34 +07:00
ff99fe529e
Don't list master m3u8 playlists in format list ( closes #12832 )
2017-04-27 21:53:17 +07:00
538eee7b6a
Add missing test m3u8 file
2017-04-25 22:26:30 +07:00
ddd258f922
[test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag
2017-04-23 11:49:57 +07:00
cb2520802d
[extractor/common] Improve m3u8 extraction ( closes #12211 )
...
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from #11507 , #11995 , #12211 and twitch vod
2017-04-22 07:01:00 +07:00
5b995f713b
[utils] add support for ttml styles
2017-04-19 14:38:40 +01:00
51350db5a3
[test_YoutubeDL] Add test for #11163
2017-04-16 01:14:05 +07:00
364a69e8c6
[test_download] Fix testing playlists with single video and add comments
2017-04-11 00:01:02 +07:00
60e5016199
[test_download] Remove unused import
2017-04-08 14:19:01 +07:00
c4d6fc6d65
[test_subtitles] Fix raiplay test
2017-04-08 14:11:20 +07:00
80b2fdf9ac
[test_download] Match info dicts against tests before matching info file
2017-04-08 14:10:12 +07:00
3bef10a50c
[test_download] typo in comment
2017-04-08 13:07:25 +07:00
a84da06f49
[test_download] Improve diagnostic on wrong 'id'
2017-04-08 13:07:25 +07:00
942b44a052
[test_compat] Do not use dash in env variables' names
2017-03-26 03:24:25 +07:00
a426ef6d78
[test_utils] Do not use dash in env variables' names
2017-03-26 03:22:48 +07:00
41c5e60dd5
[test_utils] Fix expand_path tests
2017-03-26 03:07:56 +07:00
51098426b8
[utils] Introduce expand_path
2017-03-26 02:30:10 +07:00
c6c22e984d
[test_download] Print additional IEs in summary output
2017-03-25 22:36:40 +08:00
7f3590c43b
[test_InfoExtractor] Add some realworld tests for _extract_jwplayer_data
2017-03-17 00:00:01 +07:00
4b5de77bdb
[utils] Process bytestrings in urljoin ( closes #12369 )
2017-03-06 03:57:46 +07:00
f48409c7ac
[utils] Add pkcs1pad
...
Used in daisuki.net (#4738 )
2017-02-28 22:10:31 +08:00
c9619f0a17
[aes] Add aes_cbc_encrypt
...
Used in daisuki.net (#4738 )
2017-02-28 22:10:31 +08:00
d0d9ade486
[YoutubeDL] Add support for string formatting operations in output template
2017-02-23 22:57:53 +08:00
8936f68a0b
[travis] Run tests in parallel
...
[test_download] Print test names in case of network errors
[test_download] Add comments for nose parameters
[test_download] Modify outtmpl to prevent info JSON filename conflicts
Thanks @jaimeMF for the idea.
[travis] Only download tests should be run in parallel
2017-02-19 21:26:35 +08:00
398dea3210
[test_YoutubeDL] Fix invalid escape sequences
2017-02-15 23:20:46 +07:00
db13c16ef8
[utils] Add support for quoted string literals in --match-filter ( closes #8050 , closes #12142 , closes #12144 )
2017-02-15 23:12:10 +07:00
2af12ad9d2
Introduce get_elements_by_class and get_elements_by_attribute utility functions
2017-02-11 17:16:54 +08:00
4195096ea8
[utils] Improve comments processing in js_to_json ( closes #11947 )
2017-02-03 03:04:33 +07:00
b3ee552e4b
[utils] Handle single-line comments in js_to_json
2017-02-03 03:04:33 +07:00
15846398ca
[utils] Improve parse_duration
2017-01-26 23:23:08 +07:00
cb655f34fb
[utils] Add more date formats
2017-01-12 22:39:45 +07:00
7fe1592073
[common] fix dash codec information for mixed videos and fragment url construction( #11490 )
2016-12-20 12:35:03 +01:00
b0c65c677f
[utils] Improve urljoin
2016-12-17 18:49:55 +07:00
e34c33614d
[utils] Add convenience urljoin
2016-12-13 02:23:49 +07:00
582be35847
Update coding style after pycodestyle 2.1.0
...
In pycodestyle 2.1.0, E305 was introduced, which requires two blank
lines after top level declarations, too.
See https://github.com/PyCQA/pycodestyle/issues/400
See also #10689 ; thanks @stepshal for first mentioning this issue and
initial patches
2016-11-17 19:45:42 +08:00
189935f159
[jsinterp] Fix function calls without arguments.
2016-11-11 15:36:57 +01:00
02dc0a36b7
[utils] Introduce base_url
2016-11-02 02:30:18 +07:00
e5a088dc4b
[utils] Fix --match-filter for int-like strings ( closes #11082 )
2016-10-31 23:32:08 +07:00
6f20b65e72
[test/test_http] Update tests
...
After switching to HTML5 extraction helpers in generic.py, the result
info_dict is always a playlist.
2016-10-12 01:41:41 +08:00
c6eed6b8c0
[utils] Lower priority for rare date formats and add tests
2016-09-29 23:52:29 +07:00
3e4185c396
[utils] Use native french month names
2016-09-14 23:59:38 +07:00
f6717dec8a
[utils] Improve month_by_name and add tests
2016-09-14 23:59:38 +07:00
6562d34a8c
[utils] Improve mimetype2ext
2016-09-02 22:57:48 +07:00
70852b47ca
[utils] Recognize units with full names in parse_filename
...
Reference: https://en.wikipedia.org/wiki/Template:Quantities_of_bytes
2016-08-20 00:17:26 +08:00
e4659b4547
[utils] Correct octal/hexadecimal number detection in js_to_json
2016-08-19 20:37:17 +08:00
13585d7682
[utils] Recognize lowercase units in parse_filesize
2016-08-18 23:32:00 +07:00
52aa7e7476
[test_verbose_output] Fix tests under Python 3
2016-08-13 17:36:14 +08:00
5f2c2b7936
[test_utils] add test for option with not str value
2016-08-13 09:54:12 +01:00
a8795327ca
[utils] Add support TV Parental Guidelines ratings in parse_age_limit
2016-08-07 20:45:18 +07:00
7dc2a74e0a
[utils] Fix unified_timestamp for formats parsed by parsedate_tz()
2016-08-05 11:41:55 +08:00
b070564efb
[extractor/common] Support multiple properties in _og_search_property
2016-08-02 22:55:14 +07:00
ce28252c48
[options] Add test that checks that --password=secret is hidden in verbose output
2016-08-02 17:03:46 +02:00
Sergey M․
Josh Soref
Kevin O'Connor
Remita Amine
InfernalUnderling
Jakub Wilk
Ales Jirasek
Xiao Di Guan
Enes
Dan Salmon
Ricardo Constantino
Philipp Hagemeister
squibbysquibby
Mike Fährmann
Petr Novak
Yen Chi Hsuan
Tithen-Firion
John Hawkinson
Thomas Christlieb
Michal Čihař
Kacper Michajłow