Compare commits

..

10 Commits

Author SHA1 Message Date
pukkandan
c220d9efc8
[ffmpeg] Disable avconv unless --prefer-avconv 2022-08-09 05:15:38 +05:30
pukkandan
81e0195998
[build] Fix changelog
Bug in c4b6c5c7c9
2022-08-09 03:58:29 +05:30
github-actions
f1e2d4a9a2 [version] update
Created by: pukkandan

:ci skip all :ci run dl
2022-08-08 22:15:24 +00:00
pukkandan
3157158f76
Release 2022.08.08 2022-08-09 03:40:00 +05:30
pukkandan
16d4535abc
Update to ytdl-commit-adb5294
[aenetworks] Update _THEPLATFORM_KEY and _THEPLATFORM_SECRET
adb5294177
2022-08-09 02:55:30 +05:30
Elyse
2a5e5477bc
[extractor/redbee] Unify and update extractors (#4479)
Closes #4443
Authored by: elyse0
2022-08-09 02:41:47 +05:30
Eren Kemer
e251986cbe
[extractor/harpodeon] Add extractor (#4540)
Closes #4450
Authored by: eren-kemer
2022-08-09 02:39:37 +05:30
pukkandan
f0ad6f8c51
Remove filtered entries from -J
Closes #4369
2022-08-09 02:01:40 +05:30
pukkandan
70b2340909
[build, devscripts] Add devscript to set a build variant
Closes #4471
2022-08-09 01:08:48 +05:30
pukkandan
115add4387
[devscripts] Create utils and refactor 2022-08-09 01:08:47 +05:30
41 changed files with 860 additions and 419 deletions

View File

@ -2,6 +2,13 @@ name: Broken site
description: Report broken or misfunctioning site
labels: [triage, site-bug]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: checkboxes
id: checklist
attributes:
@ -11,7 +18,7 @@ body:
options:
- label: I'm reporting a broken site
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true
@ -55,7 +62,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -63,8 +70,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell
validations:

View File

@ -2,6 +2,13 @@ name: Site support request
description: Request support for a new site
labels: [triage, site-request]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: checkboxes
id: checklist
attributes:
@ -11,7 +18,7 @@ body:
options:
- label: I'm reporting a new site support request
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true
@ -67,7 +74,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -75,8 +82,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell
validations:

View File

@ -2,6 +2,13 @@ name: Site feature request
description: Request a new functionality for a supported site
labels: [triage, site-enhancement]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: checkboxes
id: checklist
attributes:
@ -11,7 +18,7 @@ body:
options:
- label: I'm requesting a site-specific feature
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true
@ -63,7 +70,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -71,8 +78,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell
validations:

View File

@ -2,6 +2,13 @@ name: Bug report
description: Report a bug unrelated to any particular site or extractor
labels: [triage, bug]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: checkboxes
id: checklist
attributes:
@ -11,7 +18,7 @@ body:
options:
- label: I'm reporting a bug unrelated to a specific site
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true
@ -48,7 +55,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -56,8 +63,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell
validations:

View File

@ -2,6 +2,13 @@ name: Feature request
description: Request a new functionality unrelated to any particular site or extractor
labels: [triage, enhancement]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: checkboxes
id: checklist
attributes:
@ -13,7 +20,7 @@ body:
required: true
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true
@ -44,7 +51,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -52,7 +59,7 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell

View File

@ -2,12 +2,19 @@ name: Ask question
description: Ask yt-dlp related question
labels: [question]
body:
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
required: true
- type: markdown
attributes:
value: |
### Make sure you are **only** asking a question and not reporting a bug or requesting a feature.
If your question contains "isn't working" or "can you add", this is most likely the wrong template.
If you are in doubt whether this is the right template, **use another template**!
If you are in doubt whether this is the right template, **USE ANOTHER TEMPLATE**!
- type: checkboxes
id: checklist
attributes:
@ -19,7 +26,7 @@ body:
required: true
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
required: true
- label: I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- label: I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
required: true
@ -50,7 +57,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [9d339c4] (win32_exe)
[debug] yt-dlp version 2022.08.08 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
@ -58,7 +65,7 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
<more lines>
render: shell

View File

@ -2,6 +2,7 @@ name: Broken site
description: Report broken or misfunctioning site
labels: [triage, site-bug]
body:
%(no_skip)s
- type: checkboxes
id: checklist
attributes:

View File

@ -2,6 +2,7 @@ name: Site support request
description: Request support for a new site
labels: [triage, site-request]
body:
%(no_skip)s
- type: checkboxes
id: checklist
attributes:

View File

@ -2,6 +2,7 @@ name: Site feature request
description: Request a new functionality for a supported site
labels: [triage, site-enhancement]
body:
%(no_skip)s
- type: checkboxes
id: checklist
attributes:

View File

@ -2,6 +2,7 @@ name: Bug report
description: Report a bug unrelated to any particular site or extractor
labels: [triage, bug]
body:
%(no_skip)s
- type: checkboxes
id: checklist
attributes:

View File

@ -2,6 +2,7 @@ name: Feature request
description: Request a new functionality unrelated to any particular site or extractor
labels: [triage, enhancement]
body:
%(no_skip)s
- type: checkboxes
id: checklist
attributes:

View File

@ -2,12 +2,13 @@ name: Ask question
description: Ask yt-dlp related question
labels: [question]
body:
%(no_skip)s
- type: markdown
attributes:
value: |
### Make sure you are **only** asking a question and not reporting a bug or requesting a feature.
If your question contains "isn't working" or "can you add", this is most likely the wrong template.
If you are in doubt whether this is the right template, **use another template**!
If you are in doubt whether this is the right template, **USE ANOTHER TEMPLATE**!
- type: checkboxes
id: checklist
attributes:

View File

@ -1,3 +1,5 @@
**IMPORTANT**: PRs without the template will be CLOSED
### Description of your *pull request* and other information
</details>

View File

@ -89,6 +89,7 @@ jobs:
if: "env.TWINE_PASSWORD != ''"
run: |
rm -rf dist/*
python devscripts/set-variant.py pip -M "You installed yt-dlp with pip or using the wheel from PyPi; Use that to update"
python setup.py sdist bdist_wheel
twine upload dist/*
@ -256,7 +257,7 @@ jobs:
- name: Get Changelog
run: |
changelog=$(grep -oPz '(?s)(?<=### ${{ steps.bump_version.outputs.ytdlp_version }}\n{2}).+?(?=\n{2,3}###)' Changelog.md) || true
changelog=$(grep -oPz '(?s)(?<=### ${{ needs.prepare.outputs.ytdlp_version }}\n{2}).+?(?=\n{2,3}###)' Changelog.md) || true
echo "changelog<<EOF" >> $GITHUB_ENV
echo "$changelog" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV

View File

@ -285,3 +285,12 @@ odo2063
pritam20ps05
scy
sheerluck
AxiosDeminence
DjesonPV
eren-kemer
freezboltz
Galiley
haobinliang
Mehavoid
winterbird-code
yashkc2025

View File

@ -11,6 +11,92 @@
-->
### 2022.08.08
* **Remove Python 3.6 support**
* Determine merge container better by [pukkandan](https://github.com/pukkandan), [selfisekai](https://github.com/selfisekai)
* Framework for embed detection by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
* Merge youtube-dl: Upto [commit/adb5294](https://github.com/ytdl-org/youtube-dl/commit/adb5294)
* `--compat-option no-live-chat` should disable danmaku
* Fix misleading DRM message
* Import ctypes only when necessary
* Minor bugfixes by [pukkandan](https://github.com/pukkandan)
* Reject entire playlists faster with `--match-filter` by [pukkandan](https://github.com/pukkandan)
* Remove filtered entries from `-J`
* Standardize retry mechanism by [pukkandan](https://github.com/pukkandan)
* Validate `--merge-output-format`
* [downloader] Add average speed to final progress line
* [extractor] Add field `audio_channels`
* [extractor] Support multiple archive ids for one video
* [ffmpeg] Set `ffmpeg_location` in a contextvar
* [FFmpegThumbnailsConvertor] Fix conversion from GIF
* [MetadataParser] Don't set `None` when the field didn't match
* [outtmpl] Smarter replacing of unsupported characters by [pukkandan](https://github.com/pukkandan)
* [outtmpl] Treat empty values as None in filenames
* [utils] sanitize_open: Allow any IO stream as stdout
* [build, devscripts] Add devscript to set a build variant
* [build] Improve build process by [shirt-dev](https://github.com/shirt-dev)
* [build] Update pyinstaller
* [devscripts] Create `utils` and refactor
* [docs] Clarify `best*`
* [docs] Fix bug report issue template
* [docs] Fix capitalization in references by [christoph-heinrich](https://github.com/christoph-heinrich)
* [cleanup, mhtml] Use imghdr
* [cleanup, utils] Consolidate known media extensions
* [cleanup] Misc fixes and cleanup
* [extractor/angel] Add extractor by [AxiosDeminence](https://github.com/AxiosDeminence)
* [extractor/dplay] Add MotorTrend extractor by [Sipherdrakon](https://github.com/Sipherdrakon)
* [extractor/harpodeon] Add extractor by [eren-kemer](https://github.com/eren-kemer)
* [extractor/holodex] Add extractor by [pukkandan](https://github.com/pukkandan), [sqrtNOT](https://github.com/sqrtNOT)
* [extractor/kompas] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/rai] Add raisudtirol extractor by [nixxo](https://github.com/nixxo)
* [extractor/tempo] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/youtube] **Fixes for third party client detection** by [coletdjnz](https://github.com/coletdjnz)
* [extractor/youtube] Add `live_status=post_live` by [lazypete365](https://github.com/lazypete365)
* [extractor/youtube] Extract more format info
* [extractor/youtube] Parse translated subtitles only when requested
* [extractor/youtube, extractor/twitch] Allow waiting for channels to become live
* [extractor/youtube, webvtt] Extract auto-subs from livestream VODs by [fstirlitz](https://github.com/fstirlitz), [pukkandan](https://github.com/pukkandan)
* [extractor/AbemaTVTitle] Implement paging by [Lesmiscore](https://github.com/Lesmiscore)
* [extractor/archiveorg] Improve handling of formats by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
* [extractor/arte] Fix title extraction
* [extractor/arte] **Move to v2 API** by [fstirlitz](https://github.com/fstirlitz), [pukkandan](https://github.com/pukkandan)
* [extractor/bbc] Fix news articles by [ajj8](https://github.com/ajj8)
* [extractor/camtasia] Separate into own extractor by [coletdjnz](https://github.com/coletdjnz)
* [extractor/cloudflarestream] Fix video_id padding by [haobinliang](https://github.com/haobinliang)
* [extractor/crunchyroll] Fix conversion of thumbnail from GIF by [pukkandan](https://github.com/pukkandan)
* [extractor/crunchyroll] Handle missing metadata correctly by [Burve](https://github.com/Burve), [pukkandan](https://github.com/pukkandan)
* [extractor/crunchyroll:beta] Extract timestamp and fix tests by [tejing1](https://github.com/tejing1)
* [extractor/crunchyroll:beta] Use streams API by [tejing1](https://github.com/tejing1)
* [extractor/doodstream] Support more domains by [Galiley](https://github.com/Galiley)
* [extractor/ESPN] Extract duration by [ischmidt20](https://github.com/ischmidt20)
* [extractor/FIFA] Change API endpoint by [Bricio](https://github.com/Bricio), [yashkc2025](https://github.com/yashkc2025)
* [extractor/globo:article] Remove false positives by [Bricio](https://github.com/Bricio)
* [extractor/Go] Extract timestamp by [ischmidt20](https://github.com/ischmidt20)
* [extractor/hidive] Fix cookie login when netrc is also given by [winterbird-code](https://github.com/winterbird-code)
* [extractor/html5] Separate into own extractor by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
* [extractor/ina] Improve extractor by [elyse0](https://github.com/elyse0)
* [extractor/NaverNow] Change endpoint by [ping](https://github.com/ping)
* [extractor/ninegag] Extract uploader by [DjesonPV](https://github.com/DjesonPV)
* [extractor/NovaPlay] Fix extractor by [Bojidarist](https://github.com/Bojidarist)
* [extractor/orf:radio] Rewrite extractors
* [extractor/patreon] Fix and improve extractors by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
* [extractor/rai] Fix RaiNews extraction by [nixxo](https://github.com/nixxo)
* [extractor/redbee] Unify and update extractors by [elyse0](https://github.com/elyse0)
* [extractor/stripchat] Fix _VALID_URL by [freezboltz](https://github.com/freezboltz)
* [extractor/tubi] Exclude playlists from playlist entries by [sqrtNOT](https://github.com/sqrtNOT)
* [extractor/tviplayer] Improve `_VALID_URL` by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/twitch] Extract chapters for single chapter VODs by [mpeter50](https://github.com/mpeter50)
* [extractor/vgtv] Support tv.vg.no by [sqrtNOT](https://github.com/sqrtNOT)
* [extractor/vidio] Support embed link by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/vk] Fix extractor by [Mehavoid](https://github.com/Mehavoid)
* [extractor/WASDTV:record] Fix `_VALID_URL`
* [extractor/xfileshare] Add Referer by [Galiley](https://github.com/Galiley)
* [extractor/YahooJapanNews] Fix extractor by [Lesmiscore](https://github.com/Lesmiscore)
* [extractor/yandexmusic] Extract higher quality format
* [extractor/zee5] Update Device ID by [m4tu4g](https://github.com/m4tu4g)
### 2022.07.18
* Allow users to specify encoding in each config files by [Lesmiscore](https://github.com/Lesmiscore)

View File

@ -71,7 +71,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
# NEW FEATURES
* Merged with **youtube-dl v2021.12.17+ [commit/a03b977](https://github.com/ytdl-org/youtube-dl/commit/a03b9775d544b06a5b4f2aa630214c7c22fc2229)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
* Merged with **youtube-dl v2021.12.17+ [commit/adb5294](https://github.com/ytdl-org/youtube-dl/commit/adb5294177265ba35b45746dbb600965076ed150)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
@ -312,7 +312,7 @@ If you do not have the necessary dependencies for a task you are attempting, yt-
## COMPILE
### Standalone PyInstaller Builds
To build the Windows/MacOS executable, you must have Python and `pyinstaller` (plus any of yt-dlp's [optional dependencies](#dependencies) if needed). Once you have all the necessary dependencies installed, simply run `pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the Python used.
To build the standalone executable, you must have Python and `pyinstaller` (plus any of yt-dlp's [optional dependencies](#dependencies) if needed). Once you have all the necessary dependencies installed, simply run `pyinst.py`. The executable will be built for the same architecture (x86/ARM, 32/64 bit) as the Python used.
python3 -m pip install -U pyinstaller -r requirements.txt
python3 devscripts/make_lazy_extractors.py
@ -343,7 +343,8 @@ If you wish to build it anyway, install Python and py2exe, and then simply run `
### Related scripts
* **`devscripts/update-version.py`** - Update the version number based on current timestamp
* **`devscripts/update-version.py [revision]`** - Update the version number based on current date
* **`devscripts/set-variant.py variant [-M update_message]`** - Set the build variant of the executable
* **`devscripts/make_lazy_extractors.py`** - Create lazy extractors. Running this before building the binaries (any variant) will improve their startup performance. Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS=1` if you wish to forcefully disable lazy extractor loading.
You can also fork the project on github and run your fork's [build workflow](.github/workflows/build.yml) to automatically build a full release
@ -360,8 +361,8 @@ You can also fork the project on github and run your fork's [build workflow](.gi
## General Options:
-h, --help Print this help text and exit
--version Print program version and exit
-U, --update Update this program to latest version
--no-update Do not update (default)
-U, --update Update this program to the latest version
--no-update Do not check for updates (default)
-i, --ignore-errors Ignore download and postprocessing errors.
The download will be considered successful
even if the postprocessing fails

View File

@ -7,20 +7,14 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import optparse
import re
def read(fname):
with open(fname, encoding='utf-8') as f:
return f.read()
# Get the version without importing the package
def read_version(fname):
exec(compile(read(fname), fname, 'exec'))
return locals()['__version__']
from devscripts.utils import (
get_filename_args,
read_file,
read_version,
write_file,
)
VERBOSE_TMPL = '''
- type: checkboxes
@ -58,20 +52,24 @@ VERBOSE_TMPL = '''
required: true
'''.strip()
NO_SKIP = '''
- type: checkboxes
attributes:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue
options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\\* field
required: true
'''.strip()
def main():
parser = optparse.OptionParser(usage='%prog INFILE OUTFILE')
_, args = parser.parse_args()
if len(args) != 2:
parser.error('Expected an input and an output filename')
fields = {'version': read_version('yt_dlp/version.py')}
fields = {'version': read_version(), 'no_skip': NO_SKIP}
fields['verbose'] = VERBOSE_TMPL % fields
fields['verbose_optional'] = re.sub(r'(\n\s+validations:)?\n\s+required: true', '', fields['verbose'])
infile, outfile = args
with open(outfile, 'w', encoding='utf-8') as outf:
outf.write(read(infile) % fields)
infile, outfile = get_filename_args(has_infile=True)
write_file(outfile, read_file(infile) % fields)
if __name__ == '__main__':

View File

@ -7,9 +7,10 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import optparse
from inspect import getsource
from devscripts.utils import get_filename_args, read_file, write_file
NO_ATTR = object()
STATIC_CLASS_PROPERTIES = ['IE_NAME', 'IE_DESC', 'SEARCH_KEY', '_VALID_URL', '_WORKING', '_NETRC_MACHINE', 'age_limit']
CLASS_METHODS = [
@ -19,17 +20,11 @@ IE_TEMPLATE = '''
class {name}({bases}):
_module = {module!r}
'''
with open('devscripts/lazy_load_template.py', encoding='utf-8') as f:
MODULE_TEMPLATE = f.read()
MODULE_TEMPLATE = read_file('devscripts/lazy_load_template.py')
def main():
parser = optparse.OptionParser(usage='%prog [OUTFILE.py]')
args = parser.parse_args()[1] or ['yt_dlp/extractor/lazy_extractors.py']
if len(args) != 1:
parser.error('Expected only an output filename')
lazy_extractors_filename = args[0]
lazy_extractors_filename = get_filename_args(default_outfile='yt_dlp/extractor/lazy_extractors.py')
if os.path.exists(lazy_extractors_filename):
os.remove(lazy_extractors_filename)
@ -46,8 +41,7 @@ def main():
*build_ies(_ALL_CLASSES, (InfoExtractor, SearchInfoExtractor), DummyInfoExtractor),
))
with open(lazy_extractors_filename, 'wt', encoding='utf-8') as f:
f.write(f'{module_src}\n')
write_file(lazy_extractors_filename, f'{module_src}\n')
def get_all_ies():

View File

@ -5,10 +5,17 @@ yt-dlp --help | make_readme.py
This must be run in a console of correct width
"""
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import functools
import re
import sys
from devscripts.utils import read_file, write_file
README_FILE = 'README.md'
@ -38,6 +45,10 @@ switch_col_width = len(re.search(r'(?m)^\s{5,}', options).group())
delim = f'\n{" " * switch_col_width}'
PATCHES = (
( # Standardize update message
r'(?m)^( -U, --update\s+).+(\n \s.+)*$',
r'\1Update this program to the latest version',
),
( # Headings
r'(?m)^ (\w.+\n)( (?=\w))?',
r'## \1'
@ -63,12 +74,10 @@ PATCHES = (
),
)
with open(README_FILE, encoding='utf-8') as f:
readme = f.read()
readme = read_file(README_FILE)
with open(README_FILE, 'w', encoding='utf-8') as f:
f.write(''.join((
take_section(readme, end=f'## {OPTIONS_START}'),
functools.reduce(apply_patch, PATCHES, options),
take_section(readme, f'# {OPTIONS_END}'),
)))
write_file(README_FILE, ''.join((
take_section(readme, end=f'## {OPTIONS_START}'),
functools.reduce(apply_patch, PATCHES, options),
take_section(readme, f'# {OPTIONS_END}'),
)))

View File

@ -7,21 +7,13 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import optparse
from devscripts.utils import get_filename_args, write_file
from yt_dlp.extractor import list_extractor_classes
def main():
parser = optparse.OptionParser(usage='%prog OUTFILE.md')
_, args = parser.parse_args()
if len(args) != 1:
parser.error('Expected an output filename')
out = '\n'.join(ie.description() for ie in list_extractor_classes() if ie.IE_DESC is not False)
with open(args[0], 'w', encoding='utf-8') as outf:
outf.write(f'# Supported sites\n{out}\n')
write_file(get_filename_args(), f'# Supported sites\n{out}\n')
if __name__ == '__main__':

View File

@ -1,9 +1,22 @@
#!/usr/bin/env python3
import optparse
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import os.path
import re
from devscripts.utils import (
compose_functions,
get_filename_args,
read_file,
write_file,
)
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
README_FILE = os.path.join(ROOT_DIR, 'README.md')
@ -22,25 +35,6 @@ yt\-dlp \- A youtube-dl fork with additional features and patches
'''
def main():
parser = optparse.OptionParser(usage='%prog OUTFILE.md')
_, args = parser.parse_args()
if len(args) != 1:
parser.error('Expected an output filename')
outfile, = args
with open(README_FILE, encoding='utf-8') as f:
readme = f.read()
readme = filter_excluded_sections(readme)
readme = move_sections(readme)
readme = filter_options(readme)
with open(outfile, 'w', encoding='utf-8') as outf:
outf.write(PREFIX + readme)
def filter_excluded_sections(readme):
EXCLUDED_SECTION_BEGIN_STRING = re.escape('<!-- MANPAGE: BEGIN EXCLUDED SECTION -->')
EXCLUDED_SECTION_END_STRING = re.escape('<!-- MANPAGE: END EXCLUDED SECTION -->')
@ -92,5 +86,12 @@ def filter_options(readme):
return readme.replace(section, options, 1)
TRANSFORM = compose_functions(filter_excluded_sections, move_sections, filter_options)
def main():
write_file(get_filename_args(), PREFIX + TRANSFORM(read_file(README_FILE)))
if __name__ == '__main__':
main()

36
devscripts/set-variant.py Normal file
View File

@ -0,0 +1,36 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import argparse
import functools
import re
from devscripts.utils import compose_functions, read_file, write_file
VERSION_FILE = 'yt_dlp/version.py'
def parse_options():
parser = argparse.ArgumentParser(description='Set the build variant of the package')
parser.add_argument('variant', help='Name of the variant')
parser.add_argument('-M', '--update-message', default=None, help='Message to show in -U')
return parser.parse_args()
def property_setter(name, value):
return functools.partial(re.sub, rf'(?m)^{name}\s*=\s*.+$', f'{name} = {value!r}')
opts = parse_options()
transform = compose_functions(
property_setter('VARIANT', opts.variant),
property_setter('UPDATE_HINT', opts.update_message)
)
write_file(VERSION_FILE, transform(read_file(VERSION_FILE)))

View File

@ -1,5 +1,10 @@
#!/usr/bin/env python3
"""
Usage: python3 ./devscripts/update-formulae.py <path-to-formulae-rb> <version>
version can be either 0-aligned (yt-dlp version) or normalized (PyPi version)
"""
# Allow direct execution
import os
import sys
@ -11,8 +16,7 @@ import json
import re
import urllib.request
# usage: python3 ./devscripts/update-formulae.py <path-to-formulae-rb> <version>
# version can be either 0-aligned (yt-dlp version) or normalized (PyPl version)
from devscripts.utils import read_file, write_file
filename, version = sys.argv[1:]
@ -27,11 +31,9 @@ tarball_file = next(x for x in pypi_release['urls'] if x['filename'].endswith('.
sha256sum = tarball_file['digests']['sha256']
url = tarball_file['url']
with open(filename) as r:
formulae_text = r.read()
formulae_text = read_file(filename)
formulae_text = re.sub(r'sha256 "[0-9a-f]*?"', 'sha256 "%s"' % sha256sum, formulae_text, count=1)
formulae_text = re.sub(r'url "[^"]*?"', 'url "%s"' % url, formulae_text, count=1)
with open(filename, 'w') as w:
w.write(formulae_text)
write_file(filename, formulae_text)

View File

@ -7,32 +7,35 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import contextlib
import subprocess
import sys
from datetime import datetime
with open('yt_dlp/version.py') as f:
exec(compile(f.read(), 'yt_dlp/version.py', 'exec'))
old_version = locals()['__version__']
from devscripts.utils import read_version, write_file
old_version_list = old_version.split('.')
old_ver = '.'.join(old_version_list[:3])
old_rev = old_version_list[3] if len(old_version_list) > 3 else ''
def get_new_version(revision):
version = datetime.utcnow().strftime('%Y.%m.%d')
ver = datetime.utcnow().strftime("%Y.%m.%d")
if revision:
assert revision.isdigit(), 'Revision must be a number'
else:
old_version = read_version().split('.')
if version.split('.') == old_version[:3]:
revision = str(int((old_version + [0])[3]) + 1)
rev = (sys.argv[1:] or [''])[0] # Use first argument, if present as revision number
if not rev:
rev = str(int(old_rev or 0) + 1) if old_ver == ver else ''
return f'{version}.{revision}' if revision else version
VERSION = '.'.join((ver, rev)) if rev else ver
try:
sp = subprocess.Popen(['git', 'rev-parse', '--short', 'HEAD'], stdout=subprocess.PIPE)
GIT_HEAD = sp.communicate()[0].decode().strip() or None
except Exception:
GIT_HEAD = None
def get_git_head():
with contextlib.suppress(Exception):
sp = subprocess.Popen(['git', 'rev-parse', '--short', 'HEAD'], stdout=subprocess.PIPE)
return sp.communicate()[0].decode().strip() or None
VERSION = get_new_version((sys.argv + [''])[1])
GIT_HEAD = get_git_head()
VERSION_FILE = f'''\
# Autogenerated by devscripts/update-version.py
@ -40,10 +43,12 @@ VERSION_FILE = f'''\
__version__ = {VERSION!r}
RELEASE_GIT_HEAD = {GIT_HEAD!r}
VARIANT = None
UPDATE_HINT = None
'''
with open('yt_dlp/version.py', 'wt') as f:
f.write(VERSION_FILE)
print('::set-output name=ytdlp_version::' + VERSION)
write_file('yt_dlp/version.py', VERSION_FILE)
print(f'::set-output name=ytdlp_version::{VERSION}')
print(f'\nVersion = {VERSION}, Git HEAD = {GIT_HEAD}')

35
devscripts/utils.py Normal file
View File

@ -0,0 +1,35 @@
import argparse
import functools
def read_file(fname):
with open(fname, encoding='utf-8') as f:
return f.read()
def write_file(fname, content):
with open(fname, 'w', encoding='utf-8') as f:
return f.write(content)
# Get the version without importing the package
def read_version(fname='yt_dlp/version.py'):
exec(compile(read_file(fname), fname, 'exec'))
return locals()['__version__']
def get_filename_args(has_infile=False, default_outfile=None):
parser = argparse.ArgumentParser()
if has_infile:
parser.add_argument('infile', help='Input file')
kwargs = {'nargs': '?', 'default': default_outfile} if default_outfile else {}
parser.add_argument('outfile', **kwargs, help='Output file')
opts = parser.parse_args()
if has_infile:
return opts.infile, opts.outfile
return opts.outfile
def compose_functions(*functions):
return lambda x: functools.reduce(lambda y, f: f(y), functions, x)

View File

@ -1,11 +1,17 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import platform
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import platform
from PyInstaller.__main__ import run as run_pyinstaller
from devscripts.utils import read_version
OS_NAME, MACHINE, ARCH = sys.platform, platform.machine(), platform.architecture()[0][:2]
if MACHINE in ('x86_64', 'AMD64') or ('i' in MACHINE and '86' in MACHINE):
# NB: Windows x86 has MACHINE = AMD64 irrespective of bitness
@ -13,8 +19,7 @@ if MACHINE in ('x86_64', 'AMD64') or ('i' in MACHINE and '86' in MACHINE):
def main():
opts = parse_options()
version = read_version('yt_dlp/version.py')
opts, version = parse_options(), read_version()
onedir = '--onedir' in opts or '-D' in opts
if not onedir and '-F' not in opts and '--onefile' not in opts:
@ -53,13 +58,6 @@ def parse_options():
return opts
# Get the version from yt_dlp/version.py without importing the package
def read_version(fname):
with open(fname, encoding='utf-8') as f:
exec(compile(f.read(), fname, 'exec'))
return locals()['__version__']
def exe(onedir):
"""@returns (name, path)"""
name = '_'.join(filter(None, (

View File

@ -12,28 +12,18 @@ except ImportError:
from distutils.core import Command, setup
setuptools_available = False
from devscripts.utils import read_file, read_version
def read(fname):
with open(fname, encoding='utf-8') as f:
return f.read()
# Get the version from yt_dlp/version.py without importing the package
def read_version(fname):
exec(compile(read(fname), fname, 'exec'))
return locals()['__version__']
VERSION = read_version('yt_dlp/version.py')
VERSION = read_version()
DESCRIPTION = 'A youtube-dl fork with additional features and patches'
LONG_DESCRIPTION = '\n\n'.join((
'Official repository: <https://github.com/yt-dlp/yt-dlp>',
'**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
read('README.md')))
read_file('README.md')))
REQUIREMENTS = read('requirements.txt').splitlines()
REQUIREMENTS = read_file('requirements.txt').splitlines()
def packages():
@ -121,7 +111,7 @@ class build_lazy_extractors(Command):
if self.dry_run:
print('Skipping build of lazy extractors in dry run mode')
return
subprocess.run([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'])
subprocess.run([sys.executable, 'devscripts/make_lazy_extractors.py'])
params = py2exe_params() if sys.argv[1:2] == ['py2exe'] else build_params()

View File

@ -18,7 +18,7 @@
- **8tracks**
- **91porn**
- **9c9media**
- **9gag**
- **9gag**: 9GAG
- **9now.com.au**
- **abc.net.au**
- **abc.net.au:iview**
@ -64,6 +64,7 @@
- **AmericasTestKitchenSeason**
- **AmHistoryChannel**
- **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
- **Angel**
- **AnimalPlanet**
- **AnimeOnDemand**: [<abbr title="netrc machine"><em>animeondemand</em></abbr>]
- **ant1newsgr:article**: ant1news.gr articles
@ -187,6 +188,7 @@
- **Camdemy**
- **CamdemyFolder**
- **CamModels**
- **CamtasiaEmbed**
- **CamWithHer**
- **CanalAlpha**
- **canalc2.tv**
@ -232,6 +234,7 @@
- **Clippit**
- **ClipRs**
- **Clipsyndicate**
- **ClipYouEmbed**
- **CloserToTruth**
- **CloudflareStream**
- **Cloudy**
@ -473,6 +476,7 @@
- **gronkh:feed**
- **gronkh:vods**
- **Groupon**
- **Harpodeon**
- **hbo**
- **HearThisAt**
- **Heise**
@ -491,6 +495,7 @@
- **hitbox:live**
- **HitRecord**
- **hketv**: 香港教育局教育電視 (HKETV) Educational Television, Hong Kong Educational Bureau
- **Holodex**
- **HotNewHipHop**
- **hotstar**
- **hotstar:playlist**
@ -502,6 +507,7 @@
- **HRTiPlaylist**: [<abbr title="netrc machine"><em>hrti</em></abbr>]
- **HSEProduct**
- **HSEShow**
- **html5**
- **Huajiao**: 花椒直播
- **HuffPost**: Huffington Post
- **Hungama**
@ -573,6 +579,7 @@
- **KickStarter**
- **KinjaEmbed**
- **KinoPoisk**
- **KompasVideo**
- **KonserthusetPlay**
- **Koo**
- **KrasView**: Красвью
@ -715,6 +722,7 @@
- **Motherless**
- **MotherlessGroup**
- **Motorsport**: motorsport.com
- **MotorTrend**
- **MovieClips**
- **MovieFap**
- **Moviepilot**
@ -890,21 +898,10 @@
- **openrec:capture**
- **openrec:movie**
- **OraTV**
- **orf:burgenland**: Radio Burgenland
- **orf:fm4**: radio FM4
- **orf:fm4:story**: fm4.orf.at stories
- **orf:iptv**: iptv.ORF.at
- **orf:kaernten**: Radio Kärnten
- **orf:noe**: Radio Niederösterreich
- **orf:oberoesterreich**: Radio Oberösterreich
- **orf:oe1**: Radio Österreich 1
- **orf:oe3**: Radio Österreich 3
- **orf:salzburg**: Radio Salzburg
- **orf:steiermark**: Radio Steiermark
- **orf:tirol**: Radio Tirol
- **orf:radio**
- **orf:tvthek**: ORF TVthek
- **orf:vorarlberg**: Radio Vorarlberg
- **orf:wien**: Radio Wien
- **OsnatelTV**: [<abbr title="netrc machine"><em>osnateltv</em></abbr>]
- **OutsideTV**
- **PacktPub**: [<abbr title="netrc machine"><em>packtpub</em></abbr>]
@ -922,7 +919,7 @@
- **parliamentlive.tv**: UK parliament videos
- **Parlview**
- **Patreon**
- **PatreonUser**
- **PatreonCampaign**
- **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
- **PearVideo**
- **PeekVids**
@ -1030,12 +1027,14 @@
- **radlive:channel**
- **radlive:season**
- **Rai**
- **RaiNews**
- **RaiPlay**
- **RaiPlayLive**
- **RaiPlayPlaylist**
- **RaiPlaySound**
- **RaiPlaySoundLive**
- **RaiPlaySoundPlaylist**
- **RaiSudtirol**
- **RayWenderlich**
- **RayWenderlichCourse**
- **RBMARadio**
@ -1072,7 +1071,7 @@
- **RoosterTeethSeries**: [<abbr title="netrc machine"><em>roosterteeth</em></abbr>]
- **RottenTomatoes**
- **Rozhlas**
- **RTBF**
- **RTBF**: [<abbr title="netrc machine"><em>rtbf</em></abbr>]
- **RTDocumentry**
- **RTDocumentryPlaylist**
- **rte**: Raidió Teilifís Éireann TV
@ -1144,6 +1143,7 @@
- **Shahid**: [<abbr title="netrc machine"><em>shahid</em></abbr>]
- **ShahidShow**
- **Shared**: shared.sx
- **ShareVideosEmbed**
- **ShemarooMe**
- **ShowRoomLive**
- **simplecast**
@ -1268,6 +1268,7 @@
- **TeleQuebecVideo**
- **TeleTask**
- **Telewebion**
- **Tempo**
- **TennisTV**: [<abbr title="netrc machine"><em>tennistv</em></abbr>]
- **TenPlay**: [<abbr title="netrc machine"><em>10play</em></abbr>]
- **TF1**

View File

@ -144,7 +144,7 @@ from .utils import (
write_json_file,
write_string,
)
from .version import RELEASE_GIT_HEAD, __version__
from .version import RELEASE_GIT_HEAD, VARIANT, __version__
if compat_os_name == 'nt':
import ctypes
@ -1797,6 +1797,8 @@ class YoutubeDL:
})
if self._match_entry(entry_copy, incomplete=True) is not None:
# For compatabilty with youtube-dl. See https://github.com/yt-dlp/yt-dlp/issues/4369
resolved_entries[i] = (playlist_index, NO_DEFAULT)
continue
self.to_screen('[download] Downloading video %s of %s' % (
@ -1817,7 +1819,8 @@ class YoutubeDL:
resolved_entries[i] = (playlist_index, entry_result)
# Update with processed data
ie_result['requested_entries'], ie_result['entries'] = tuple(zip(*resolved_entries)) or ([], [])
ie_result['requested_entries'] = [i for i, e in resolved_entries if e is not NO_DEFAULT]
ie_result['entries'] = [e for _, e in resolved_entries if e is not NO_DEFAULT]
# Write the updated info to json
if _infojson_written is True and self._write_info_json(
@ -3676,6 +3679,8 @@ class YoutubeDL:
write_debug = lambda msg: self._write_string(f'[debug] {msg}\n')
source = detect_variant()
if VARIANT not in (None, 'pip'):
source += '*'
write_debug(join_nonempty(
'yt-dlp version', __version__,
f'[{RELEASE_GIT_HEAD}]' if RELEASE_GIT_HEAD else '',

View File

@ -631,6 +631,7 @@ from .gronkh import (
GronkhVodsIE
)
from .groupon import GrouponIE
from .harpodeon import HarpodeonIE
from .hbo import HBOIE
from .hearthisat import HearThisAtIE
from .heise import HeiseIE
@ -1235,7 +1236,6 @@ from .paramountplus import (
ParamountPlusIE,
ParamountPlusSeriesIE,
)
from .parliamentliveuk import ParliamentLiveUKIE
from .parlview import ParlviewIE
from .patreon import (
PatreonIE,
@ -1406,6 +1406,7 @@ from .rcti import (
RCTIPlusTVIE,
)
from .rds import RDSIE
from .redbee import ParliamentLiveUKIE, RTBFIE
from .redbulltv import (
RedBullTVIE,
RedBullEmbedIE,
@ -1439,7 +1440,6 @@ from .rokfin import (
from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE
from .rottentomatoes import RottenTomatoesIE
from .rozhlas import RozhlasIE
from .rtbf import RTBFIE
from .rte import RteIE, RteRadioIE
from .rtlnl import (
RtlNlIE,

View File

@ -0,0 +1,70 @@
from .common import InfoExtractor
from ..utils import unified_strdate
class HarpodeonIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?harpodeon\.com/(?:video|preview)/\w+/(?P<id>\d+)'
_TESTS = [{
'url': 'https://www.harpodeon.com/video/The_Smoking_Out_of_Bella_Butts/268068288',
'md5': '727371564a6a9ebccef2073535b5b6bd',
'skip': 'Free video could become unavailable',
'info_dict': {
'id': '268068288',
'ext': 'mp4',
'title': 'The Smoking Out of Bella Butts',
'description': 'md5:47e16bdb41fc8a79c83ab83af11c8b77',
'creator': 'Vitagraph Company of America',
'release_date': '19150101'
}
}, {
'url': 'https://www.harpodeon.com/preview/The_Smoking_Out_of_Bella_Butts/268068288',
'md5': '6dfea5412845f690c7331be703f884db',
'info_dict': {
'id': '268068288',
'ext': 'mp4',
'title': 'The Smoking Out of Bella Butts',
'description': 'md5:47e16bdb41fc8a79c83ab83af11c8b77',
'creator': 'Vitagraph Company of America',
'release_date': '19150101'
}
}, {
'url': 'https://www.harpodeon.com/preview/Behind_the_Screen/421838710',
'md5': '7979df9ca04637282cb7d172ab3a9c3b',
'info_dict': {
'id': '421838710',
'ext': 'mp4',
'title': 'Behind the Screen',
'description': 'md5:008972a3dc51fba3965ee517d2ba9155',
'creator': 'Lone Star Corporation',
'release_date': '19160101'
}
}]
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
title, creator, release_year = self._search_regex(
r'''(?x)
<div[^>]+videoInfo[^<]*<h2[^>]*>(?P<title>[^>]+)</h2>
(?:\s*<p[^>]*>\((?P<creator>.+),\s*)?(?P<release_year>\d{4})?''',
webpage, 'title', group=('title', 'creator', 'release_year'),
fatal=False) or (None, None, None)
hp_base = self._html_search_regex(r'hpBase\(\s*["\']([^"\']+)', webpage, 'hp_base')
hp_inject_video, hp_resolution = self._search_regex(
r'''(?x)
hpInjectVideo\([\'\"](?P<hp_inject_video>\w+)[\'\"],
[\'\"](?P<hp_resolution>\d+)[\'\"]''',
webpage, 'hp_inject_video', group=['hp_inject_video', 'hp_resolution'])
return {
'id': video_id,
'title': title,
'url': f'{hp_base}{hp_inject_video}_{hp_resolution}.mp4',
'http_headers': {'Referer': url},
'description': self._html_search_meta('description', webpage, fatal=False),
'creator': creator,
'release_date': unified_strdate(f'{release_year}0101')
}

View File

@ -141,6 +141,10 @@ class MediasetIE(ThePlatformBaseIE):
# iframe twitter (from http://www.wittytv.it/se-prima-mi-fidavo-zero/)
'url': 'https://static3.mediasetplay.mediaset.it/player/index.html?appKey=5ad3966b1de1c4000d5cec48&programGuid=FAFU000000665104&id=665104',
'only_matching': True,
}, {
# embedUrl (from https://www.wittytv.it/amici/est-ce-que-tu-maimes-gabriele-5-dicembre-copia/)
'url': 'https://static3.mediasetplay.mediaset.it/player/v2/index.html?partnerId=wittytv&configId=&programGuid=FD00000000153323&autoplay=true&purl=http://www.wittytv.it/amici/est-ce-que-tu-maimes-gabriele-5-dicembre-copia/',
'only_matching': True,
}, {
'url': 'mediaset:FAFU000000665924',
'only_matching': True,

View File

@ -1,77 +0,0 @@
import json
import uuid
from .common import InfoExtractor
from ..utils import (
unified_timestamp,
try_get,
)
class ParliamentLiveUKIE(InfoExtractor):
IE_NAME = 'parliamentlive.tv'
IE_DESC = 'UK parliament videos'
_VALID_URL = r'(?i)https?://(?:www\.)?parliamentlive\.tv/Event/Index/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
_TESTS = [{
'url': 'http://parliamentlive.tv/Event/Index/c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
'info_dict': {
'id': 'c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
'ext': 'mp4',
'title': 'Home Affairs Committee',
'timestamp': 1395153872,
'upload_date': '20140318',
},
}, {
'url': 'http://parliamentlive.tv/event/index/3f24936f-130f-40bf-9a5d-b3d6479da6a4',
'only_matching': True,
}]
def _real_extract(self, url):
video_id = self._match_id(url)
video_info = self._download_json(f'https://www.parliamentlive.tv/Event/GetShareVideo/{video_id}', video_id)
_DEVICE_ID = str(uuid.uuid4())
auth = 'Bearer ' + self._download_json(
'https://exposure.api.redbee.live/v2/customer/UKParliament/businessunit/ParliamentLive/auth/anonymous',
video_id, headers={
'Origin': 'https://videoplayback.parliamentlive.tv',
'Accept': 'application/json, text/plain, */*',
'Content-Type': 'application/json;charset=utf-8'
}, data=json.dumps({
'deviceId': _DEVICE_ID,
'device': {
'deviceId': _DEVICE_ID,
'width': 653,
'height': 368,
'type': 'WEB',
'name': ' Mozilla Firefox 91'
}
}).encode('utf-8'))['sessionToken']
video_urls = self._download_json(
f'https://exposure.api.redbee.live/v2/customer/UKParliament/businessunit/ParliamentLive/entitlement/{video_id}/play',
video_id, headers={'Authorization': auth, 'Accept': 'application/json, text/plain, */*'})['formats']
formats = []
for format in video_urls:
if not format.get('mediaLocator'):
continue
if format.get('format') == 'DASH':
formats.extend(self._extract_mpd_formats(
format['mediaLocator'], video_id, mpd_id='dash', fatal=False))
elif format.get('format') == 'SMOOTHSTREAMING':
formats.extend(self._extract_ism_formats(
format['mediaLocator'], video_id, ism_id='ism', fatal=False))
elif format.get('format') == 'HLS':
formats.extend(self._extract_m3u8_formats(
format['mediaLocator'], video_id, m3u8_id='hls', fatal=False))
self._sort_formats(formats)
return {
'id': video_id,
'formats': formats,
'title': video_info['event']['title'],
'timestamp': unified_timestamp(try_get(video_info, lambda x: x['event']['publishedStartTime'])),
'thumbnail': video_info.get('thumbnailUrl'),
}

361
yt_dlp/extractor/redbee.py Normal file
View File

@ -0,0 +1,361 @@
import json
import re
import time
import urllib.parse
import uuid
from .common import InfoExtractor
from ..utils import (
ExtractorError,
float_or_none,
int_or_none,
strip_or_none,
traverse_obj,
unified_timestamp,
)
class RedBeeBaseIE(InfoExtractor):
_DEVICE_ID = str(uuid.uuid4())
@property
def _API_URL(self):
"""
Ref: https://apidocs.emp.ebsd.ericsson.net
Subclasses must set _REDBEE_CUSTOMER, _REDBEE_BUSINESS_UNIT
"""
return f'https://exposure.api.redbee.live/v2/customer/{self._REDBEE_CUSTOMER}/businessunit/{self._REDBEE_BUSINESS_UNIT}'
def _get_bearer_token(self, asset_id, jwt=None):
request = {
'deviceId': self._DEVICE_ID,
'device': {
'deviceId': self._DEVICE_ID,
'name': 'Mozilla Firefox 102',
'type': 'WEB',
},
}
if jwt:
request['jwt'] = jwt
return self._download_json(
f'{self._API_URL}/auth/{"gigyaLogin" if jwt else "anonymous"}',
asset_id, data=json.dumps(request).encode('utf-8'), headers={
'Content-Type': 'application/json;charset=utf-8'
})['sessionToken']
def _get_formats_and_subtitles(self, asset_id, **kwargs):
bearer_token = self._get_bearer_token(asset_id, **kwargs)
api_response = self._download_json(
f'{self._API_URL}/entitlement/{asset_id}/play',
asset_id, headers={
'Authorization': f'Bearer {bearer_token}',
'Accept': 'application/json, text/plain, */*'
})
formats, subtitles = [], {}
for format in api_response['formats']:
if not format.get('mediaLocator'):
continue
fmts, subs = [], {}
if format.get('format') == 'DASH':
fmts, subs = self._extract_mpd_formats_and_subtitles(
format['mediaLocator'], asset_id, fatal=False)
elif format.get('format') == 'SMOOTHSTREAMING':
fmts, subs = self._extract_ism_formats_and_subtitles(
format['mediaLocator'], asset_id, fatal=False)
elif format.get('format') == 'HLS':
fmts, subs = self._extract_m3u8_formats_and_subtitles(
format['mediaLocator'], asset_id, fatal=False)
formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles)
return formats, subtitles
class ParliamentLiveUKIE(RedBeeBaseIE):
IE_NAME = 'parliamentlive.tv'
IE_DESC = 'UK parliament videos'
_VALID_URL = r'(?i)https?://(?:www\.)?parliamentlive\.tv/Event/Index/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
_REDBEE_CUSTOMER = 'UKParliament'
_REDBEE_BUSINESS_UNIT = 'ParliamentLive'
_TESTS = [{
'url': 'http://parliamentlive.tv/Event/Index/c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
'info_dict': {
'id': 'c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
'ext': 'mp4',
'title': 'Home Affairs Committee',
'timestamp': 1395153872,
'upload_date': '20140318',
'thumbnail': r're:https?://[^?#]+c1e9d44d-fd6c-4263-b50f-97ed26cc998b[^/]*/thumbnail',
},
}, {
'url': 'http://parliamentlive.tv/event/index/3f24936f-130f-40bf-9a5d-b3d6479da6a4',
'only_matching': True,
}, {
'url': 'https://parliamentlive.tv/Event/Index/27cf25e4-e77b-42a3-93c5-c815cd6d7377',
'info_dict': {
'id': '27cf25e4-e77b-42a3-93c5-c815cd6d7377',
'ext': 'mp4',
'title': 'House of Commons',
'timestamp': 1658392447,
'upload_date': '20220721',
'thumbnail': r're:https?://[^?#]+27cf25e4-e77b-42a3-93c5-c815cd6d7377[^/]*/thumbnail',
},
}]
def _real_extract(self, url):
video_id = self._match_id(url)
formats, subtitles = self._get_formats_and_subtitles(video_id)
self._sort_formats(formats)
video_info = self._download_json(
f'https://www.parliamentlive.tv/Event/GetShareVideo/{video_id}', video_id, fatal=False)
self._sort_formats(formats, ['res', 'proto'])
return {
'id': video_id,
'formats': formats,
'subtitles': subtitles,
'title': traverse_obj(video_info, ('event', 'title')),
'thumbnail': traverse_obj(video_info, 'thumbnailUrl'),
'timestamp': traverse_obj(
video_info, ('event', 'publishedStartTime'), expected_type=unified_timestamp),
}
class RTBFIE(RedBeeBaseIE):
_VALID_URL = r'''(?x)
https?://(?:www\.)?rtbf\.be/
(?:
video/[^?]+\?.*\bid=|
ouftivi/(?:[^/]+/)*[^?]+\?.*\bvideoId=|
auvio/[^/]+\?.*\b(?P<live>l)?id=
)(?P<id>\d+)'''
_NETRC_MACHINE = 'rtbf'
_REDBEE_CUSTOMER = 'RTBF'
_REDBEE_BUSINESS_UNIT = 'Auvio'
_TESTS = [{
'url': 'https://www.rtbf.be/video/detail_les-diables-au-coeur-episode-2?id=1921274',
'md5': '8c876a1cceeb6cf31b476461ade72384',
'info_dict': {
'id': '1921274',
'ext': 'mp4',
'title': 'Les Diables au coeur (épisode 2)',
'description': '(du 25/04/2014)',
'duration': 3099.54,
'upload_date': '20140425',
'timestamp': 1398456300,
},
'skip': 'No longer available',
}, {
# geo restricted
'url': 'http://www.rtbf.be/ouftivi/heros/detail_scooby-doo-mysteres-associes?id=1097&videoId=2057442',
'only_matching': True,
}, {
'url': 'http://www.rtbf.be/ouftivi/niouzz?videoId=2055858',
'only_matching': True,
}, {
'url': 'http://www.rtbf.be/auvio/detail_jeudi-en-prime-siegfried-bracke?id=2102996',
'only_matching': True,
}, {
# Live
'url': 'https://www.rtbf.be/auvio/direct_pure-fm?lid=134775',
'only_matching': True,
}, {
# Audio
'url': 'https://www.rtbf.be/auvio/detail_cinq-heures-cinema?id=2360811',
'only_matching': True,
}, {
# With Subtitle
'url': 'https://www.rtbf.be/auvio/detail_les-carnets-du-bourlingueur?id=2361588',
'only_matching': True,
}, {
'url': 'https://www.rtbf.be/auvio/detail_investigation?id=2921926',
'md5': 'd5d11bb62169fef38d7ce7ac531e034f',
'info_dict': {
'id': '2921926',
'ext': 'mp4',
'title': 'Le handicap un confinement perpétuel - Maladie de Lyme',
'description': 'md5:dcbd5dcf6015488c9069b057c15ccc52',
'duration': 5258.8,
'upload_date': '20220727',
'timestamp': 1658934000,
'series': '#Investigation',
'thumbnail': r're:^https?://[^?&]+\.jpg$',
},
}, {
'url': 'https://www.rtbf.be/auvio/detail_la-belgique-criminelle?id=2920492',
'md5': '054f9f143bc79c89647c35e5a7d35fa8',
'info_dict': {
'id': '2920492',
'ext': 'mp4',
'title': '04 - Le crime de la rue Royale',
'description': 'md5:0c3da1efab286df83f2ab3f8f96bd7a6',
'duration': 1574.6,
'upload_date': '20220723',
'timestamp': 1658596887,
'series': 'La Belgique criminelle - TV',
'thumbnail': r're:^https?://[^?&]+\.jpg$',
},
}]
_IMAGE_HOST = 'http://ds1.ds.static.rtbf.be'
_PROVIDERS = {
'YOUTUBE': 'Youtube',
'DAILYMOTION': 'Dailymotion',
'VIMEO': 'Vimeo',
}
_QUALITIES = [
('mobile', 'SD'),
('web', 'MD'),
('high', 'HD'),
]
_LOGIN_URL = 'https://login.rtbf.be/accounts.login'
_GIGYA_API_KEY = '3_kWKuPgcdAybqnqxq_MvHVk0-6PN8Zk8pIIkJM_yXOu-qLPDDsGOtIDFfpGivtbeO'
_LOGIN_COOKIE_ID = f'glt_{_GIGYA_API_KEY}'
def _perform_login(self, username, password):
if self._get_cookies(self._LOGIN_URL).get(self._LOGIN_COOKIE_ID):
return
self._set_cookie('.rtbf.be', 'gmid', 'gmid.ver4', secure=True, expire_time=time.time() + 3600)
login_response = self._download_json(
self._LOGIN_URL, None, data=urllib.parse.urlencode({
'loginID': username,
'password': password,
'APIKey': self._GIGYA_API_KEY,
'targetEnv': 'jssdk',
'sessionExpiration': '-2',
}).encode('utf-8'), headers={
'Content-Type': 'application/x-www-form-urlencoded',
})
if login_response['statusCode'] != 200:
raise ExtractorError('Login failed. Server message: %s' % login_response['errorMessage'], expected=True)
self._set_cookie('.rtbf.be', self._LOGIN_COOKIE_ID, login_response['sessionInfo']['login_token'],
secure=True, expire_time=time.time() + 3600)
def _get_formats_and_subtitles(self, url, media_id):
login_token = self._get_cookies(url).get(self._LOGIN_COOKIE_ID)
if not login_token:
self.raise_login_required()
session_jwt = self._download_json(
'https://login.rtbf.be/accounts.getJWT', media_id, query={
'login_token': login_token.value,
'APIKey': self._GIGYA_API_KEY,
'sdk': 'js_latest',
'authMode': 'cookie',
'pageURL': url,
'sdkBuild': '13273',
'format': 'json',
})['id_token']
return super()._get_formats_and_subtitles(media_id, jwt=session_jwt)
def _real_extract(self, url):
live, media_id = self._match_valid_url(url).groups()
embed_page = self._download_webpage(
'https://www.rtbf.be/auvio/embed/' + ('direct' if live else 'media'),
media_id, query={'id': media_id})
data = self._parse_json(self._html_search_regex(
r'data-media="([^"]+)"', embed_page, 'media data'), media_id)
error = data.get('error')
if error:
raise ExtractorError('%s said: %s' % (self.IE_NAME, error), expected=True)
provider = data.get('provider')
if provider in self._PROVIDERS:
return self.url_result(data['url'], self._PROVIDERS[provider])
title = data['subtitle']
is_live = data.get('isLive')
height_re = r'-(\d+)p\.'
formats = []
m3u8_url = data.get('urlHlsAes128') or data.get('urlHls')
if m3u8_url:
formats.extend(self._extract_m3u8_formats(
m3u8_url, media_id, 'mp4', m3u8_id='hls', fatal=False))
fix_url = lambda x: x.replace('//rtbf-vod.', '//rtbf.') if '/geo/drm/' in x else x
http_url = data.get('url')
if formats and http_url and re.search(height_re, http_url):
http_url = fix_url(http_url)
for m3u8_f in formats[:]:
height = m3u8_f.get('height')
if not height:
continue
f = m3u8_f.copy()
del f['protocol']
f.update({
'format_id': m3u8_f['format_id'].replace('hls-', 'http-'),
'url': re.sub(height_re, '-%dp.' % height, http_url),
})
formats.append(f)
else:
sources = data.get('sources') or {}
for key, format_id in self._QUALITIES:
format_url = sources.get(key)
if not format_url:
continue
height = int_or_none(self._search_regex(
height_re, format_url, 'height', default=None))
formats.append({
'format_id': format_id,
'url': fix_url(format_url),
'height': height,
})
mpd_url = data.get('urlDash')
if mpd_url and (self.get_param('allow_unplayable_formats') or not data.get('drm')):
formats.extend(self._extract_mpd_formats(
mpd_url, media_id, mpd_id='dash', fatal=False))
audio_url = data.get('urlAudio')
if audio_url:
formats.append({
'format_id': 'audio',
'url': audio_url,
'vcodec': 'none',
})
subtitles = {}
for track in (data.get('tracks') or {}).values():
sub_url = track.get('url')
if not sub_url:
continue
subtitles.setdefault(track.get('lang') or 'fr', []).append({
'url': sub_url,
})
if not formats:
fmts, subs = self._get_formats_and_subtitles(url, media_id)
formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles)
self._sort_formats(formats, ['res', 'proto'])
return {
'id': media_id,
'formats': formats,
'title': title,
'description': strip_or_none(data.get('description')),
'thumbnail': data.get('thumbnail'),
'duration': float_or_none(data.get('realDuration')),
'timestamp': int_or_none(data.get('liveFrom')),
'series': data.get('programLabel'),
'subtitles': subtitles,
'is_live': is_live,
}

View File

@ -1,156 +0,0 @@
import re
from .common import InfoExtractor
from ..utils import (
ExtractorError,
float_or_none,
int_or_none,
strip_or_none,
)
class RTBFIE(InfoExtractor):
_VALID_URL = r'''(?x)
https?://(?:www\.)?rtbf\.be/
(?:
video/[^?]+\?.*\bid=|
ouftivi/(?:[^/]+/)*[^?]+\?.*\bvideoId=|
auvio/[^/]+\?.*\b(?P<live>l)?id=
)(?P<id>\d+)'''
_TESTS = [{
'url': 'https://www.rtbf.be/video/detail_les-diables-au-coeur-episode-2?id=1921274',
'md5': '8c876a1cceeb6cf31b476461ade72384',
'info_dict': {
'id': '1921274',
'ext': 'mp4',
'title': 'Les Diables au coeur (épisode 2)',
'description': '(du 25/04/2014)',
'duration': 3099.54,
'upload_date': '20140425',
'timestamp': 1398456300,
}
}, {
# geo restricted
'url': 'http://www.rtbf.be/ouftivi/heros/detail_scooby-doo-mysteres-associes?id=1097&videoId=2057442',
'only_matching': True,
}, {
'url': 'http://www.rtbf.be/ouftivi/niouzz?videoId=2055858',
'only_matching': True,
}, {
'url': 'http://www.rtbf.be/auvio/detail_jeudi-en-prime-siegfried-bracke?id=2102996',
'only_matching': True,
}, {
# Live
'url': 'https://www.rtbf.be/auvio/direct_pure-fm?lid=134775',
'only_matching': True,
}, {
# Audio
'url': 'https://www.rtbf.be/auvio/detail_cinq-heures-cinema?id=2360811',
'only_matching': True,
}, {
# With Subtitle
'url': 'https://www.rtbf.be/auvio/detail_les-carnets-du-bourlingueur?id=2361588',
'only_matching': True,
}]
_IMAGE_HOST = 'http://ds1.ds.static.rtbf.be'
_PROVIDERS = {
'YOUTUBE': 'Youtube',
'DAILYMOTION': 'Dailymotion',
'VIMEO': 'Vimeo',
}
_QUALITIES = [
('mobile', 'SD'),
('web', 'MD'),
('high', 'HD'),
]
def _real_extract(self, url):
live, media_id = self._match_valid_url(url).groups()
embed_page = self._download_webpage(
'https://www.rtbf.be/auvio/embed/' + ('direct' if live else 'media'),
media_id, query={'id': media_id})
data = self._parse_json(self._html_search_regex(
r'data-media="([^"]+)"', embed_page, 'media data'), media_id)
error = data.get('error')
if error:
raise ExtractorError('%s said: %s' % (self.IE_NAME, error), expected=True)
provider = data.get('provider')
if provider in self._PROVIDERS:
return self.url_result(data['url'], self._PROVIDERS[provider])
title = data['title']
is_live = data.get('isLive')
height_re = r'-(\d+)p\.'
formats = []
m3u8_url = data.get('urlHlsAes128') or data.get('urlHls')
if m3u8_url:
formats.extend(self._extract_m3u8_formats(
m3u8_url, media_id, 'mp4', m3u8_id='hls', fatal=False))
fix_url = lambda x: x.replace('//rtbf-vod.', '//rtbf.') if '/geo/drm/' in x else x
http_url = data.get('url')
if formats and http_url and re.search(height_re, http_url):
http_url = fix_url(http_url)
for m3u8_f in formats[:]:
height = m3u8_f.get('height')
if not height:
continue
f = m3u8_f.copy()
del f['protocol']
f.update({
'format_id': m3u8_f['format_id'].replace('hls-', 'http-'),
'url': re.sub(height_re, '-%dp.' % height, http_url),
})
formats.append(f)
else:
sources = data.get('sources') or {}
for key, format_id in self._QUALITIES:
format_url = sources.get(key)
if not format_url:
continue
height = int_or_none(self._search_regex(
height_re, format_url, 'height', default=None))
formats.append({
'format_id': format_id,
'url': fix_url(format_url),
'height': height,
})
mpd_url = data.get('urlDash')
if mpd_url and (self.get_param('allow_unplayable_formats') or not data.get('drm')):
formats.extend(self._extract_mpd_formats(
mpd_url, media_id, mpd_id='dash', fatal=False))
audio_url = data.get('urlAudio')
if audio_url:
formats.append({
'format_id': 'audio',
'url': audio_url,
'vcodec': 'none',
})
self._sort_formats(formats)
subtitles = {}
for track in (data.get('tracks') or {}).values():
sub_url = track.get('url')
if not sub_url:
continue
subtitles.setdefault(track.get('lang') or 'fr', []).append({
'url': sub_url,
})
return {
'id': media_id,
'formats': formats,
'title': title,
'description': strip_or_none(data.get('description')),
'thumbnail': data.get('thumbnail'),
'duration': float_or_none(data.get('realDuration')),
'timestamp': int_or_none(data.get('liveFrom')),
'series': data.get('programLabel'),
'subtitles': subtitles,
'is_live': is_live,
}

View File

@ -61,6 +61,18 @@ class VVVVIDIE(InfoExtractor):
'params': {
'skip_download': True,
},
}, {
# video_type == 'video/dash'
'url': 'https://www.vvvvid.it/show/683/made-in-abyss/1542/693786/nanachi',
'info_dict': {
'id': '693786',
'ext': 'mp4',
'title': 'Nanachi',
},
'params': {
'skip_download': True,
'format': 'mp4',
},
}, {
'url': 'https://www.vvvvid.it/show/434/perche-dovrei-guardarlo-di-dario-moccia/437/489048',
'only_matching': True
@ -202,6 +214,9 @@ class VVVVIDIE(InfoExtractor):
})
is_youtube = True
break
elif video_type == 'video/dash':
formats.extend(self._extract_m3u8_formats(
embed_code, video_id, 'mp4', m3u8_id='hls', fatal=False))
else:
formats.extend(self._extract_wowza_formats(
'http://sb.top-ix.org/videomg/_definst_/mp4:%s/playlist.m3u8' % embed_code, video_id))

View File

@ -20,12 +20,13 @@ from .postprocessor import (
SponsorBlockPP,
)
from .postprocessor.modify_chapters import DEFAULT_SPONSORBLOCK_CHAPTER_TITLE
from .update import detect_variant
from .update import detect_variant, is_non_updateable
from .utils import (
OUTTMPL_TYPES,
POSTPROCESS_WHEN,
Config,
expand_path,
format_field,
get_executable_path,
join_nonempty,
remove_end,
@ -333,11 +334,13 @@ def create_parser():
general.add_option(
'-U', '--update',
action='store_true', dest='update_self',
help='Update this program to latest version')
help=format_field(
is_non_updateable(), None, 'Check if updates are available. %s',
default='Update this program to the latest version'))
general.add_option(
'--no-update',
action='store_false', dest='update_self',
help='Do not update (default)')
help='Do not check for updates (default)')
general.add_option(
'-i', '--ignore-errors',
action='store_true', dest='ignoreerrors',

View File

@ -109,7 +109,8 @@ class FFmpegPostProcessor(PostProcessor):
return {p: p for p in programs}
if not os.path.exists(location):
self.report_warning(f'ffmpeg-location {location} does not exist! Continuing without ffmpeg')
self.report_warning(
f'ffmpeg-location {location} does not exist! Continuing without ffmpeg', only_once=True)
return {}
elif os.path.isdir(location):
dirname, basename = location, None
@ -171,9 +172,9 @@ class FFmpegPostProcessor(PostProcessor):
return self.probe_basename
def _get_version(self, kind):
executables = (kind, self._ffmpeg_to_avconv[kind])
executables = (kind, )
if not self._prefer_ffmpeg:
executables = reversed(executables)
executables = (kind, self._ffmpeg_to_avconv[kind])
basename, version, features = next(filter(
lambda x: x[1], ((p, *self._get_ffmpeg_version(p)) for p in executables)), (None, None, {}))
if kind == 'ffmpeg':

View File

@ -18,7 +18,7 @@ from .utils import (
traverse_obj,
version_tuple,
)
from .version import __version__
from .version import UPDATE_HINT, VARIANT, __version__
REPOSITORY = 'yt-dlp/yt-dlp'
API_URL = f'https://api.github.com/repos/{REPOSITORY}/releases'
@ -47,7 +47,7 @@ def _get_variant_and_executable_path():
def detect_variant():
return _get_variant_and_executable_path()[0]
return VARIANT or _get_variant_and_executable_path()[0]
_FILE_SUFFIXES = {
@ -64,13 +64,16 @@ _NON_UPDATEABLE_REASONS = {
**{variant: f'Auto-update is not supported for unpackaged {name} executable; Re-download the latest release'
for variant, name in {'win32_dir': 'Windows', 'darwin_dir': 'MacOS', 'linux_dir': 'Linux'}.items()},
'source': 'You cannot update when running from source code; Use git to pull the latest changes',
'unknown': 'It looks like you installed yt-dlp with a package manager, pip or setup.py; Use that to update',
'other': 'It looks like you are using an unofficial build of yt-dlp; Build the executable again',
'unknown': 'You installed yt-dlp with a package manager or setup.py; Use that to update',
'other': 'You are using an unofficial build of yt-dlp; Build the executable again',
}
def is_non_updateable():
return _NON_UPDATEABLE_REASONS.get(detect_variant(), _NON_UPDATEABLE_REASONS['other'])
if UPDATE_HINT:
return UPDATE_HINT
return _NON_UPDATEABLE_REASONS.get(
detect_variant(), _NON_UPDATEABLE_REASONS['unknown' if VARIANT else 'other'])
def _sha256_file(path):

View File

@ -1,5 +1,9 @@
# Autogenerated by devscripts/update-version.py
__version__ = '2022.07.18'
__version__ = '2022.08.08'
RELEASE_GIT_HEAD = '135f05ef6'
RELEASE_GIT_HEAD = '3157158f7'
VARIANT = None
UPDATE_HINT = None