Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YouTube: Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "BaW_jenozKc") #9554

Open
9 of 11 tasks
rredford6 opened this issue Mar 28, 2024 · 20 comments · May be fixed by #9553
Labels
site-bug Issue with a specific website

Comments

@rredford6
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

United States

Provide a description that is worded well enough to be understood

This is the crux of the issue:

WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "BaW_jenozKc")

This was previously solved in #4593. It was first rediscovered on #9546 (comment)

This can be reproduced on multiple YouTube videos. This warning is not seen on every download.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version master@2024.03.22.222728 from yt-dlp/yt-dlp-master-builds [e5d4f1110] (zip)
[debug] Python 3.8.10 (CPython x86_64 64bit) - Linux-5.4.0-163-generic-x86_64-with-glibc2.29 (OpenSSL 1.1.1i  8 Dec 2020, glibc 2.31)
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, requests-2.27.1, sqlite3-3.31.1, urllib3-1.26.9, websockets-10.4
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1807 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-master-builds/releases/latest
Latest version: master@2024.03.22.222728 from yt-dlp/yt-dlp-master-builds
yt-dlp is up to date (master@2024.03.22.222728 from yt-dlp/yt-dlp-master-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading ios player API JSON
[youtube] BaW_jenozKc: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "BaW_jenozKc")
[debug] Loading youtube-nsig.3b96d06c from cache
[debug] [youtube] Decrypted nsig -Dr9JVw16PnpN3kB => IO0gtddH8cbvwQ
[debug] Loading youtube-nsig.3b96d06c from cache
[debug] [youtube] Decrypted nsig LbPw5yQFOwpSkCGf => Noiw4nN5a0OI9g
[youtube] BaW_jenozKc: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] BaW_jenozKc: Downloading 1 format(s): 248+251
[debug] Invoking http downloader on "https://"
[download] Destination: youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].f248.webm
[download] 100% of  965.31KiB in 00:00:00 at 3.34MiB/s
[debug] Invoking http downloader on "https://"
[download] Destination: youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].f251.webm
[download] 100% of  138.96KiB in 00:00:00 at 861.80KiB/s
[Merger] Merging formats into "youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:youtube-dl test video "'"'"'⧸⧹ä↭𝕐 [BaW_jenozKc].f248.webm' -i 'file:youtube-dl test video "'"'"'⧸⧹ä↭𝕐 [BaW_jenozKc].f251.webm' -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart 'file:youtube-dl test video "'"'"'⧸⧹ä↭𝕐 [BaW_jenozKc].temp.webm'
Deleting original file youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].f251.webm (pass -k to keep)
Deleting original file youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].f248.webm (pass -k to keep)
@rredford6 rredford6 added site-bug Issue with a specific website triage Untriaged issue labels Mar 28, 2024
@bashonly
Copy link
Member

will be "fixed" by #9553 ( 😢 ). yt doesn't want us to have nice things

@bashonly bashonly removed the triage Untriaged issue label Mar 28, 2024
@coletdjnz coletdjnz linked a pull request Mar 28, 2024 that will close this issue
9 tasks
@absidue
Copy link

absidue commented Mar 29, 2024

will be "fixed" by #9553 ( 😢 ). yt doesn't want us to have nice things

I'm pretty sure YouTube adding that has nothing to do with yt-dlp and is entirely because of ReVanced and yt-dlp is just collateral damage. As ReVanced modifies the official YouTube app (unlike NewPipe which is a separate app written from scratch), it's no longer enough for YouTube to just check if you are using the Android app, they now need to check if you are using a modded version of it.

@pukkandan
Copy link
Member

To maintainers: Don't close the issue till release. From experience, keeping issue open seems to reduce number of duplicates

@kiwib1rd1337
Copy link

kiwib1rd1337 commented Mar 30, 2024

I can confirm this happening in New Zealand also (no VPN, native IPv4/IPv6):

[youtube] Extracting URL: https://youtu.be/luXZz9AqBbQ
[youtube] luXZz9AqBbQ: Downloading webpage
[youtube] luXZz9AqBbQ: Downloading ios player API JSON
[youtube] luXZz9AqBbQ: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "luXZz9AqBbQ")
[youtube] luXZz9AqBbQ: Downloading m3u8 information
[info] luXZz9AqBbQ: Downloading 1 format(s): 18
[download] Destination: r⧸AskOuija where they talk one LETTER at a time [luXZz9AqBbQ].mp4
[download] 100% of 10.50MiB in 00:00:02 at 4.20MiB/s
[Metadata] Adding metadata to "r⧸AskOuija where they talk one LETTER at a time [luXZz9AqBbQ].mp4"
`
Downloading the "aQvGIIdgFDM" video gives a file titled "Video Not Available [aQvGIIdgFDM].mp4", length of 5 minutes, black background, circled exclamation point on left, "The following content is not available in this app. Watch this content on the latest version of YouTube" on the right.

Whatever kind of black magic is YT trying to pull on us now, and why it is happening to Android client APIs specifically is currently a mystery to me. Why is this not happening on the iOS one as well?

UPDATE: Tested on the 'https://youtu.be/BaW_jenozKc' video as well, failed to replicate on that video:

[youtube] Extracting URL: https://youtu.be/BaW_jenozKc
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading ios player API JSON
[youtube] BaW_jenozKc: Downloading android player API JSON
[youtube] BaW_jenozKc: Downloading m3u8 information
[info] BaW_jenozKc: Downloading 1 format(s): 18
[download] Destination: youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].mp4
[download] 100% of 184.94KiB in 00:00:02 at 73.93KiB/s
[Metadata] Adding metadata to "youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].mp4"

Possible A/B testing perhaps? Knowing YT's shady testing behaviour, it's certainly could be...

@xpalidotious
Copy link

will be "fixed" by #9553

For an end-user is there any significant difference between that fix and simply using "--extractor-args youtube:player_client=ios,web" in your config?

The docs mention that

By default, ios,android,web is used, but tv_embedded and creator variants are added as required for age-gated videos.

and I'm not sure if explicitly specifying player clients overrides that behavior (and are those variants ever needed anyway if you're using cookies for a logged in youtube account?)

From experience, keeping issue open seems to reduce number of duplicates

That's probably because people like me tend to come here only when they have an issue themselves and don't realize that closed issues are hidden by default, and so think no one has posted about their issue yet.

@pukkandan
Copy link
Member

pukkandan commented Mar 30, 2024

From experience, keeping issue open seems to reduce number of duplicates

That's probably because people like me tend to come here only when they have an issue themselves and don't realize that closed issues are hidden by default, and so think no one has posted about their issue yet.

I know. We specifically ask to search closed issues, but many don't. It's what it is... Not trying to blame anyone. I was just passing the wisdom onto other maintainers since they are the ones handling most issues nowadays

  • I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates

For an end-user is there any significant difference between that fix and simply using "--extractor-args youtube:player_client=ios,web" in your config?

By default, ios,android,web is used, but tv_embedded and creator variants are added as required for age-gated videos.

and I'm not sure if explicitly specifying player clients overrides that behavior

No. The age-gate clients are still added when required. You could also simply ignore the warning.

(and are those variants ever needed anyway if you're using cookies for a logged in youtube account?)

It is not. If your account can directly access the video, we don't need any bypass


Whatever kind of black magic is YT trying to pull on us now, and why it is happening to Android client APIs specifically is currently a mystery to me.

Youtube is using Safetynet's AndroidGuard to validate that the requests are coming from their app; possibly in an effort to block revanced, though they have already implemented a bypass.


Possible A/B testing perhaps?

Yes, it happens randomly, though expect it to be completely blocked in few days

@pukkandan
Copy link
Member

Fixed, but leaving open till release

@pukkandan pukkandan reopened this Mar 31, 2024
@SantiiRepair
Copy link

This is happening in Venezuela also

@gamer191
Copy link
Collaborator

gamer191 commented Apr 6, 2024

@SantiiRepair try running yt-dlp --update-to master, and if it still keeps happening, add --verbose to your command and send the output

@Javierchik
Copy link

@SantiiRepair try running yt-dlp --update-to master, and if it still keeps happening, add --verbose to your command and send the output

Thanks for response, but I got this:

Current version: stable@2024.03.10 from yt-dlp/yt-dlp
Latest version: master@2024.04.08.232116 from yt-dlp/yt-dlp-master-builds

Looks like GitHub master branch needs to be updated?
Thx

@pukkandan
Copy link
Member

That is not a complete verbose log

@pukkandan
Copy link
Member

Closing since release is out

aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this issue Apr 21, 2024
@gamer191

This comment was marked as outdated.

@bashonly
Copy link
Member

Reopening this, since it's happening again. Note that as of today, it happens very infrequently. Presumably, this is due to some sort of A/B testing and the problem will become more prevalent eventually. It's likely that the new android player params' end-of-life date is near. This issue will be "fixed" by #9553

@kolbdog323

This comment was marked as duplicate.

@kclauhk
Copy link
Contributor

kclauhk commented Apr 25, 2024

removing the android params seems work

@bashonly
Copy link
Member

@kclauhk the stream URL expires after 1 minute resulting in a 403 error during download if the params are removed

@kclauhk
Copy link
Contributor

kclauhk commented Apr 25, 2024

ok

how is this "{1: {1: 1}}" converted to "CgIIAQ=="?

@bashonly
Copy link
Member

how is this "{1: {1: 1}}" converted to "CgIIAQ=="?

It's protobuf. If you want to experiment with the params you could use https://github.com/Grub4K/qpb

@randombyte-developer
Copy link

For a temporary fix you can install the fix from this PR #9553:

python3 -m pip install -U pip hatchling wheel
python3 -m pip install --force-reinstall "yt-dlp[default] @ https://github.com/coletdjnz/yt-dlp-dev/archive/fix/youtube/remove-android-as-default.tar.gz"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging a pull request may close this issue.