1
0
mirror of https://git.sr.ht/~cadence/NewLeaf synced 2024-11-14 12:07:30 +00:00
Commit Graph

99 Commits

Author SHA1 Message Date
Cadence Ember
1ea86101fd
Support premiere videos on channel 2021-07-01 23:42:53 +12:00
7d3b79b1cd
Change cookies to skip EU cookie consent page
See https://github.com/benbusby/whoogle-search/issues/311 for some
context.
We're now implementing
a726009987/youtube_dl/extractor/youtube.py (L263-L264)
2021-05-15 22:29:44 +12:00
Cadence Ember
18f5ef4c62
Quote json keys correctly 2021-05-14 18:46:46 +12:00
f0c9708d99
Fix search extractor ad section filtering
The ads sections had a carouselAdRenderer property, now they have a
promotedSparklesTextSearchRenderer property instead. As this may
change again in the future, we should just get all items as we
discriminate/filter them as videos afterwards with the videoRenderer
property.
2021-05-14 18:46:08 +12:00
Cadence Ember
57b0a88a2e
Detect channels that do not exist
If error alerts exist, they will be logged. But it is reasonable to
assume that not all errors will be fatal, so we don't necessarily quit
parsing if we find one.

This also normalises the text error of the /latest response for a
missing channel, without changing its identifier.
2021-05-02 01:20:53 +12:00
Cadence Ember
50a4b7af45
Add a handler for ytdl search request message 2021-04-28 00:55:29 +12:00
Cadence Ember
e3595a455e
Remove "unknown download error" prefix
The reason for the error is known and is returned.
2021-04-28 00:08:07 +12:00
7737ea3ba5
Fix #26 append detailed error message returned by yt-dlp in video extractor
Fixes https://todo.sr.ht/~cadence/tube/26
2021-04-18 15:23:28 +12:00
Cadence Ember
5125bb9461
Don't fail if captions field is missing 2021-04-10 12:50:18 +12:00
be8a2dad5f
Remove extraneous " align:start position:0%" on auto-generated captions 2021-04-10 00:44:10 +12:00
Cadence Ember
1d52fca3a0
Support auto-generated captions
The caption extraction is now entirely in our own hands.
2021-04-05 01:23:54 +12:00
bopol
aaf7d65b32
change CONSENT cookie value
FX is accept all tracking, PENDING should imply no tracking
2021-04-04 14:45:54 +12:00
5f47e1a71b
Fix extracting with cookie consent page in EU
Fix #27 use maintained yt-dlp lib instead of youtube-dlc

Because of the following changes in YT, we have to switch to a
maintained library https://github.com/ytdl-org/youtube-dl/issues/28604
While yt-dlp is not fixed today, youtube-dl is fixed in master and as
yt-dlp is quick to merge upstream changes back to their repo, we can
hope the issue will also be fixed there timely.

For requests sent by us directly, we include the cookies.

Ref https://github.com/ytdl-org/youtube-dl/issues/28604
2021-04-03 15:09:58 +13:00
Cadence Ember
fe04a4dbd6
Fix temporary file removal again 2021-04-03 14:57:51 +13:00
Cadence Ember
20fa40dd3d
Add front page 2021-04-03 14:42:30 +13:00
Cadence Ember
ccd3513c46
Add robots.txt 2021-04-03 14:00:05 +13:00
Cadence Ember
0f877b06bc
Fix temporary file removal 2021-03-28 23:58:54 +13:00
ABeltramo
7ed3248104
Docker updates
- Removed default config file from Dockerfile
- added .git folder to dockerignore
2021-03-23 23:31:49 +13:00
Olivier
70c95f4b63
Allow configuring the bind host address and port. 2021-03-12 00:18:07 +13:00
Cadence Ember
6dfceea6a0
Move endpoint status to documentation 2021-02-27 15:08:39 +13:00
Cadence Ember
e8e68150de
Add link to documentation repo 2021-02-27 14:57:03 +13:00
Cadence Ember
abd6c8df2f
Rename to NewLeaf 2021-02-27 13:09:31 +13:00
Cadence Ember
c8b4699922
Support topic channels with no videos tab
https://second.cadence.moe/api/v1/channels/UCr-iHMODX8D4a6MVQ_RtdQg
2021-02-19 01:17:54 +13:00
80b41c7725
Fix broken channel videos extraction failing with KeyError: 'gridVideoRenderer' 2021-02-19 00:59:55 +13:00
Cadence Ember
0ae2a8a126
Add navigation 2021-02-12 00:31:07 +13:00
Cadence Ember
268457394f
Split out file cleanup code 2021-01-26 01:05:40 +13:00
Cadence Ember
b454ebd6e5
Remove origin from caption URL 2021-01-20 23:00:26 +13:00
Cadence Ember
6c2b67fb6f
Use lowercase only for headers 2021-01-20 22:59:54 +13:00
Cadence Ember
c837828a22
Captions: Error checking 2021-01-20 17:37:39 +13:00
Cadence Ember
8e69928756
Captions: Python code cleanup and optimisation 2021-01-20 17:36:50 +13:00
bopol
6709aa30c2
Implement captions
Automatic subtitles are not supported, because youtube_dlc does not
provide them.
2021-01-20 17:36:49 +13:00
Cadence Ember
985f0c1c32
Fix ggpht response call order 2021-01-20 17:36:30 +13:00
Cadence Ember
39425f994a
Fix subscriber count extraction 2021-01-17 14:56:17 +13:00
Cadence Ember
f1ddf66f50
Touch up Bopol's patch 2021-01-17 14:55:57 +13:00
bopol
6cc921c2dc
fix channel extraction when header is not available 2021-01-17 14:30:34 +13:00
Cadence Ember
adb5dc93d5
Add ggpht proxy (channel-related images) 2021-01-15 01:07:05 +13:00
Cadence Ember
8a0fb5cf21
Support originals with no view count 2021-01-01 17:28:38 +13:00
Cadence Ember
20b133dbb6
Fix manifest 2020-12-18 19:54:06 +13:00
Cadence Ember
e95d814709
Fix channel extraction when subscribers not available 2020-12-09 16:53:22 +13:00
Cadence Ember
10f8009101
Gracefully fail on feed fetch for invalid channel 2020-12-06 15:39:28 +13:00
Cadence Ember
554cd8cc3a
Improve ytInitialData extraction 2020-12-03 17:00:06 +13:00
Cadence Ember
ba88c53857
Fix search; use youtube-dlc 2020-12-03 16:32:31 +13:00
Cadence Ember
be1b1bc3a9
Add requests to requirements 2020-11-20 15:00:32 +13:00
Cadence Ember
87c7730fbc
Fetch pages using en locale 2020-10-25 18:02:05 +13:00
Cadence Ember
861f441f9f
Fix search 2020-10-24 00:36:20 +13:00
Cadence Ember
ad1f8df7cb
Correctly handle videos with 0 views 2020-10-15 00:32:06 +13:00
Cadence Ember
0b9874a4f4
Fix channels having videos 2020-10-04 18:38:41 +13:00
Cadence Ember
e1bcc306b3
Fix for if channel has no videos 2020-10-03 01:17:23 +13:00
Cadence Ember
097b6bf584
Add Access-Control-Allow-Origin header 2020-10-02 23:40:39 +13:00
Cadence Ember
c506f65c71
Use empty string instead of null if no description 2020-09-24 01:06:47 +12:00