Lomanic/bibliogram
1
0
Fork 0
mirror of https://git.sr.ht/~cadence/bibliogram synced 2025-11-19 20:56:31 +00:00
Code Issues Releases Wiki Activity

Add /tv/ and /reel/ to robots.txt

Browse source 
Prevent search engine crawling of instances, just like how /p/ is
already in robots.txt.
This commit is contained in:
Cadence Ember 2022-05-18 00:13:21 +12:00
parent 50f509742e
commit 49445ffb73
No known key found for this signature in database
GPG key ID: BC1C2C61CF521B17
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
src/site/html/robots.txt
View file

@ -1,5 +1,7 @@
User-Agent: * User-Agent: *
Disallow: /u/ Disallow: /u/
Disallow: /p/ Disallow: /p/
Disallow: /tv/
Disallow: /reel/
Disallow: /fragment/ Disallow: /fragment/
Disallow: /api/ Disallow: /api/