From f091b366be3fc68245f29ec057fb34ae268bc65b Mon Sep 17 00:00:00 2001
From: BBaoVanC <bbaovanc@bbaovanc.com>
Date: Mon, 8 Sep 2025 19:27:12 -0500
Subject: [PATCH] Attempt to fix robots.txt blocking /blog/*.md for all bots

According to Semrush, 741b3bc01be35794ca3047f1c45bbafd0965b4dd added a
block for /blog/*.md for User-Agent: *, even though everything I can
read says that it shouldn't apply since the blocks are separated.

I tried reordering the rules and explicitly adding an `Allow: /` rule,
both to match what's shown on Google Search documentation[1].

[1]: https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt,
and
---
 layouts/robots.txt | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/layouts/robots.txt b/layouts/robots.txt
index 61a2af3..7c24138 100644
--- a/layouts/robots.txt
+++ b/layouts/robots.txt
@@ -1,6 +1,8 @@
-User-Agent: *
-Sitemap: {{ "sitemap.xml" | absLangURL }}
-
 {{/* It complains about the raw markdown output having no title tag */ -}}
 User-Agent: Bingbot
 Disallow: /blog/*.md$
+
+User-Agent: *
+Allow: /
+
+Sitemap: {{ "sitemap.xml" | absLangURL }}