More blog fixes?
A while back I fixed some issues that Google reckoned were hurting indexing, and sure enough I’m now back up over 1,000 pages indexed. Hurrah! This really only matters if I want to search for something, but apparently others have found some shit useful enough to click on it anyway.
Last week, they emailed me again to say there’s another new issue, so off to check it out. Google decided that a page under /pages/ was the same as a different page, and overrode my user-selected canonical URL for it. That’s weird… I shouldn’t have anything at that URL.
You see for things that aren’t really blog entries (things that I may come back and update, and aren’t really date-specific) I throw them under content/pages, but the slug of it typically renders elsewhere. A good example is the sysadmin section. Some of these won’t render unless there’s an _index.md in them though, for reasons which have long-since escaped me.
So how to fix it? First, let’s add /pages/ to robots.txt to stop search engines crawling it. Easy. Next, let’s make it so Hugo won’t render them, but will still generate the empty directories:
# Generate a page.md which won't render
cat >> /tmp/page.md
---
build:
render: never
---
^D
# Put that empty page.md everywhere there's an _index.md
find content/pages/ -name '_index.md' | while read i; do cp /tmp/page.md "$i"; done
Commit these changes, rebuild, and public/pages no longer exists, but all the other URLs appear to work. Hurrah!
However, for some reason Netlify are still showing them on the live site, and I’m not sure why.
Update: 2024-04-12: Looks like Netlify still have the /pages/ despite it not existing locally for me on a build. When I look at the build results, they’re showing ~14 more pages than my local environment builds, so looks like I have more investigative work to do yet?
First thing I tried, updating HUGO_VERSION, which was previously 0.59.1, to my current local version 0.124.1. That’s… pretty far behind. Let’s see if that fixes it?
