fix: Improve SEO canonicalization and block non-indexable pages #2706

HarshMN2345 · 2026-01-20T07:51:22Z

What does this PR do?

(Provide a description of what this PR does.)

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.)

Related PRs and Issues

(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)

Have you read the Contributing Guidelines on issues?

(Write your answer here.)

Summary by CodeRabbit

Chores
- Canonical docs version standardized to the cloud variant; canonicalization logic simplified.
SEO / Indexing
- Added noindex handling for internal, staging, and non-canonical doc versions via response headers and meta tags.
Sitemap & robots.txt
- Tightened sitemap exclusions and expanded robots rules to block internal/auth routes, tracking parameters, and non-cloud docs references while exposing canonical content.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2026-01-20T07:52:04Z

Walkthrough

Adds SEO controls: a new middleware seoOptimization (run after initSession) injects x-robots-tag: noindex, nofollow for configured internal paths and staging hosts; sitemap generation adds an INTERNAL_PATHS list and excludes internal/auth routes, /threads/*, .json/.xml files, and docs references not under /docs/references/cloud/; CANONICAL_VERSION was removed and normalizeDocsVersion treats 'cloud' as the canonical version; a docs references layout emits a meta robots noindex for non‑canonical versions; robots.txt rules were expanded to cover tracking params, internal/auth pages, and versioned docs rules.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: Improve SEO canonicalization and block non-indexable pages' clearly and concisely summarizes the main changes across all modified files, accurately reflecting the core objectives of enhancing SEO practices.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@src/routes/robots.txt/`+server.ts:
- Line 26: Remove the unnecessary backslashes used to escape double quotes
inside the template literal that builds the robots.txt content (the literal that
blocks versioned docs and references "cloud"); change occurrences of \"cloud\"
(and any other \"...\" instances) to plain "cloud" (and "..." respectively) so
the template uses normal double quotes within the template literal, then re-run
linting to ensure ESLint warnings are resolved.

🧹 Nitpick comments (1)

src/routes/robots.txt/+server.ts (1)

27-33: Consider future-proofing version blocking.

The current approach explicitly blocks versions starting with 0, 1, 2, 3. If new major versions (e.g., 4.x, 5.x) are released in the future, additional Disallow rules would need to be added manually.

An alternative approach would be to invert the logic—disallow all of /docs/references/ and then explicitly allow only /docs/references/cloud/. However, robots.txt precedence rules vary by crawler (Google uses most-specific-wins, others may use first-match), so the current explicit approach is safer and more predictable.

coderabbitai · 2026-01-20T11:24:47Z

src/routes/robots.txt/+server.ts

+Disallow: /console/register
+Disallow: /v1/
+
+# Block all versioned docs references; only \"cloud\" should be indexable


⚠️ Potential issue | 🟡 Minor

Remove unnecessary escape characters in template literal.

In JavaScript template literals, double quotes don't require escaping. The backslashes before the quotes are unnecessary and trigger ESLint errors.

Proposed fix

-# Block all versioned docs references; only \"cloud\" should be indexable +# Block all versioned docs references; only "cloud" should be indexable

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

# Block all versioned docs references; only \"cloud\" should be indexable

# Block all versioned docs references; only "cloud" should be indexable

🧰 Tools

🪛 ESLint

[error] 26-26: Unnecessary escape character: ".

(no-useless-escape)

[error] 26-26: Unnecessary escape character: ".

(no-useless-escape)

🤖 Prompt for AI Agents

In `@src/routes/robots.txt/`+server.ts at line 26, Remove the unnecessary backslashes used to escape double quotes inside the template literal that builds the robots.txt content (the literal that blocks versioned docs and references "cloud"); change occurrences of \"cloud\" (and any other \"...\" instances) to plain "cloud" (and "..." respectively) so the template uses normal double quotes within the template literal, then re-run linting to ensure ESLint warnings are resolved.

ItzNotABug · 2026-01-27T08:07:39Z

src/routes/robots.txt/+server.ts

+Disallow: /docs/references/0
+Disallow: /docs/references/1
+Disallow: /docs/references/2
+Disallow: /docs/references/3
+
+# Allow canonical cloud docs
+Allow: /docs/references/cloud/`;


why no just -

# Block all versioned docs references Disallow: /docs/references/*/ # Allow canonical cloud docs Allow: /docs/references/cloud/

ItzNotABug · 2026-01-27T08:08:30Z

src/hooks.server.ts

+    /^\/v1\//
+];
+
+const NOINDEX_HOSTS = [/^stage\./i, /^fra\./i, /^internal\./i];


where did internal come from? what about other regions? maybe any *.cloud.appwrite.io makes more sense?

ItzNotABug · 2026-01-27T08:10:09Z

src/routes/robots.txt/+server.ts

+Disallow: /*&ref=
+Disallow: /*&adobe_mc=
+Disallow: /*&trk=
+


wildcards should work here -

# Block tracking parameters Disallow: /*?utm_ Disallow: /*&utm_ Disallow: /*?ref= Disallow: /*&ref= Disallow: /*?trk= Disallow: /*&trk= Disallow: /*?adobe_mc= Disallow: /*&adobe_mc=

fix:Improve SEO canonicalization and block non-indexable pages

b4cdea6

use cloud

a909359

coderabbitai bot reviewed Jan 20, 2026

View reviewed changes

HarshMN2345 added 3 commits January 21, 2026 11:09

Merge branch 'main' into fix-SER-1068-search-console-issues

0a30c67

Merge branch 'main' into fix-SER-1068-search-console-issues

3858842

Merge branch 'main' into fix-SER-1068-search-console-issues

1000a63

ItzNotABug requested changes Jan 27, 2026

View reviewed changes

HarshMN2345 and others added 2 commits January 27, 2026 18:01

Merge branch 'main' into fix-SER-1068-search-console-issues

e6f1675

addressed some comments

17fe5a7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Improve SEO canonicalization and block non-indexable pages #2706

fix: Improve SEO canonicalization and block non-indexable pages #2706

HarshMN2345 commented Jan 20, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Jan 20, 2026 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Jan 20, 2026

Uh oh!

ItzNotABug Jan 27, 2026

Uh oh!

ItzNotABug Jan 27, 2026

Uh oh!

ItzNotABug Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# Block all versioned docs references; only \"cloud\" should be indexable
	# Block all versioned docs references; only "cloud" should be indexable

fix: Improve SEO canonicalization and block non-indexable pages #2706

Are you sure you want to change the base?

fix: Improve SEO canonicalization and block non-indexable pages #2706

Conversation

HarshMN2345 commented Jan 20, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Test Plan

Related PRs and Issues

Have you read the Contributing Guidelines on issues?

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Estimated code review effort

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

ItzNotABug Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

ItzNotABug Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

ItzNotABug Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HarshMN2345 commented Jan 20, 2026 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Jan 20, 2026 •

edited

Loading