SEO for dartdoc - consider canonical pages, etc

[kevmoo]

#1016 made me think of this

Search engines like a clear notion of "the best" page where there is a sea of very similar pages.

When docs are indexed, I'm worried that the best page for pkg will change from /documentation/pkg/1.0.0 to /documentation/pkg/1.0.1 to /documentation/pkg/1.0.2

I think doc URLs should be like package URLs

https://pub.dartlang.org/packages/pana does not redirect to https://pub.dartlang.org/packages/pana/versions/0.10.2

Instead https://pub.dartlang.org/packages/pana/versions/0.10.2 has a cononical tag that points search engines back to the "main" page

<link rel="canonical" href="https://pub.dartlang.org/packages/pana"/>

We should do the same w/ docs.

Thoughts?

[kevmoo]

CC @isoos @jakobr-google @mit-mit

[kevmoo]

See https://support.google.com/webmasters/answer/66359?hl=en

[isoos]

I think that is reasonable, but we'll need to have a clear break from the current url structure, because /documentation/pkg/1.0.2 needs to become /documentation/pkg/versions/1.0.2, or otherwise each request should check whether the path segment after the package is a version or a directory name.

Another alternative is to move documentation under pub.dartlang.org/packages/<pkg>/doc and pub.dartlang.org/packages/<pkg>/versions/<version>/doc...

[kevmoo]

Another alternative is to move documentation under pub.dartlang.org/packages//doc and pub.dartlang.org/packages//versions//doc...

Sure. I don't think the versioned flavor of the URLs needs to be super pretty.

I think having good SEO here will be a big help

[isoos]

I think having good SEO here will be a big help

There are two huge changes coming up with the new customization:

• Previously dartdocs.org had no links back to the pub site, now we do have a generic link on each page.
• Previously dartdocs.org pointed to the package homepage, now we point back to the package page on pub (on each page).

While previously dartdocs.org drained out pagerank scores, now it will be helping it.

[kevmoo]

No huge rush here – let's get the migration done.

…

On Tue, Feb 27, 2018 at 4:37 AM, István Soós ***@***.***> wrote: I think having good SEO here will be a big help There are two huge changes coming up with the new customization: - Previously dartdocs.org had no links back to the pub site, now we do have a generic link on each page. - Previously dartdocs.org pointed to the package homepage, now we point back to the package page on pub (on each page). While previously dartdocs.org drained out pagerank scores, now it will be helping it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1017 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABCikoVpxAAwNw5FlyvUfZVeXr3wpcdks5tY_bwgaJpZM4SRQpi> .

[isoos]

With #1165 closed we have one new issue here:
documentation/pana/latest/ will have a canonical meta tag like this:

<link rel="canonical" href="https://pub.dartlang.org/documentation/pana/0.10.5/index.html">

This will be a bit "confusing" for search engines, but rewriting it to pana/latest/index.html could also introduce a lag while two packages reference the same /latest/ as canonical version. I think the lesser wrong is to rewrite it to pana/latest/index.html, WDYT?

[kevmoo]

Here's another idea.

• pkg page links to docs/pkg/[version]

  • The landing page for docs/pkg/[version] has a canonical link pointing to docs/pkg/latest
  • All other pages under docs/pkg/[version] are marked as "don't index"

• Also host docs/pkg/latest which always points to latest (of course) – don't redirect!

  • These are NOT excluded from index
  • The root of these pages is also added to the sitemap

No redirect crazy.
Users link to the right version of the docs
Search is pointed always at "latest"

Maybe?

[isoos]

I'd slightly modify it:

When generating the dartdocs, we have two versions:

• latestStable will get the docs/pkg/latest as canonical and will not be excluded from index,
• everything else will get the docs/pkg/version as canonical and will be excluded from index.

The difference from yours is that we will host the same version of the page under pkg/latest and pkg/2.0.0 (if that is the latest), and the package page will link to pkg/latest on the main page and pkg/version on every other page.

That means we can keep most of the doc generation and serving logic (no need to have two versions for the latest package), while keeping most of the benefits.

[kevmoo]

SGTM. Michael should weigh in.

…

On Thu, Apr 5, 2018 at 2:25 AM István Soós ***@***.***> wrote: I'd slightly modify it: When generating the dartdocs, we have two versions: - latestStable will get the docs/pkg/latest as canonical and will not be excluded from index, - everything else will get the docs/pkg/version as canonical and will be excluded from index. The difference from yours is that we will host the same version of the page under pkg/latest and pkg/2.0.0 (if that is the latest), and the package page will link to pkg/latest on the main page and pkg/version on every other page. That means we can keep most of the doc generation and serving logic (no need to have two versions for the latest package), while keeping most of the benefits. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1017 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABCioEsmNXB8JDsEQGyS7zudTKtfKBMks5tleMLgaJpZM4SRQpi> .