Scaladoc generate flatten structure for API rather then put everything in api directory

[romanowski]

Previously scaladoc keeps always API pages under api dir. This PR makes this flat, so documentation for foo.Bar is now at /foo/Bar.html rather then /api/foo/Bar.html. In case of conflicts with static pages or resources errors are reported.

This PR also introduces -Ylegacy-api-layout that restores old layout.

As a bonus for @julienrf I've added an option to generate just API in provided directory using scaladoc/generateScalaDocumentation [output] --justAPI task in sbt.

[smarter]

Is this really a better default if it can lead to conflict when the previous version didn't? Looking at scala/docs.scala-lang#2136 it seems that @julienrf can make symbolic links on the webserver, so he could use whatever path he wants anyway (also I don't really understand why we're moving away from having the documentation and the api all accessible from the same menu as in http://dotty.epfl.ch/docs/)

[julienrf]

it seems that @julienrf can make symbolic links on the webserver, so he could use whatever path he wants anyway

True, but that would be simpler to just upload our API documentation on the server, without having to make a symbolic link.

also I don't really understand why we're moving away from having the documentation and the api all accessible from the same menu as in http://dotty.epfl.ch/docs

I agree it’s nice to have everything in one place, but I don’t think this is feasible now because we have legacy Jekyll websites, docs.scala-lang.org and scala-lang.org, whose content can’t be just dismissed. So, my strategy was to move the content of dotty.epfl.ch in docs.scala-lang.org so that we can have the Scala 2 and Scala 3 documentation available at the same place, and the last step is to publish the API documentation in scala-lang.org/api, like we do with all the other versions of Scala.

[julienrf]

@romanowski Would it be possible to generate the documentation for 3.0.0 and 3.0.1 with this?

Also, what do you think of the following suggestion?

• if there is no -siteroot option, we generate only the API documentation at the top-level (no api/ subdirectory)
• if there is a -siteroot option, we generate the whole static site, including the documentation under the api/ subdirectory
  • we provide a setting to let users provide an alternative path instead of the default api/, if they want

[romanowski]

Is this really a better default if it can lead to conflict when the previous version didn't?

I feared the same initially but then I realized that we can have conflicts only if one has top-level package called docs and then have a .md files matching API members (so you would need to have a file foo/bar/Baz.md and e.g. class docs.foo.bar.Baz) or if one has a top-level package called scripts, images or resources and have a .html file in those dirs that matches fqn name of class/object. So basically chances from conflicts are rather small hece having an error reported in such situation should be good enough. I realized that I should make errors more verbose, suggesting adding -Ylegacy-api-layout to resolve the problem.

Would it be possible to generate the documentation for 3.0.0 and 3.0.1 with this?

It depends if tasty has changed. Changes are pretty small. so I am sure it would be possible to cherry-pick the changes and run from those branches.

• if there is no -siteroot option, we generate only the API documentation at the top-level (no api/ subdirectory)

I think that having different layouts will be confusing and when one would like to introduce a static site then all paths will change

• if there is a -siteroot option, we generate the whole static site, including the documentation under the api/ subdirectory
  we provide a setting to let users provide an alternative path instead of the default api/, if they want

I am not sure how helpful it will be since HTML generated by scaladoc are self-contained and placing API outside structure will render it useless. I am not really sure if making supath of API configurable is really worth added complexity.

[BarkingBad]

@romanowski Would it be possible to generate the documentation for 3.0.0 and 3.0.1 with this?

There was PR that was supposed to make automatic releases of older docs with new scaladoc user interface #12784 Nonetheless, if we want to publish them just once, we can:

• take them from here (old scaladoc interface, no color themes etc.)
• publish 3.0.0 artifacts then supply them to the new version of scaladoc manually (it's only once and we get docs with the current user interface)

If we want to get automatic releases we should revive this PR though
Also, we can use feature for easy linking scaladoc versions but beforehand we have to settle up how the docs will be published and will they be updated for older versions to improve UI

Also, what do you think of the following suggestion?

• if there is no -siteroot option, we generate only the API documentation at the top-level (no api/ subdirectory)
• if there is a -siteroot option, we generate the whole static site, including the documentation under the api/ subdirectory
  we provide a setting to let users provide an alternative path instead of the default api/, if they want

I think I like this idea since we are growing fast in number of scaladoc settings and this seems right to don't bother user about nesting of the docs, on the other hand Krzysztof is right that it can be misleading for some users.

[julienrf]

I think that having different layouts will be confusing and when one would like to introduce a static site then all paths will change

Good point.

On the other hand, I must say that I still feel a bit uncomfortable with having the paths of the types and values defined in the projects mixed with the paths of the static pages. I feel like putting them in a sub-path (e.g. api/) would be cleaner (in terms of structure, regardless of the small possibility of having conflicts).

Maybe it’s OK to document clearly that -siteroot moves the API documentation to a sub-path, and that users should take as early as possible the decision of using -siteroot?

@BarkingBad It would be really nice to have a dropdown allowing us to switch between API documentation versions, while still browsing the API documentation for some version, but I would not say it is urgent to have. I would say the top priority is to get the documentation automatically published on https://scala-lang.org/api/3.x/ for every new release.

[romanowski]

Maybe it’s OK to document clearly that -siteroot moves the API documentation to a sub-path, and that users should take as early as possible the decision of using -siteroot?

The whole point is that static documentation leaves in docs and blog subdirectories already so one is not able to add foo/bar/Baz.md top level (it will be ignored) it needs to be placed in docs

[julienrf]

@romanowski I am not much concerned by possible clashes (still, one can have a page docs/foo.md and a package foo in its project, right?), but just about the structure of the resulting website. It feels a bit awkward to me to have the API documentation mixed with the other pages.

[romanowski]

Is this that a big problem from the user's perspective? I've generated a version of dotty website that merges docs and APIs and I think I prefer to have it that way. (https://scala3docstorage.blob.core.windows.net/pr-flat-api-demo/scala3 is a root of documentation)

[julienrf]

After a discussion, I realized that my comment was wrong: there is already a separation between the API documentation and the static pages (which are rendered in the docs/ or blog/ sub-paths), so it’s all fine.

Now, I wonder if we should really maintain the -Ylegacy-api-layout? Why not drop it?

[julienrf]

Could you add a comment explaining what this does? I’m a bit lost.

[BarkingBad]

It seems it just a rewrite (no new logic is added here).

Before:
we got one file by calling siteContext.indexTemplate()

Now:
we call siteContext.indexTemplates() which returns collection (effectively only one file), then we pattern match on it just to replace it with emptyIndextemplate (which was embedded in old siteContext.indexTemplate()) if it was empty or get identity otherwise and then append it as a collection.

Can you @romanowski explain if I get something wrong here? The only thing is granularity (we have more designated functions to call), but on the other hand main misconception here is that indexTemplates() actually can yield 0 or 1 file, never 2.

[romanowski]

I will rewrite to Option

[BarkingBad]

It should be indexFile as we get only one file (either html or md) but not both at once. Moreover it would never be files becuase of assertion in line 25

[BarkingBad]

It seems it just a rewrite (no new logic is added here).

Before:
we got one file by calling siteContext.indexTemplate()

Now:
we call siteContext.indexTemplates() which returns collection (effectively only one file), then we pattern match on it just to replace it with emptyIndextemplate (which was embedded in old siteContext.indexTemplate()) if it was empty or get identity otherwise and then append it as a collection.

Can you @romanowski explain if I get something wrong here? The only thing is granularity (we have more designated functions to call), but on the other hand main misconception here is that indexTemplates() actually can yield 0 or 1 file, never 2.

[julienrf]

Suggested change

	BooleanSetting("-Yapi-subdirectory", "Keep all api member inside `api` directory", false)
	BooleanSetting("-Yapi-subdirectory", "Put the API documentation pages inside a directory `api/`", false)

[julienrf]

@romanowski How can I generate the API documentation for a specific version of the scala3-library with this? I usually just run scaladoc/generateScalaDocumentation, which generates the documentation for the current scala3-library.

[BarkingBad]

How can I generate the API documentation for a specific version of the scala3-library with this?

I partially answered it in this comment #13130 (comment)

For now the easiest option is to checkout to specific version and run scaladoc/generateScalaDocumentation. Also you can checkout older commit and release jars then publish them with new scaladoc manually from the CLI (for new scaladoc UI).

We can adjust the draft PR I mentioned in the comment I already mentioned so we would could do it automatically.

[julienrf]

Also you can checkout older commit and release jars then publish them with new scaladoc manually from the CLI (for new scaladoc UI).

This is what I want to achieve, thanks! May I ask you to provide the suite of commands I have to invoke? Especially the part related to invoking the CLI.