Split sidebars out into separate HTML files #3384

srawlins · 2023-04-07T18:41:18Z

This takes us in the direction of a Single Page App. The idea is to solve the output size issues (#3311, #2995, ...) by not generating duplicate sidebars everywhere.

Instead we can generate the sidebars as separate pages, and load them dynamically. This PR implements #3079.

Since we do not have functional tests for the generated output (web tests), I'll verify that I manually tested these things:

pub package
- PACKAGE_NAME=collection PACKAGE_VERSION=1.17.1 dart run grinder serve-pub-package
- tested the collection package, version 1.17.1. Both left and right sidebar links work, breadcrumbs work, search links work, doc comment reference links work, type signature links work.
- The body tag looks like <body data-base-href="../" data-using-base-href="false">.
- total output size before/after: 3.6 MB / 3.0 MB -- savings of 17%
- total file count before/after: 314 / 330 -- increase of 5%
flutter
- The body tag looks like <body data-base-href="../" data-using-base-href="false">.
- total output size before/after: 9301 MB / 752 MB -- savings of 92%
- total file count before/after: 77273 / 81470 -- increase of 5%
Dart SDK
- dart run grinder serve-sdk-docs
- tested, same links as above.
- The body tag looks like <body data-base-href="../" data-using-base-href="true">.
- total output size before/after: 309 MB / 147 MB -- savings of 53%
- total file count before/after: 13878 / 15078 -- increase of 9%

jcollins-g · 2023-04-19T18:15:23Z

I think this came in when I was in the weeds of the 3.0 analytics push. I think this is great! It looks like we are having some problems with the tests though, maybe a sync / rebuild?

srawlins · 2023-04-20T22:04:38Z

CI is green 😁

srawlins · 2023-04-20T22:06:16Z

lib/src/generator/template_data.dart

@@ -99,6 +99,10 @@ abstract class TemplateData<T extends Documentable> extends TemplateDataBase {
  @override
  T get self;

+  String? get aboveSidebarPath;


Each template now has an optional aboveSidebarPath and a belowSidebarPath which is included in the main HTML, and the JS uses those paths to request each sidebar client-side.

srawlins · 2023-04-20T22:07:23Z

lib/src/generator/template_data.dart

 }

 /// Template data for Mixin declarations.
 class MixinTemplateData extends InheritingContainerTemplateData<Mixin> {
-  MixinTemplateData(super.htmlOptions, super.packageGraph, super.library,
-      super.mixin, super.sidebarForLibrary, super.sidebarForContainer);
+  MixinTemplateData(


These constructors no longer need to take sidebarForLibrary and sidebarForContainer, the previously computed-and-cached HTML blobs.

srawlins · 2023-04-20T22:09:39Z

lib/src/model/container.dart

@@ -268,6 +268,15 @@ abstract class Container extends ModelElement
  @override
  String get filePath => '${library.dirName}/$fileName';

+  /// The full path of this element's sidebar file.
+  String get sidebarPath;


Containers have a notion of their own sidebar path (their "below" sidebar path), and then their child elements can call enclosingElement.sidebarPath for their aboveSidebarPaths.

srawlins · 2023-04-20T22:13:20Z

lib/src/generator/html_generator.dart

@@ -37,6 +37,52 @@ class HtmlGeneratorBackend extends GeneratorBackendBase {
  HtmlGeneratorBackend(
      super.options, super.templates, super.writer, super.resourceProvider);

+  @override
+  void generateClass(PackageGraph packageGraph, Library library, Class clazz) {


Each of these which used to just render one HTML file must now render two, with the addition of their sidebar file, so they are overridden here, adding a lot of repeated boilerplate.

srawlins · 2023-04-20T22:21:20Z

lib/src/generator/generator_backend.dart

@@ -56,19 +56,6 @@ class DartdocGeneratorBackendOptions implements TemplateOptions {
        resourcesDir = context.resourcesDir;
 }

-class SidebarGenerator<T extends TemplateData> {


We used to cache the sidebars generated, and no longer need to.

srawlins · 2023-04-20T22:21:46Z

lib/src/generator/generator_backend.dart

@@ -205,8 +184,7 @@ abstract class GeneratorBackendBase implements GeneratorBackend {

  @override
  void generateClass(PackageGraph packageGraph, Library library, Class clazz) {
-    var data = ClassTemplateData(options, packageGraph, library, clazz,


We no longer need to pass around these sidebar generators.

jcollins-g

This seems fine as long as any performance regression is not too severe. See my one comment.

jcollins-g · 2023-04-21T17:20:19Z

lib/src/generator/html_generator.dart

+  void generateClass(PackageGraph packageGraph, Library library, Class clazz) {
+    super.generateClass(packageGraph, library, clazz);
+    var data = ClassTemplateData(options, packageGraph, library, clazz);
+    var sidebarContent = templates.renderSidebarForContainer(data);


Doesn't this undo the performance benefits of caching the sidebar? We're going to be running the template for the sidebar repeatedly unless I'm misreading something.

This shouldn't undo the caching benefits we had earlier (it should more-or-less mimic them): The problem we had before caching is that each page would independently generate it's left sidebar (sibling elements) and right sidebar (children), so that the left sidebar was being generated n+1 times for n siblings (by each of the n siblings, and once by the parent). The caching mechanism ensured each sidebar was generated once. This change does something similar: each TemplateData object is responsible for generating its own right sidebar, containing is children elements.

OK, I had been misreading the code then. LGTM

srawlins · 2023-04-27T20:15:33Z

Some timing notes. I checked three cases, and ran multiple times. I used the "Documented in n seconds" text as the metric.

collection (a very small package): 17.8s before, and 17.9s after. A slow-down of 0.6%.
win32 (has tons of constants): 690s before, and 59.2s after. An improvement of 91%.
flutter (huge): 1684s before, and 1665s after. An improvement of 1.1%.

There is a huge deviation in the times to document flutter, and even a moderate deviation in the others. So anything under 5 or 10% is probably within the margin of error. I'm considering it a no-op for typical packages, and even large packages like flutter. It's an improvement for packages with massive numbers of constants and variables and functions relative to their containers.

jcollins-g · 2023-05-01T17:36:30Z

Looks like a resync/grind build run should be all that's needed here to land?

srawlins · 2023-05-03T19:43:30Z

Ok, 😮‍💨 I got CI back to green. I think this is good to go if you do.

jcollins-g · 2023-05-11T18:52:25Z

Yes, let's get this landed. I want to publish now that 3.0 is out, and it would be great to have this included.

…c pages (#128442) This updates dartdoc to 6.3.0. Release notes are available, here: https://github.com/dart-lang/dartdoc/releases/tag/v6.3.0 Most important for Flutter are the reduction in the size of generated HTML files (dart-lang/dartdoc#3384) and a new dartdoc directive to hide constant implementations from indicated classes (dart-lang/dartdoc#3398), which fixes the longstanding issue (dart-lang/dartdoc#2657). I've also added the api documentation zip to `.gitignore` and the `{@hideConstantImplementations}` dartdoc directive to the motivating example. A screenshot: ![Screenshot 2023-06-07 at 9 54 58 AM](https://github.com/flutter/flutter/assets/14116827/1ad9c1f0-b224-462f-a8e3-706d9858f0d8) I assert that this change to icons.dart should be test-exempt as existing tests cover whether or not dartdoc directives are recognized or are leaking into HTML, and the impact of adding the directive was tested in dart-lang/dartdoc#3398.

srawlins added 2 commits April 7, 2023 11:21

Remove all the sidebar overhead!

a752db8

Above and below sidebars

b087388

jonasfj mentioned this pull request Apr 11, 2023

Single Page App design with static files #3079

Closed

srawlins added 4 commits April 19, 2023 12:04

Merge branch 'master' into spa

51c87e9

regenerate

ef43a73

remove spa-link

3d40e19

Tweaks and fixes

080569e

Remove SidebarGenerator

2dc244b

srawlins commented Apr 20, 2023

View reviewed changes

jcollins-g approved these changes Apr 21, 2023

View reviewed changes

srawlins added 5 commits May 2, 2023 12:16

regenerate

408247b

regenerate

c38a515

regenerate

9f688df

Merge branch 'main' into spa

17cd765

Merge branch 'main' into spa

8d544d8

srawlins added 2 commits May 11, 2023 15:52

Merge branch 'main' into spa

6922b3a

remove-stuff

8309836

srawlins merged commit 2ae78bc into dart-lang:main May 12, 2023

srawlins deleted the spa branch May 12, 2023 20:34

jcollins-g mentioned this pull request Jun 7, 2023

Update flutter to dartdoc 6.3.0 and hide Icons implementation from doc pages flutter/flutter#128442

Merged

8 tasks

Levi-Lesches mentioned this pull request Jul 20, 2023

Left sidebar not showing on some docs #3467

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split sidebars out into separate HTML files #3384

Split sidebars out into separate HTML files #3384

srawlins commented Apr 7, 2023 •

edited

Loading

jcollins-g commented Apr 19, 2023

srawlins commented Apr 20, 2023

srawlins Apr 20, 2023

srawlins Apr 20, 2023

srawlins Apr 20, 2023

srawlins Apr 20, 2023

srawlins Apr 20, 2023

srawlins Apr 20, 2023

jcollins-g left a comment

jcollins-g Apr 21, 2023

srawlins Apr 26, 2023

jcollins-g Apr 26, 2023

srawlins commented Apr 27, 2023

jcollins-g commented May 1, 2023

srawlins commented May 3, 2023

jcollins-g commented May 11, 2023

Split sidebars out into separate HTML files #3384

Split sidebars out into separate HTML files #3384

Conversation

srawlins commented Apr 7, 2023 • edited Loading

jcollins-g commented Apr 19, 2023

srawlins commented Apr 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcollins-g left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srawlins commented Apr 27, 2023

jcollins-g commented May 1, 2023

srawlins commented May 3, 2023

jcollins-g commented May 11, 2023

srawlins commented Apr 7, 2023 •

edited

Loading