Add BaseRenderer to render page to text, and BaseHTMLRenderer as example #91

jamalex · 2020-01-05T17:32:17Z

Based on a discussion in #53 (comment), made a provisional example of an extensible "renderer" class could look like in Notion-py.

There's a base class, called BaseRenderer, and an example implementation of a renderer with BaseHTMLRenderer. Renderer classes can be extended, and parts of their behavior modified by overriding particular methods.

An example of using it:

from notion.renderers import BaseHTMLRenderer
page = client.get_block("...")
html = BaseHTMLRenderer(page).render()
print(html)

It's currently functional, but styling etc isn't great for many things. Luckily... it's extensible!

Cobertos

I really like this, having a general renderer and an example one (BaseHTMLRenderer) would make it super simple to make more renderers. Especially for a markdown export. A markdown export could even rely on the HTMLRenderer to provide things it doesn't support (like toggle blocks and how GitHub markdown still supports this)

Inline styles are my biggest concern. I know that if I were to use this output in my website, I'd have to do a ton of cleanup and reformat a lot of things. It might be nice to have like an UnformattedBaseHTMLRenderer and a FormattedBaseHTMLRenderer because I can see wanting nice output right from the get go if you don't want to style anything yourself.

Cobertos · 2020-01-05T22:37:22Z

notion/renderer.py

+		text = ""
+		for i in range(len(kids)):
+			text += self.render_block(kids[i], level=level)
+		return text


Possible simplification to "".join([self.render_block(child) for child in (block.children or [])])
(unless kids is always at least an array, then you can omit the or [])

Cobertos · 2020-01-05T22:39:28Z

notion/renderer.py

+
+	def create_opening_tag(self, tagname, attributes={}):
+		attrs = "".join(' {}="{}"'.format(key, val) for key, val in attributes.items())
+		return "<{tagname}{attrs}>".format(tagname=tagname, attrs=attrs)


Any reason to not use f"" strings? Python 3.6 feature though and it looks like this library targets 3.5+.

Cobertos · 2020-01-05T22:45:45Z

notion/renderer.py

+		return "{opentag}{innerhtml}</{tagname}>".format(opentag=opentag, tagname=tagname, innerhtml=innerhtml)
+
+	def left_margin_for_level(self, level):
+		return {"display": "margin-left: {}px;".format(level * 20)}


Is this mean to be {"style"... and not {"display"...? for inline styles?

Cobertos · 2020-01-05T22:47:03Z

notion/renderer.py

+		innerhtml = markdown2.markdown(getattr(block, fieldname))
+		return "{opentag}{innerhtml}</{tagname}>".format(opentag=opentag, tagname=tagname, innerhtml=innerhtml)
+
+	def left_margin_for_level(self, level):


IMO, inline styles would really complicate things for people wanting to post-process the output HTML or stick it into a HBS template or Angular/React/Vue app.

It'd be nicer if we could either add classes and provide a tiny CSS stylesheet (to make it easier to integrate into larger websites) or leave styling completely up to the dev calling this endpoint. This might be okay for now though, as a dev can always just Array.from(document.querySelectorAll('*')).forEach((el)=>el.setAttribute('style', '')) after importing.

Cobertos · 2020-01-05T22:54:07Z

notion/renderer.py

+		iframetag = self.create_opening_tag("iframe", attributes={
+			"src": block.display_source or block.source,
+			"frameborder": 0,
+			"sandbox": "allow-scripts allow-popups allow-forms allow-same-origin",


Are these defaults for embed sandbox sourced from somewhere/based on a best practice/needed for some of the default embeds that notion supports?

Cobertos · 2020-01-05T23:00:23Z

I might play with this in the coming weeks because I still haven't implemented a way to export all my Notion notes as blog posts on my website.

Cobertos · 2020-01-09T08:29:12Z

Not sure what the best way to make a PR of a PR is, but here's a draft PR of this revised #93

I basically stripped out most of the styling and rewrote the string stuff with dominate. I want to loop back and add a StyledHTMLRenderer that is the same thing but overrides the specific functions that would need specific styles and just adds attributes to the dominate.dom_tag objects.

Let me know of your thoughts, and perhaps if there's a better way to make this PR of a PR thingy, lol

indirectlylit · 2020-01-09T20:58:07Z

PR of a PR

You should be able to target your PR to the renderers branch:

Cobertos · 2020-01-11T07:55:11Z

PR of a PR

You should be able to target your PR to the renderers branch:

snip -

Ah yeah, that works a bit better, thanks.

dragonwocky · 2020-05-22T12:28:12Z

@Cobertos @jamalex I've used the concept of recursive per-block rendering to build a working HTMLRenderer with notion-py (renderer.py implemented by build.py - live @ https://dragonwocky.me/#posts).

It currently supports all notion content types (excluding of course factories / template buttons). Potentially unexpected but so far best-solution or unavoidable behaviours:

databases are linked to instead of rendered
pdf embeds are treated as file embeds
google drive embeds are treated as bookmarks
if exporting a non-public page, media (files + images etc.) will be inaccessible to viewers
not signed in to your notion account as they are not downloaded but served from notion's aws

I'd be happy for this to be integrated into notion-py (after some improvements if necessary). The project doesn't seem to have been recently updated, so I'm hoping for a response... but if I don't get one I'll probably go ahead and release this as a separate package anyway.

Cobertos · 2020-05-22T20:24:35Z

@dragonwocky

@jamalex works on this in bursts I think from what I've seen previously. So if you were to get it integrated into here it might be a little bit.

Also those are probably fine behaviors for a base renderer. I took a stab at implementing a similar thing but hadn't made it feature complete yet. #93 . Was mainly using it for my own blog, overriding specific behaviors like images and audio so that they save to disk where it'll deploy later to S3.

Fix lists to consider pre and postblocks

Add BaseRenderer to render page to text, and BaseHTMLRenderer as example

7121320

jamalex mentioned this pull request Jan 5, 2020

add_new doesn't handle markdown for nested blocks #53

Open

1 task

Cobertos reviewed Jan 5, 2020

View reviewed changes

dragonwocky mentioned this pull request May 11, 2020

Feature request / Question #141

Open

James Meyer and others added 2 commits June 5, 2020 18:54

Fix lists to consider pre and postblocks

a4dc56d

Merge pull request #146 from jamie-meyer/patch-1

a512a26

Fix lists to consider pre and postblocks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add BaseRenderer to render page to text, and BaseHTMLRenderer as example #91

Add BaseRenderer to render page to text, and BaseHTMLRenderer as example #91

jamalex commented Jan 5, 2020

Cobertos left a comment

Cobertos Jan 5, 2020

Cobertos Jan 5, 2020

Cobertos Jan 5, 2020

Cobertos Jan 5, 2020

Cobertos Jan 5, 2020

Cobertos commented Jan 5, 2020

Cobertos commented Jan 9, 2020 •

edited

Loading

indirectlylit commented Jan 9, 2020

Cobertos commented Jan 11, 2020 •

edited

Loading

dragonwocky commented May 22, 2020

Cobertos commented May 22, 2020

Add BaseRenderer to render page to text, and BaseHTMLRenderer as example #91

Are you sure you want to change the base?

Add BaseRenderer to render page to text, and BaseHTMLRenderer as example #91

Conversation

jamalex commented Jan 5, 2020

Cobertos left a comment

Choose a reason for hiding this comment

Cobertos Jan 5, 2020

Choose a reason for hiding this comment

Cobertos Jan 5, 2020

Choose a reason for hiding this comment

Cobertos Jan 5, 2020

Choose a reason for hiding this comment

Cobertos Jan 5, 2020

Choose a reason for hiding this comment

Cobertos Jan 5, 2020

Choose a reason for hiding this comment

Cobertos commented Jan 5, 2020

Cobertos commented Jan 9, 2020 • edited Loading

indirectlylit commented Jan 9, 2020

Cobertos commented Jan 11, 2020 • edited Loading

dragonwocky commented May 22, 2020

Cobertos commented May 22, 2020

Cobertos commented Jan 9, 2020 •

edited

Loading

Cobertos commented Jan 11, 2020 •

edited

Loading