Feature request: add meta tag that matches actual output character encoding

[AthanasiusOfAlex]

Tidy currently allows you to output a document in 14 character encodings. The problem is, unless you use ascii, the web browser generally gets confused by it. It would be nice if Tidy had the option of adding the "meta" tag that matches the encoding, e.g.,

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

for UTF-8 (which is the default). For the moment, I have to add this line manually.

[geoffmcl]

@AthanasiusOfAlex thanks for the Feature Request

It seems a reasonable feature request, as a new option, to add this, styled depending on the document type detected, like <meta charset="utf-8"> for html5, especially since it seems the W3C validator barks at you if it is missing...

Maybe the option could be say --add-meta-charset, or something... Boolean, default no...

There is an OPTIONS.md which details the adding of an option... quite simple...

Would certainly consider this addition, if you or others presented code... thanks...

[marcoscaceres]

I'm currently learning c, so I wouldn't mind taking this on (as it seems like a simple enough feature request).

@geoffmcl, could you mentor me on it? (i.e., give me some pointers as to where it might be good place to insert this in the code tree, etc.).

[geoffmcl]

@marcoscaceres, thanks for your offer to take this on, and for sure I will help where I can...

As mentioned OPTIONS.md gives the 4 steps to do this... and we should discuss and decide here the values for 1, 2, and 3... my first suggestion is as follows, but am open to other ideas -

1. ID - TidyMetaCharset
2. table - MS, "add-meta-charset", BL, no, ParseBool, boolPicks
3. description - "This option adds a charset meta tag, appropriate to the document type, in the document head."

As stated, these are just quick, first thoughts, suggestions, and look forward to them being massaged...

As to part 4., using it in the code, take a look at AddGenerator, in lexer.c. You could add a new say TY_(AddMetaCharset)(doc), which does a similar thing - FindHEAD, and if found, search for, and maybe fix any existing meta charset entry, and return, or continue down and add a new one if none, inserting it in the node tree...

Note the different styles, based on whether this is html5, xhtml, or a legacy document... and like the TidyMark, this could be called from tidyDocCleanAndRepair() in tidylib.c, if the option is on...

Look forward to your code, either as a patch, or make a fork, add say a branch, and present it as a PR... thanks!

[marcoscaceres]

@geoffmcl thanks for all the pointers, this is super helpful!

Only additional requirement is to make sure that the meta charset appears in the first 512 bytes, as per HTML5. I'll try to send this in parts over the next week.

[marcoscaceres]

Regarding

1. TidyMetaCharset, SGTM.
2. add-meta-charset, SGTM.
3. Description: /**< Adds/checks/fixes meta charset in the head, based on document type */

Let's see how I go with 4 now :)

[geoffmcl]

See WIP PR #458

[geoffmcl]

Since PR #458 has already been move to milestone 5.5, doing likewise here...

I hope you get the time to finish this great WIP soon... please ask if you need further help... thanks...

[marcoscaceres]

Thanks @geoffmcl - doing my best to find time. If anyone wants to jump in and do a bit more hacking on it, I would certainly be supportive. It's nearly there!