DevTools SetCookie support

[mitchcapper]

the cef SetCookie support is limited not supporting things like SameSite settings. DevTools does not have this problem.

Do we want to start a dev tools extension class? I can submit a PR but here is setCookie working:

public enum CookieSameSite { None, Strict, Lax }

		public async Task<bool> SetCookie(String name, String value, String domain, long? expires = null, bool secure = false, bool httpOnly = false, String path = "", CookieSameSite same_site = CookieSameSite.None) {
			var res = await browser.ExecuteDevToolsMethodAsync(0, "Network.setCookie", new Dictionary<string, object> {
				{ "name", name}, { "value", value}, { "domain",domain}, { "expires", expires },
				{ "secure", secure },{ "httpOnly", httpOnly },{ "path",path ?? ""},{"sameSite",same_site.ToString()}
			});
			return res != 0;
		}
		public Task<bool> SetCookie(String name, String value, String domain, DateTimeOffset? expires = null, bool secure = false, bool httpOnly = false, String path = "", CookieSameSite same_site = CookieSameSite.None) => SetCookie(name, value, domain, expires?.ToUnixTimeSeconds(), secure, httpOnly, path, same_site);

[amaitland]

I created issue #3165 with the intention of writing a wrapper API around CEF DevTools methods, just not quite clear what that implementation looks like just yet, if you've got some suggestions input is welcome. I had a look at what other .Net DevTools protocol implentations with the plan on using Roslyn to generate the API wrappers based on the actual DevTools protocol definition, haven't made any meaningful progress yet.

By all means submit a PR, there is already https://github.com/cefsharp/CefSharp/blob/master/CefSharp/DevToolsExtensions.cs which your welcome to add to. Default builds still use vs2015 so you'll be limited in syntax that you can use for now.

[mitchcapper]

It is true, we likely could write something that either parsed the native code to generate the devtool methods automatically or even did so from a documentation POV. Given the fact dev tool methods likely don't change that often it could be done as a one off to get the initial work done and then manually tweak them by hand to make them a bit more user friendly.

I was wondering if we wanted to do them all as extension methods or if we wanted to create a class to put them under. The syntax could be browser.DevToolMethods().SetCookie to avoid cluttering the main browser object. If we are trying to mimic the official devtools syntax we should probably stick to subclasses as well so browser.DevToolMethods().Network.SetCookie

granted if devtools renames something we would have to decide if we wanted to break back compat, use a placeholder function, or stick with the original name.

[amaitland]

DevTools protocol is defined in two pdl files, there's a python script to convert them to JSON which can easily be parsed.

https://source.chromium.org/chromium/chromium/src/+/master:third_party/inspector_protocol/convert_protocol_to_json.py

It looks like we might be able to get the JSON definitions from https://github.com/ChromeDevTools/devtools-protocol/tree/master/json

The finer details like how the different DevTools domains (page, network, etc) are represented isn't clear. I've been meaning to have another look at puppeteer/puppeteer-Sharp determine if it makes sense to model our API off an established standard.

As part of #3165 I plan to add a DevToolsClient class that will represent a single IBrowser instance, there's some object lifecycle management that we need to consider.

Having a tool that can be run manually on the rare occasion the API changes is what I was planning.

[amaitland]

Two existing .Net projects I've looked at briefly.

https://github.com/BaristaLabs/chrome-dev-tools-generator
https://github.com/MasterDevs/ChromeDevTools

[mitchcapper]

Do you have a timeframe on auto-generating the classes? I am fine submitting a PR so we could do manual wrappers for things in the mean time. I have already done some of the other storage classes could add user agent, and clearing cookies/cache/storage as well.

[amaitland]

I'm hoping to have the generated classes done for the 85 release.

A PR with some manually written extension methods would be welcome 👍 I can include them in the upcoming 84 release.

[amaitland]

Looking at the example provided in a little more detail

public async Task<bool> SetCookie(String name, String value, String domain, long? expires = null, bool secure = false, bool httpOnly = false, String path = "", CookieSameSite same_site = CookieSameSite.None)
{
	var res = await browser.ExecuteDevToolsMethodAsync(0, "Network.setCookie", new Dictionary<string, object>
	{
		{ "name", name}, { "value", value}, { "domain",domain}, { "expires", expires },
		{ "secure", secure },{ "httpOnly", httpOnly },{ "path",path ?? ""},{"sameSite",same_site.ToString()}
	});
	return res != 0;
}

The ExecuteDevToolsMethodAsync currently returns the message Id. The res != 0 check will only determine if a message Id was returned at not the success of setting the cookie.

Looking at the Network.setCookie should return json response containing a success (bool). To determine if the cookie was set successfully we'll need to add an observer to get the results.

I'll look at adding a generic observer that can be used to obtain command results initially.

[mitchcapper]

Yes essentially the commands I have done thus far only return false if the dev tools message can't be sent (as it returns 0 in this case. Obviously getting data out of dev tools is a plus.

I can submit up the manual extensions without an issue, mainly I don't want to cause an issue for you in terms of maintainability down the line when you do automate.

For one, as mentioned, if we want to do them all at a flat level right now, or if I should emulate the depth of the real commands. If there is a DevToolsClient (and maybe a browser.GetDevToolsClient() or similar) it would then be DevToolsClient.Network.setCookie. I would assume if hoping to move to an automated framework generation trying to have methods that stick to the official protocol and parameters is best (with the potential of additional overloads or similar to simplify things like with the cookie TimeSpan version). I am also not sure the best way to handle things like custom types. I could do the official signature that just takes a string, and then an overload with a custom enum like I did for CookieSameSite.

Some of the other ones I assume are decently useful could do extensions for

Network.clearBrowserCache
Network.clearBrowserCookies
Storage.clearDataForOrigin
Emulation.setUserAgentOverride
Emulation.setEmitTouchEventsForMouse
Emulation.setGeolocationOverride
Emulation.setTimezoneOverride
Emulation.setVisibleSize

[amaitland]

es essentially the commands I have done thus far only return false if the dev tools message can't be sent (as it returns 0 in this case. Obviously getting data out of dev tools is a plus.

I have a rough DevToolsClient class, which usage wise looks something like https://github.com/amaitland/CefSharp/blob/d535d4c04b096f9483dc7c51e4bc1a60cd10c0f3/CefSharp.Example/DevTools/DevToolsExtensions.cs#L36-L54

I can submit up the manual extensions without an issue, mainly I don't want to cause an issue for you in terms of maintainability down the line when you do automate.

Thanks 👍

it would then be DevToolsClient.Network.setCookie

I've seen this approach used and I think that makes sense, each class would have methods and events that match the DevTools protocol.

I am also not sure the best way to handle things like custom types. I could do the official signature that just takes a string, and then an overload with a custom enum like I did for CookieSameSite.

Not sure, let me have a think about it overnight and I'll get back to you.

[amaitland]

Looking at https://github.com/ChromeDevTools/devtools-protocol/blob/master/json/browser_protocol.json and I think parsing and generating at least all the enum should be pretty trivial, I'll hack something together. I'm thinking it might be better to just generate everything save double handling. I'll hack something together hopefully in the next few days.

[amaitland]

I have the start of a parser/generator at https://gist.github.com/amaitland/721cbf556b3c6cf53434c8ea2944b711

It's not pretty as I'm not totally fluent in Roslyn. I'll hopefully have it generating methods in the next few days.

[amaitland]

See amaitland@551f1aa for the first draft of the generated classes. It compiles and some of the simpler methods probably execute ok. Complex ones there's still some work to do.

TODO:

• Map complex params/enums
• Strongly typed return types (only return the generic DevToolsMethodResult at the moment).