Skip to content
This repository was archived by the owner on Mar 17, 2025. It is now read-only.

Add serial protocol #59

Merged
merged 39 commits into from
Apr 8, 2016
Merged
Changes from 9 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
5c75177
Create serial_protocol.md
ed7coyne Feb 2, 2016
eff6087
Update serial_protocol.md
ed7coyne Feb 3, 2016
7bddea6
Update serial_protocol.md
ed7coyne Feb 3, 2016
81788aa
Update serial_protocol.md
ed7coyne Feb 3, 2016
04bdf6e
Update serial_protocol.md
ed7coyne Feb 3, 2016
8557658
Update serial_protocol.md
ed7coyne Feb 4, 2016
67f51e6
Update serial_protocol.md
ed7coyne Feb 4, 2016
e323a1c
Update serial_protocol.md
ed7coyne Feb 4, 2016
8a3a56c
Update serial_protocol.md
ed7coyne Feb 4, 2016
6150e54
Update serial_protocol.md
ed7coyne Feb 9, 2016
3d96c9c
Update serial_protocol.md
ed7coyne Feb 9, 2016
d554f5d
Update serial_protocol.md
ed7coyne Feb 9, 2016
246f2e8
Update serial_protocol.md
ed7coyne Feb 9, 2016
d3da4e9
Update serial_protocol.md
ed7coyne Feb 9, 2016
07c43db
Update serial_protocol.md
ed7coyne Feb 9, 2016
ebb181b
Update serial_protocol.md
ed7coyne Feb 9, 2016
51921e3
Update serial_protocol.md
ed7coyne Feb 9, 2016
1b318b3
Update serial_protocol.md
ed7coyne Feb 9, 2016
6297660
Update serial_protocol.md
ed7coyne Feb 9, 2016
ecea516
Update serial_protocol.md
ed7coyne Feb 9, 2016
6f30401
Update serial_protocol.md
ed7coyne Feb 9, 2016
8dac3f3
Update serial_protocol.md
ed7coyne Feb 9, 2016
b0d90d2
Added Serial client example.
ed7coyne Feb 9, 2016
724fde5
Merge branch 'master' of https://github.com/ed7coyne/firebase-arduino
ed7coyne Feb 9, 2016
1c422b9
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 9, 2016
fc6a0f4
Update FirebaseSerialClient.ino
ed7coyne Feb 9, 2016
9f146de
Added buttons and led logic
ed7coyne Feb 10, 2016
76c746c
merged
ed7coyne Feb 10, 2016
c39e371
Update serial_protocol.md
ed7coyne Feb 10, 2016
6ca12be
updated example for changes to protocol
ed7coyne Feb 10, 2016
c485ef7
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 10, 2016
6be74cd
Update serial_protocol.md
ed7coyne Feb 10, 2016
e8c9623
Update serial_protocol.md
ed7coyne Feb 10, 2016
d9bdee6
Move serial client to another branch
ed7coyne Feb 10, 2016
6530949
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 10, 2016
f004ae5
Update serial_protocol.md
ed7coyne Feb 16, 2016
bba8045
Update serial_protocol.md
ed7coyne Feb 16, 2016
8ced4fb
Update serial_protocol.md
ed7coyne Mar 25, 2016
e5570d6
switched variable marker from $ to %%
ed7coyne Apr 8, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions serial_protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#Protocol:
During the first use, or when the chiplet changes environments a “NETWORK” call is expected to initialize the wifi parameters.

Every time a serial connection is established we will expect a “INIT” call after which any subsequent calls can be made, until the connection is closed.

In the following examples we use $ to represent variables. For example $Host represents your firebase host.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: since $ is already used in the protocol, maybe we should using a different delimeter for placeholder like %VALUE%, [VALUE] or <VALUE>


##Response Format Byte
All responses will be prefixed with one of the following bytes signifying the response type.
```
+ If response is ok and a raw string value.
* If response is ok and a raw string value prefixed by count of bytes in response then new line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can merge this case with $jsonvalue

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the more detail about the value we can easily provide the better.

Like if you are reading a string from https:/.../client/value. Then someone writes a https:/.../client/value/child then your response will change from a raw string to a json tree and you should change your parsing to match. In that case it would be nice if the response type changed so the issue would be more obvious to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was more asking if you see a usecases for having size-prefixed raw string value that are not json strings.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see so you are more asking if we can merge this with "+", i.e. only return raw strings as "simple strings".

This goes back to our discussions elsewhere of the value of sending a size at all. The big usecases are binary data, which would be transferred as a string and large blobs of text that you may need to prepare storage or other devices for.

Firebase has a 10mb limit on cells at that size on a MCU you need to do some prep work to handle it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see so you are more asking if we can merge this with "+"

Not really, all I'm saying is that when we return large size prefixed string, we might as well return them as "quoted" json string, therefor making the protocol simpler: small value are prefixed +, large value are size prefixed, json encoded and prefixed with '$'. Or did you want to be able to return large size prefixed raw string without json escaping?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I got ya. Sorry in my mind json encoded meant key-value and in {} brackets. Keep forgetting that raw values can be json encoded as well.
I am fine with that.

# If response is ok and a integer value.
. If response is ok and a float value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json doesn't have a difference between float and int: so maybe just :number

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json doesn't but I think there is a lot of value in providing the distinction for c++ parsing. So you can do a atoi() call with confidence knowing that the string will parse. I think we can determine pretty easily in the library (check for a '.').

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be down for always return float, so that we can always use atofon the receiving size.

We never know if something is intended to be stored as an int, or as a float on the firebase side: it might just be that the current intermediate value is a round number. It seems easy enough for the client to cast it back to an int when needed (or maybe we could expose a typed GET that always return int).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see where that could be a problem if the value changes from float to int due to chance. Alright I am onboard with leaving the ambiguity up to the user to resolve. Just label them a "number", the user will probably know with domain context what to expect.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that's why I think we might just need : for number and just return what we get from firebase.

$ If response is ok and a boolean value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think of ? for bool?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, done.

& If response is ok and json formatted and prefixed by count of bytes in response then new line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use $ here to match redis bulk string, and add a case for "long quoted strings" which would also be valid value.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with swapping symbols, not really attached to any of them.

The "long quoted string" would be *.

- If response is an error
```
##NETWORK
Only needs to be called when the chiplet is in a new environment and needs to connect to a new network. This setting will be stored by the chiplet and persisted through power cycles.
###Usage
NETWORK $SSID
NETWORK $SSID $PASSWORD
###Response
CONNECTED - When connected to new network
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be +CONNECTED :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on nvm, this is then correct in the example.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

UNABLE_TO_CONNECT - When unable to connect
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-UNABLE_TO_CONNECT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

###Examples
>> NETWORK home-guest
<< +CONNECTED
>> NETWORK home-private MySecretPassword
<< +CONNECTED
>> NETWORK home-guest
<< -UNABLE_TO_CONNECT

##INIT
Must be called after creating a Serial connection, it can take either just a host for accessing public variables or you may also provide a secret for accessing protected variables in the database.
###Usage
INIT $Host
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could have that be FIREBASE $HOST $SECRET

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not sure about that one, INIT implies that it needs to be called everytime to initialize the connection.

Maybe CONNECT $HOST $SECRET?

Just It seems these should all be commands to the system and phrased as such where FIREBASE isn't really a command.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that if the CHIPLET was programmed with multiple APIs, it might expose similar commands derived from REST verbs over different APIs (hence the explicit 'FIREBASE').

But that may be a premature discussion, what about 'BEGIN' to make it Arduinoish?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah got ya. Hadn't considered the other APIs. BEGIN works too, we can do BEGIN_FIREBASE if we want to leave the option open for other APIs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do only begin for now ;) I agree we should worry about multiple API when we have more than 1 binding ;) sorry for the disgression.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then what about:

API firebase version
BEGIN someurl

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really sold on adding an extra call. I think the fewer calls you need to make to get the env setup to start communicating the better.

I guess for versions I would just change the begin call. i.e. create a BEGIN_FIREBASE2 if we ever need to implement a breaking change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I liked about the API prelude is that it makes it feels declarative. Like an #include or an import. Later we might add some extra calls that allow you to manage the lifecycle of the APIs loaded in your chiplet: list version, OTA, etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I am on board with something like that. We can have a concept of "default api" that will load at startup.

So for now we can just leave this out all together and address it later on when we introduce other APIs or versions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I am on board with something like that. We can have a concept of "default api" that will load at startup.

Yes, so for now I think it's fine to just have BEGIN url

INIT $Host $Secret
###Response
OK - Accepted initialization parameters
###Examples
>> INIT https://samplechat.firebaseio-demo.com
<< +OK
>> INIT https://samplechat.firebaseio-demo.com nnz...sdf
<< +OK
##GET
Fetches the value at $Path and returns it on the serial line. If $PATH points to a leaf node you will get the raw value back, if it points to an internal node you will get a JSON string with all children.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Path/PATH` to match the "Usage"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

###Usage
GET $PATH
###Response
$DATA_AT_PATH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be +DATA_AT_PATH? maybe add a separator between the two.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the confusing part of using "$" for bulk strings. I am using it in all of the documentation to represent a variable (ala bash). So this should stay variable "DATA_AT_PATH" because we cannot say what type it will be.

$JSON_DATA_BYTE_COUNT \n\r $JSON_DATA
###Examples
>>GET /user/aturing/first
<<+Alan
>>GET /user/aturing
<<&39
<<{ "first" : "Alan", "last" : "Turing" }

##GET_BULK
Same as GET but returns value with size prefix. useful when value is expected to be large so you can know the size before accepting value.
Also only returns values at leaf nodes, if called on internal node returns error.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that the bulk MODE would return the data always as it is coming from fbase (json escaped), otherwise it's a strict subset of GET behavior.

Maybe we need separate flag/mode for json/size/type and keep the default very simple and readable (leaf value, un escaped, no prefix), wdyt?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with just dropping the bulk mode for now. We can add it back if we see a need for it in practice.
I do think the return from GET needs a prefix though, to allow reliable testing to see if it is an error or valid response before you try to parse it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love if we added all of those as optional stateful flags you could turn on and off so that we can experiment with the wire protocol and see how it feels without adding all the different combination as separate commands.

Then we can debate which one is the default, but everything you listed sounds nice to have:

  • type prefix
  • size prefix
  • decoded leaf value
  • disabling non leaf value

###Usage
GET_BULK $PATH
###Response
$DATA_BYTE_COUNT $DATA_AT_PATH
###Examples
>>GET_BULK /user/aturing/first
<<*4
<<Alan
>>GET /user/aturing
<<$39
<<{ "first" : "Alan", "last" : "Turing" }
##Set
##Push
##Remove
##Stream