Skip to content
This repository was archived by the owner on Mar 17, 2025. It is now read-only.

Add serial protocol #59

Merged
merged 39 commits into from
Apr 8, 2016
Merged
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
5c75177
Create serial_protocol.md
ed7coyne Feb 2, 2016
eff6087
Update serial_protocol.md
ed7coyne Feb 3, 2016
7bddea6
Update serial_protocol.md
ed7coyne Feb 3, 2016
81788aa
Update serial_protocol.md
ed7coyne Feb 3, 2016
04bdf6e
Update serial_protocol.md
ed7coyne Feb 3, 2016
8557658
Update serial_protocol.md
ed7coyne Feb 4, 2016
67f51e6
Update serial_protocol.md
ed7coyne Feb 4, 2016
e323a1c
Update serial_protocol.md
ed7coyne Feb 4, 2016
8a3a56c
Update serial_protocol.md
ed7coyne Feb 4, 2016
6150e54
Update serial_protocol.md
ed7coyne Feb 9, 2016
3d96c9c
Update serial_protocol.md
ed7coyne Feb 9, 2016
d554f5d
Update serial_protocol.md
ed7coyne Feb 9, 2016
246f2e8
Update serial_protocol.md
ed7coyne Feb 9, 2016
d3da4e9
Update serial_protocol.md
ed7coyne Feb 9, 2016
07c43db
Update serial_protocol.md
ed7coyne Feb 9, 2016
ebb181b
Update serial_protocol.md
ed7coyne Feb 9, 2016
51921e3
Update serial_protocol.md
ed7coyne Feb 9, 2016
1b318b3
Update serial_protocol.md
ed7coyne Feb 9, 2016
6297660
Update serial_protocol.md
ed7coyne Feb 9, 2016
ecea516
Update serial_protocol.md
ed7coyne Feb 9, 2016
6f30401
Update serial_protocol.md
ed7coyne Feb 9, 2016
8dac3f3
Update serial_protocol.md
ed7coyne Feb 9, 2016
b0d90d2
Added Serial client example.
ed7coyne Feb 9, 2016
724fde5
Merge branch 'master' of https://github.com/ed7coyne/firebase-arduino
ed7coyne Feb 9, 2016
1c422b9
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 9, 2016
fc6a0f4
Update FirebaseSerialClient.ino
ed7coyne Feb 9, 2016
9f146de
Added buttons and led logic
ed7coyne Feb 10, 2016
76c746c
merged
ed7coyne Feb 10, 2016
c39e371
Update serial_protocol.md
ed7coyne Feb 10, 2016
6ca12be
updated example for changes to protocol
ed7coyne Feb 10, 2016
c485ef7
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 10, 2016
6be74cd
Update serial_protocol.md
ed7coyne Feb 10, 2016
e8c9623
Update serial_protocol.md
ed7coyne Feb 10, 2016
d9bdee6
Move serial client to another branch
ed7coyne Feb 10, 2016
6530949
Merge branch 'Serial' of https://github.com/ed7coyne/firebase-arduino…
ed7coyne Feb 10, 2016
f004ae5
Update serial_protocol.md
ed7coyne Feb 16, 2016
bba8045
Update serial_protocol.md
ed7coyne Feb 16, 2016
8ced4fb
Update serial_protocol.md
ed7coyne Mar 25, 2016
e5570d6
switched variable marker from $ to %%
ed7coyne Apr 8, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 188 additions & 0 deletions serial_protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
#Protocol:
During the first use, or when the chiplet changes environments a “NETWORK” call is expected to initialize the wifi parameters.

Every time a serial connection is established we will expect a “BEGIN” call after which any subsequent calls can be made, until the connection is closed.

In the following examples we use %% to represent variables. For example %Host% represents your firebase host.

##Response Format Byte
All responses will be prefixed with one of the following bytes signifying the response type.
```
+ If response is ok and a raw string value.
$ If response is ok, raw string value will be json formatted and prefixed by the byte count and a new line.
: If response is ok and a number, this could be float or int.
? If response is ok and a boolean value.
- If response is an error
```
##NETWORK
Only needs to be called when the chiplet is in a new environment and needs to connect to a new network. This setting will be stored by the chiplet and persisted through power cycles.
###Usage
NETWORK %SSID%
NETWORK %SSID% %PASSWORD%
###Response
+CONNECTED - When connected to new network
-UNABLE_TO_CONNECT - When unable to connect
###Examples
>> NETWORK home-guest
<< +CONNECTED
>> NETWORK home-private MySecretPassword
<< +CONNECTED
>> NETWORK home-guest
<< -UNABLE_TO_CONNECT

##BEGIN
Must be called after creating a Serial connection, it can take either just a host for accessing public variables or you may also provide a secret for accessing protected variables in the database.
###Usage
BEGIN %Host%
BEGIN %Host% %Secret%
###Response
+OK - Accepted initialization parameters
###Examples
>> BEGIN https://samplechat.firebaseio-demo.com
<< +OK
>> BEGIN https://samplechat.firebaseio-demo.com nnz...sdf
<< +OK
##GET
Fetches the value at %PATH% and returns it on the serial line. If %PATH% points to a leaf node you will get the raw value back, if it points to an internal node you will get a JSON string with all children.
###Usage
GET %PATH%
###Response
%DATA_AT_PATH%
$%JSON_DATA_BYTE_COUNT% \r\n %JSON_DATA
###Examples
>>GET /user/aturing/first
<<+Alan
>>GET /user/aturing
<<$39
<<{ "first" : "Alan", "last" : "Turing" }

##GET{+,*,#,.,?,$}
Same as GET but will either return the value in the format specified (by the format byte) or return an error.
###Usage
GET+ %PATH%
GET: %PATH%
GET? %PATH%
GET$ %PATH%
###Response
%FORMATED_RESPONSE%
###Examples
>>GET? /user/aturing/was_human
<<?true
>>GET? /user/aturing/first
<<-ERROR_INCORRECT_FORMAT
##SET
Store the data provided at the path provided. This method should be used for simple strings and will assume the first newline is the end of the data.
###Usage
SET %PATH% %DATA%
###Response
+OK
-FAIL
###Examples
>>SET /user/aturing/first Alan
<<+OK
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@proppy
The REST api returns the value you just wrote. I see no value in this and think that we should behave differently and simply return OK. It just seems like extra traffic over the wire and an extra hassle for the client to drain it. Let me know if you disagree.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might matter for server side value like .sv which might get expanded after you POST: https://www.firebase.com/docs/rest/api/#section-server-values but we could simplify things.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, interesting wasn't aware of .sv. I am still in favor of just a simple response but don't feel strongly either way.

##SET$
Similar to SET above but used to write multiline strings or raw binary data. Data format is similar to the batch string ($) return type, we specify the $DATA_BYTE_COUNT on the same line as SET$ then a newline and all data. However which the batch string ($) return type returns data json escaped and quoted you may provide raw data and we will handle the escaping.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: we are using batch string here, and raw string before, maybe we should just settle on json string? (unless we still take care of the escaping)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to take care of escaping on the modem side. Make it as easy for the client as possible.


Receiver will wait until a timeout for client to send %DATA_BYTE_COUNT% worth of data before becoming responsive again.
###Usage
SET$ %PATH%
%DATA_BYTE_COUNT%
%DATA%
###Response
+OK
-FAIL
-FAIL_TIMEOUT
###Examples
>>SET /user/aturing/address
>>24
>>78 High Street,
>>Hampton
<<+OK

##SET{+,*,#,.,?}
Same as SET but will force the value to be stored in the given type or return an error if we cannot parse it as that type.
###Usage
SET+ %PATH% %VALUE%
SET: %PATH% %VALUE%
SET? %PATH% %VALUE%
###Response
+OK
-INCORRECT_TYPE
###Examples
>>SET? /user/aturing/was_human true
<<+OK
>>SET? /user/aturing/was_human He was not a computer.
<<-INCORRECT_TYPE

##REMOVE
Deletes the value located at the path provided.
###Usage
REMOVE %PATH%
###Response
+OK
-FAIL
###Examples
>>REMOVE /user/aturing
<<+OK

##PUSH
Adds a value to the list located at the path provided and returns the key at which the new object is located.
###Usage
PUSH %PATH% %DATA%
###Response
%KEY%
###Examples
>>PUSH /user/aturing/login_timestamps 1455052043
<<+-K94eLnB0rAAvfkh_WC2

##PUSH$
Similar to PUSH but used to write multiline strings or raw binary data. Data format is similar to the batch string ($) return type, we specify the $DATA_BYTE_COUNT on the same line as SET$ then a newline and all data. However you are not required to json escape all of your data, that will be handled for you.

Receiver will wait until a timeout for client to send $DATA_BYTE_COUNT worth of data before becoming responsive again.
###Usage
PUSH$ %PATH% %DATA_BYTE_COUNT%
%DATA%
###Response
%KEY%
###Examples
>>PUSH$ /user/aturing/quotes 91
>>We can only see a short distance ahead,
>>but we can see plenty there that needs to be done.
<<+-K94eLnB0rAAvfkh_WC3

##BEGIN_STREAM
Used to register to receive a stream of events that occur to the object at the provided path.

After registering you will start receiving events on the response line. They will be formatted as one line with the event type {PUT,PATCH,etc..} followed by the sub_path that changed and the other line with the data associated with that event type. This data will be formatted similar to GET results and can have multi-line batch strings (*) or json strings (&).

The event stream will continue until you send END_STREAM.

When there is an ongoing event stream no other commands can be processed until you call END_STREAM as the event stream owns the return line.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this note in here as well. I don't see a clean way around this, let me know if you disagree.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need different command for CANCEL_STREAM and END_STREAM?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: "no other commands can be processed", since our protocol is serial based the client will have to poll if Serial.available() Serial.readline() for stream result, maybe it wouldn't hurt to make that more explicit with a command? that way this could be multiplexed with other operation.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed CANCEL_STREAM to END_STREAM, I think they should be the same.

I am not certain what you mean by the second comment. What would you like to make explicit? Do you want to cache the stream on the modem and have the client call "NEXT_EVENT"? We may be able to do that but memory could be an issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't memory already an issue if we print something to Serial and nobody is reading on the other side?

We could have it read from the wifi connection only when we receive the NEXT_EVENT command?

I'm not sure which one is better, I was just thinking out loud.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the serial buffer fills up I think the serial write will just block until it is read. Also if the client can't do anything else with the connection they are less likely to leave it blocked than if they can just start a stream then ignore it while they do other things.

If we leave it in the wifi queue we are back to the state of not being able to do other actions while streaming. All other commands would have to return an error.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said we can address the issue directly and measure memory usage, if it exceeds a certain amount shutdown the stream and return an error when they call "NEXT_EVENT".

However if we can stream directly to the serial line than I think we can handle much larger/faster streams more efficiently. Maybe are back to adding more modes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we leave it in the wifi queue we are back to the state of not being able to do other actions while streaming. All other commands would have to return an error.

So practically, it would have to be a different "connection", since the stream endpoint is different from the API endpoint (remember the 2x redirect you can after calling /stream)

###Usage
BEGIN_STREAM %PATH%
###Response
%EVENT_NAME% %SUB_PATH%
%DATA%
+OK
###Examples
>>BEGIN_STREAM /user/aturing
<<+PUT /last_login
<<:1455052043
<<+PUT /address
<<$24
<<"78 High Street,\r\nHampton"

##END_STREAM
Used to stop listening to events at a given path. This must be the same path provided to a previous BEGIN_STREAM call.

###Usage
END_STREAM %PATH%
###Response
+OK
-NOT_STREAMING_PATH
###Examples
>>END_STREAM /user/aturing
<<-NOT_STREAMING_PATH
>>BEGIN_STREAM /user/aturing
>>END_STREAM /user/aturing
<<+OK