serial transparent link with websocket #61
Comments
|
do your printer has a binary protocol like repetier host for example then you may need to transfare the data with |
|
Hi @Links2004, l am sorry to jump into discussion but I think it better to ask here than esp8266/Arduino#1769 as @mkeyno request is more related to your library and than ESP core itself. @mkeyno I understand the need of security using SSL and Basic auth, but I do not get how repetier host will handle the websocket authentication, as of today it does not do as far I know, repetier host does not have any authentication between host and printer, when repetier host server has authentication but only for front end, not for communication with printer itself. @Links2004 am I wrong ? Sorry if my question looks stupid but I miss something I think |
|
hi @luc-github yes you are right , we don't have any secure link between repetier host server and CNC contrl board and that's why we need put the both feature in ESP module , websocket is good one whereas we could host it in the module flash, also we have enough room to put the @openhardwarecoza in it https://github.com/openhardwarecoza/LaserWeb |
|
Sounds awesome! LaserWeb2 doesn't need the nodejs server anymore, right
|
|
thanks @openhardwarecoza (peter) to join , yes I think we need this link to enhanced your excellent web interface, local serial connection is like stick us to ground |
|
Did you see my Smoothieboard based work with onboard ESP? (; Keep following me on Google Plus for my most interesting posts though, I hate blogging (; |
|
@mkeyno sorry again I try to understand your idea, so to sum up your idea, you mean put host software on ESP FW and not using repetier host like application ? In this case what is the advantage of websocket vs web authentication+https on esp web server ? Because what you describe is everything but transparent bridge, so I am confused |
|
hi @luc-github , I think when we use the websocket , we dont need be worry for unexpected other client request and keep the connection alive during the printing just like paring the Bluetooth, and also we could put light webpage such as @openhardwarecoza LaserWeb2 in ESP so need no more third party software such as repetier host, all in one |
|
hi @openhardwarecoza , I knew about your board but I think the Due shield is more flexible to make any prototype , and I really appreciated if you could my cnc shield and let me have your comments |
|
@mkeyno so if no repetier host, then websocket communicate with what ? what is for ? |
|
@luc-github - I am jumping into someone elses conversation but what I have a need for is a websocket->serial bridge Where the application (and for that matter what the application is) is irrelevant for the most part, it could be a web page like Chilipeppr, or a standalone app like LaserWeb. Adding websocket support allows us to reach the serial bridge, without any "agent" software on the machine. (ie no serialport-json-server or node-serialport needed) So in my example, where I have http://openhardwarecoza.github.io/LaserWeb2 -> instead of connecting to SPJS which then needs a Pi or PC to bridge to Serial, I could have a ESP with a websocket server, that I can connect to and do a websocket send to: https://github.com/openhardwarecoza/LaserWeb2/blob/gh-pages/js/spjs.js |
|
Websocket of course is a bidirectional method (unlike having to use ajax and poll status updates etc) The ESP could send out any serial to the websocket, as well as listen for command... |
|
The webpage hosted from the ESP could include same JS and speak back to the ESP over WS - or external web apps. |
|
thanks @openhardwarecoza Peter for explain the idea , |
|
@openhardwarecoza yes fully understand - your application use websocket to communicate to ESP/printer, that is clear I also planned to integrated websocket support as alternative of pure TCP connection in my project (https://github.com/luc-github/ESP8266) thanks to the great library of Markus, but I wanted to understand other people needs first, I do not use laser cnc yet (I plan to build one soon) but I guess needs are same as 3D printers. |
|
some notes to the websockets server and ssl. i working on project which does tunnel serial to the browser (remote debugging / router conf) |
|
thanks @Links2004 , so you say I have to wait till tunnel serial to the browser be ready ?regards to other notes and codes of @openhardwarecoza and @luc-github is it good option to choose another alternative? |
|
you can do it in you own way if you like, WebSocketsServer webSocket = WebSocketsServer(81);
#define SEND_SERIAL_TIME (50)
class SerialTerminal {
public:
void setup() {
_lastRX = 0;
resetBuffer();
Serial.begin(921600);
}
void loop() {
unsigned long t = millis();
bool forceSend = false;
size_t len = (_bufferWritePtr - &_buffer[0]);
int free = (sizeof(_buffer) - len);
int available = Serial.available();
if(available > 0 && free > 0) {
int readBytes = available;
if(readBytes > free) {
readBytes = free;
}
readBytes = Serial.readBytes(_bufferWritePtr, readBytes);
_bufferWritePtr += readBytes;
_lastRX = t;
}
// check for data in buffer
len = (_bufferWritePtr - &_buffer[0]);
if(len >= sizeof(_buffer)) {
forceSend = true;
}
if(len > (WEBSOCKETS_MAX_HEADER_SIZE + 1)) {
if(((t - _lastRX) > SEND_SERIAL_TIME) || forceSend) {
//Serial1.printf("broadcastBIN forceSend: %d\n", forceSend);
webSocket.broadcastBIN(&_buffer[0], (len - WEBSOCKETS_MAX_HEADER_SIZE), true);
resetBuffer();
}
}
}
protected:
uint8_t _buffer[1460];
uint8_t * _bufferWritePtr;
unsigned long _lastRX;
void resetBuffer() {
// offset for adding Websocket header
_bufferWritePtr = &_buffer[WEBSOCKETS_MAX_HEADER_SIZE];
addChar('T');
}
inline void addChar(char c) {
*_bufferWritePtr = (uint8_t) c; // message type for Webinterface
_bufferWritePtr++;
}
};
SerialTerminal term;
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t lenght) {
switch(type) {
case WStype_DISCONNECTED:
Serial1.printf("[%u] Disconnected!\n", num);
break;
case WStype_CONNECTED: {
IPAddress ip = webSocket.remoteIP(num);
Serial1.printf("[%u] Connected from %d.%d.%d.%d url: %s\n", num, ip[0], ip[1], ip[2], ip[3], payload);
// send message to client
webSocket.sendTXT(num, "Connected");
}
break;
case WStype_TEXT:
Serial1.printf("[%u] get Text: %s\n", num, payload);
break;
case WStype_BIN:
if(lenght > 0) {
if(payload[0] == 'T') {
if(lenght > 1) {
Serial.write((const char *) (payload + 1), (lenght - 1));
// Serial1.write((const char *) (payload + 1), (lenght - 1));
}
} else {
Serial1.printf("[%u] get binary lenght: %u\n", num, lenght);
hexdump(payload, lenght);
}
}
break;
}
}
void setup()
{
// use Serial 1 for debug out
Serial1.begin(921600);
Serial1.setDebugOutput(true);
Serial1.println();
Serial1.println();
Serial1.println();
for(uint8_t t = 4; t > 0; t--) {
Serial1.printf("[SETUP] BOOT WAIT %d...\n", t);
delay(1000);
}
Serial1.printf("[SETUP] HEAP: %d\n", ESP.getFreeHeap());
webSocket.begin();
webSocket.onEvent(webSocketEvent);
WiFi.mode(WIFI_AP_STA);
WiFi.softAP("ESP_SLDT", "yesSecureHere");
WiFiMulti.addAP("wifi", "yesSecureHere");
term.setup();
// disable WiFi sleep for more performance
WiFi.setSleepMode(WIFI_NONE_SLEEP);
}
void loop()
{
term.loop();
webSocket.loop();
if(WiFiMulti.run() == WL_CONNECTED) {
}
} |
|
hi @openhardwarecoza peter , have you check new codes of Markus on serial bridge |
|
Sorry no, i have been stuck in hardware mode his week.. (: Designed some
|
|
pick & place !! wow you are so cool buddy I also have plan to make small pick & place machine such as TVM802A |
|
Anyone want to get paid to code up a websocket to serial bridge? https://plus.google.com/+PetervanderWalt/posts/BkzJmtaanGc |
|
websocket to serial code is existing: AP config code is existing too: upload to SD card via serial is possible, no need for dealing with it on the ESP8266 part. with some hardware hacking its may possible to share the SD card between ESP and Arduino in exclusive way, best i have seen for write to SD from ESP is ~140KB/s. |
ghost
mentioned this issue
|
@Links2004 ok I dug into this today
Digging a little deeper, it seems that the message arrives at type TEXT |
|
Okay, got it mostly working: /*
* WebSocketServer.ino
*
* Created on: 22.05.2015
*
*/
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <WebSocketsServer.h>
#include <Hash.h>
const char* ssid = "openhardwarecoza";
const char* password = "aabbccddeeff";
WebSocketsServer webSocket = WebSocketsServer(80);
#define SEND_SERIAL_TIME (50)
class SerialTerminal {
public:
void setup() {
_lastRX = 0;
resetBuffer();
Serial.begin(115200);
}
void loop() {
unsigned long t = millis();
bool forceSend = false;
size_t len = (_bufferWritePtr - &_buffer[0]);
int free = (sizeof(_buffer) - len);
int available = Serial.available();
if(available > 0 && free > 0) {
int readBytes = available;
if(readBytes > free) {
readBytes = free;
}
readBytes = Serial.readBytes(_bufferWritePtr, readBytes);
_bufferWritePtr += readBytes;
_lastRX = t;
}
// check for data in buffer
len = (_bufferWritePtr - &_buffer[0]);
if(len >= sizeof(_buffer)) {
forceSend = true;
}
if(len > (WEBSOCKETS_MAX_HEADER_SIZE + 1)) {
if(((t - _lastRX) > SEND_SERIAL_TIME) || forceSend) {
//Serial1.printf("broadcastBIN forceSend: %d\n", forceSend);
webSocket.broadcastBIN(&_buffer[0], (len - WEBSOCKETS_MAX_HEADER_SIZE), true);
resetBuffer();
}
}
}
protected:
uint8_t _buffer[1460];
uint8_t * _bufferWritePtr;
unsigned long _lastRX;
void resetBuffer() {
// offset for adding Websocket header
_bufferWritePtr = &_buffer[WEBSOCKETS_MAX_HEADER_SIZE];
// addChar('T');
}
inline void addChar(char c) {
*_bufferWritePtr = (uint8_t) c; // message type for Webinterface
_bufferWritePtr++;
}
};
SerialTerminal term;
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t lenght) {
switch(type) {
case WStype_DISCONNECTED:
Serial1.printf("[%u] Disconnected!\n", num);
break;
case WStype_CONNECTED: {
IPAddress ip = webSocket.remoteIP(num);
Serial1.printf("[%u] Connected from %d.%d.%d.%d url: %s\n", num, ip[0], ip[1], ip[2], ip[3], payload);
// send message to client
webSocket.sendTXT(num, "Connected");
}
break;
case WStype_TEXT:
Serial1.printf("[%u] get Text: %s\n", num, payload);
if(lenght > 0) {
Serial.write((const char *) (payload), (lenght));
}
Serial.print('\n');
break;
}
}
void setup()
{
// use Serial 1 for debug out
Serial1.begin(921600);
Serial1.setDebugOutput(true);
Serial1.println();
Serial1.println();
Serial1.println();
for(uint8_t t = 4; t > 0; t--) {
Serial1.printf("[SETUP] BOOT WAIT %d...\n", t);
delay(1000);
}
Serial1.printf("[SETUP] HEAP: %d\n", ESP.getFreeHeap());
webSocket.begin();
webSocket.onEvent(webSocketEvent);
WiFi.mode(WIFI_AP_STA);
WiFi.begin(ssid, password);
WiFi.softAP("ESP_SLDT", "yesSecureHere");
term.setup();
// disable WiFi sleep for more performance
WiFi.setSleepMode(WIFI_NONE_SLEEP);
}
void loop()
{
term.loop();
webSocket.loop();
} |
|
I use binary to support Repetier protocol too (which is binary). sending binary example: |
|
Even though using the arraybuffer send as I have in it at the moment, sort of works, I think a ascii implementation would be a little more solid since we have no binary data. Do you have a ASCII example version? ws.onmessage = function(e){
console.log(e.data)
var data = "";
if(e.data instanceof ArrayBuffer){
var bytes = new Uint8Array(e.data);
for (var i = 0; i < bytes.length; i++) {
data += String.fromCharCode(bytes[i]);
}
} else {
data = e.data; // Suspect might be more solid
}I'd like to receive the data emitted by the serialport, such that I dont need the reassembly of the arraybuffer (I am having some glitches in replies coming back from the controller... ) I suspect this function in your sketch from above is also optimised for Repetier's binary protocol? void loop() {
unsigned long t = millis();
bool forceSend = false;
size_t len = (_bufferWritePtr - &_buffer[0]);
int free = (sizeof(_buffer) - len);
int available = Serial.available();
if(available > 0 && free > 0) {
int readBytes = available;
if(readBytes > free) {
readBytes = free;
}
readBytes = Serial.readBytes(_bufferWritePtr, readBytes);
_bufferWritePtr += readBytes;
_lastRX = t;
}
// check for data in buffer
len = (_bufferWritePtr - &_buffer[0]);
if(len >= sizeof(_buffer)) {
forceSend = true;
}
if(len > (WEBSOCKETS_MAX_HEADER_SIZE + 1)) {
if(((t - _lastRX) > SEND_SERIAL_TIME) || forceSend) {
//Serial1.printf("broadcastBIN forceSend: %d\n", forceSend);
webSocket.broadcastBIN(&_buffer[0], (len - WEBSOCKETS_MAX_HEADER_SIZE), true);
resetBuffer();
}
}
}Do you have a ASCII example version? |
|
change Note: value to low --> to many TCP packages (WS Messages) --> IP Stack overload --> data lost --> print fail the function is written to handle any kind of data including binary, have used the same code to debug a router remotely ;) |
|
Alright! Thanks! Much better already In my nodejs based serial to websocket bridge, we get a websocket message for each line from the controller. Acting on these events on the JS side makes things a lot easier With this sketch, I get a couple lines at a time as a response back in one websocket message |
|
you can simple force a send when "\n" is found. for(size_t i = WEBSOCKETS_MAX_HEADER_SIZE; i < len; i++) {
if(_buffer[i] == '\n') {
forceSend = true;
break;
}
}after if(len >= sizeof(_buffer)) {
forceSend = true;
}but doing the reassembly on the javascript side is a better option from my point of view. pseudocode: var buffer = "";
event.on('data', function(new_text) {
buffer += new_text;
var split = buffer.split("\n");
buffer = split.pop(); //last not fin data back to buffer
for (var line in split) {
console.log(line); // trigger line handling event here
}
}); |
|
Alright!!! Now we are solid, and optimised! (: LaserWeb/deprecated-LaserWeb3@ebd13f0 I really really owe you a couple beers! @mkeyno - guess we can close this one now? I'll make a pull request for the total sketch so we can include it in this Library's examples (: |
|
good job @openhardwarecoza but as you aware most developer use async approach to make robust connection line, I've used @hallard async bridge for my project , even Luc has revised all his code by async sever, after I've changed it to async approach my websocket service never broke again |
|
@mkeyno the websocket lib can run sync and async ;) |
|
I spent a couple days now trying to get this working 100% solidly: I still sometimes get Note that o2 response - looks like something gets mangled Here's my host https://github.com/openhardwarecoza/CoPrint Specifically - here's my websocket implementationhttps://github.com/openhardwarecoza/CoPrint/blob/gh-pages/js/esp8266.js and here's my Arduino sketch for the ESP8266 https://github.com/LaserWeb/LaserWeb3/blob/master/ESP8266%20Wifi%20Bridge/websocketserver.ino/websocketserver.ino.ino |
|
Print via ESP8266 Print via USB |
|
@raykholo @DarklyLabs - just FYI above, still a little instability on the websocket side |
|
after the TCP layer there is no Possibility for bit corruption, have you a scope or logic analyzer to check if the baud rates are matching exact? may there is some EMC problem, to get the print more stable I recommend to use the |
|
Hi @Links2004, thanks for your explanations and codes! I'm trying to do a simple validation of the content (string) of & _buffer [0] in the following line: webSocket.broadcastTXT(&_buffer[0], (len - WEBSOCKETS_MAX_HEADER_SIZE), true); However, I have no solid knowledge of c / c ++. I tried something like:
But I did not succeed :( Can you help me? Thank you |
|
the code use the first bytes to store the webSocket header (bin data) you can try: when you serial data is ascii it will work fine. |
hi @markus , I'm working on the 3d printer board which its serial pins connected to serial pin of ESP module. , in the module couple of html page store in its flash , these file send g-code lines to printer, my plan is to lunch the secure websocket link to printer and send command by its web interface , however I could send the code to printer but its seems the websocket cant handle the respond message of the printer , I find couple of efforts to make wifi-serial bridge but the code in written in lua and I not love lua as much as love @igrr SDK code so any help really appreciated
The text was updated successfully, but these errors were encountered: