Reuse buffers when parsing #52
Conversation
Conflicts: packets.go
This commit does not reuse bytes.Buffer, but merely lays the groundwork for reusing buffers in the future.
Adding the bytes.Buffer version bloated the code slightly. I've added a helper method that replicates the old functionality, so that changes are more miniminal.
ghost
assigned 
|
Unfortunately, I notice performance degradation of ~ 8-9% (avg.) in Go 1.1beta2 and ~10% in Go 1.0.3. |
|
What about keeping the buffer and still killing the unneccessary allocations? |
No extra allocation when refilling the buffer Closes #52
|
I tried another approach: completely remove the 2nd buffer (date / packet buffer). Instead we work with slices directly from the buffer. This results in more or less zero copy. Performance: After |
|
I worked on such a "zero copy buffer" before but stopped when I found a reason why this concept couldn't work - at least I assumed it the be a reason. |
|
@julienschmidt Nice! Feel free to close this in favor of that one. I was just reading over the code again, and realized why this was slow (and wrong, to boot!) I was resetting the buffer per-column, rather than per result row. This would have had some bad aliasing effects 👎, and negated the perf benefits. I think you'd have to store the buffer inside the conn, and reuse it every time columns were being parsed. I'm going to prototype a stream-based parser and serializer that uses that idea; I think there are more gains to be had in this direction. Will keep you informed. Cheers. |
|
Thanks for all you efforts! Continued at #55 |
We try to reduce the number of allocations that we do in the common case by keeping our buffers around, and reusing them, if possible. Note that this requires us to be slightly careful about handing out byte slices, because the backing buffer might be reused. In most cases, we don't have to care, because the database/sql package will copy for us.
This already has #51 merged into it, so it is recommended to review that first.