Skip to content

zero-copy buffer #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 24, 2013
Merged

zero-copy buffer #55

merged 6 commits into from
Apr 24, 2013

Conversation

julienschmidt
Copy link
Member

Continued from #52

This approach removes the 2nd buffer (date / packet buffer). Instead we work with slices directly from the buffer. This results in more or less zero copy.

Performance:
Before


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 1]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.1241215s [ 4708 queries/second ]
PreparedQuery: 1.854106s [ 5393 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 13.3857656s [ 7471 queries/second ]
SimpleQueryRow: 7.2264133s [ 13838 queries/second ]
PreparedQueryRow: 6.8483917s [ 14602 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 3.6992116s [ 27033 queries/second ]
PreparedExec: 3.4701985s [ 28817 queries/second ]


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 2]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 2.3471342s [ 4261 queries/second ]
PreparedQuery: 1.871107s [ 5344 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 13.5567754s [ 7376 queries/second ]
SimpleQueryRow: 7.3384198s [ 13627 queries/second ]
PreparedQueryRow: 6.7943886s [ 14718 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 3.6942113s [ 27069 queries/second ]
PreparedExec: 3.5642039s [ 28057 queries/second ]

After

*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 1]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 1.9131094s [ 5227 queries/second ]
PreparedQuery: 1.555089s [ 6431 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 13.1377515s [ 7612 queries/second ]
SimpleQueryRow: 6.8833938s [ 14528 queries/second ]
PreparedQueryRow: 6.4733703s [ 15448 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 3.6292076s [ 27554 queries/second ]
PreparedExec: 3.5092007s [ 28497 queries/second ]


*************************************************************
   BENCHMARKING Go-MySQL-Driver [run 2]
*************************************************************

-------------------------------------------------------------
   [10000 * Query 100 Rows]
-------------------------------------------------------------
SimpleQuery: 1.8621065s [ 5370 queries/second ]
PreparedQuery: 1.5790903s [ 6333 queries/second ]

-------------------------------------------------------------
   [100 * QueryRow] * 1000
-------------------------------------------------------------
AutoQueryRow: 13.2367571s [ 7555 queries/second ]
SimpleQueryRow: 6.9683986s [ 14350 queries/second ]
PreparedQueryRow: 6.4303678s [ 15551 queries/second ]

-------------------------------------------------------------
   [100000 * Exec]
-------------------------------------------------------------
SimpleExec: 3.6432084s [ 27448 queries/second ]
PreparedExec: 3.4141953s [ 29289 queries/second ]

I worked on such a "zero copy buffer" before but stopped when I found a reason why this concept couldn't work - at least I assumed it the be a reason.
Until now I found no case where the new buffer causes problems, but please test it as much as you can.
Especially cases where sql.RawBytes is used excessively could be interesting.

if err != nil {
errLog.Print(err.Error())
return nil, driver.ErrBadConn
}

// Packet Length [24 bit]
pktLen := uint32(data[0]) | uint32(data[1])<<8 | uint32(data[2])<<16
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leave as is, cast to int in L54 readNext?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the benefit of that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly - none. Just one change less (and I dislike parens around long expressions)

@ghost ghost assigned julienschmidt Apr 24, 2013
@arnehormann
Copy link
Member

LGTM

julienschmidt added a commit that referenced this pull request Apr 24, 2013
@julienschmidt julienschmidt merged commit 1caf647 into master Apr 24, 2013
@julienschmidt julienschmidt deleted the zero-copy branch April 24, 2013 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants