Skip to content

Fix SEGV when using skip lines #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 15 commits into from
Closed

Fix SEGV when using skip lines #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
0af9aa5
Fix segv
bdrosen96 Sep 29, 2016
569acbc
second fix
bdrosen96 Sep 29, 2016
52754b5
more fixes
bdrosen96 Sep 29, 2016
443aa40
tracing and some fixes
bdrosen96 Sep 29, 2016
b90a7c0
comment more verbose parts
bdrosen96 Sep 29, 2016
79382c5
less verbose
bdrosen96 Sep 29, 2016
e2245fa
less verbose
bdrosen96 Sep 29, 2016
b03e27b
slight changes
bdrosen96 Sep 29, 2016
728cc6e
fix macro
bdrosen96 Sep 29, 2016
93bf296
fix bug
bdrosen96 Sep 29, 2016
eca4b35
issue with grow_buffer
bdrosen96 Sep 29, 2016
29ea1d3
backport skip line improvements
bdrosen96 Sep 30, 2016
5baf897
remove troubleshooting code
bdrosen96 Sep 30, 2016
b022a0e
more cleanup
bdrosen96 Sep 30, 2016
ed47cfe
final cleanup
bdrosen96 Sep 30, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions pandas/parser.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,7 @@ cdef class TextReader:
cdef:
size_t rows_read = 0
chunks = []
int status

if rows is None:
while True:
Expand All @@ -747,7 +748,9 @@ cdef class TextReader:
else:
chunks.append(chunk)

parser_trim_buffers(self.parser)
status = parser_trim_buffers(self.parser)
if status < 0:
raise_parser_error('Error trimming data', self.parser)

if len(chunks) == 0:
raise StopIteration
Expand Down Expand Up @@ -812,7 +815,10 @@ cdef class TextReader:
# trim
parser_consume_rows(self.parser, rows_read)
if trim:
parser_trim_buffers(self.parser)
status = parser_trim_buffers(self.parser)

if status < 0:
raise_parser_error('Error trimming data', self.parser)
self.parser_start -= rows_read

self._end_clock('Parser memory cleanup')
Expand Down
Loading