BUG: cells are missing in the excel file when exporting excel using xlsxwriter with option constant_memory set to True (#15392) #27624
Conversation
…t_memory` set to `True`.
…arn users. merged cells of index label will forbid writing cells row by row which is the prerequsite of `xlsxwriter` when writing excel cells with constant memory.
…aframe row by row
…a dataframe row by row
KenTsui
changed the title
xlsxwriter with option constant_memory set to True (#15392)
pandas/core/generic.py
Outdated
| @@ -2230,6 +2230,15 @@ def to_excel( | |||
| ): | |||
| df = self if isinstance(self, ABCDataFrame) else self.to_frame() | |||
|
|
|||
| from pandas.io.excel._xlsxwriter import _XlsxWriter | |||
There was a problem hiding this comment.
this is not the appropriate place for this, it needs to be in _XlsxWriter
There was a problem hiding this comment.
Need to disable merge_cell based on 1) if it is using _XlsxWriter and 2) if it has set constant_memory True. In _XlsxWriter, you cannot control merge_cell. I will try to move it this piece of logic to pandas\io\formats\excel.py if you agree.
There was a problem hiding this comment.
I think placing it in excel.py is a better place than generic.py for starters. We can then evaluate feasibility of moving _XlsxWriter if necessary, though your points are well-taken.
There was a problem hiding this comment.
Now the testing if using _XlsxWriter and constant_memory==True is moved to excel.py.
And by referencing the solution of an issue of xlsxwriter, there is a perk in the updated code that we can keep merged cells even if the two aforementioned conditions are present.
gfyoung
added
Error Reporting
…mory=True Reference to: jmcnamara/XlsxWriter#292
| @@ -106,7 +106,7 @@ MultiIndex | |||
| I/O | |||
| ^^^ | |||
|
|
|||
| - | |||
| - Bug in :func:`DataFrame.to_excel()` where cells are missing in the excel file when exporting excel using ``xlsxwriter`` with option ``constant_memory`` set to ``True`` ((:issue:`15392`) | |||
There was a problem hiding this comment.
Don't need the parens after to_excel.
Mismatched parens in the :issue:.
|
I think all of this logic should be contained within the |
It seems not quite possible as the sequence of generating cells, which is the crux of this bug, is not controlled by BTW, I am stuck on how to debug using azure pipelines. Does anyone have some good resources to learn about that? |
|
@KenTsui can you merge master? As far as log failures go if you click on next to failing logs you should be able to click through and find test failure output. Should match what you would see locally in a lot of cases |
|
@KenTsui is this still active? |
|
Closing as stale but ping if you'd like to pick back up |
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff