Skip to content

Commit 93089c0

Browse files
cmaloneyvstinner
andauthored
gh-130806: Emit ResourceWarning if GzipFile unclosed (#130905)
This may indicate accidental data loss. Ways to make sure all data is written: 1. Use the [file-like object](https://docs.python.org/3/glossary.html#term-file-object) as a [“With Statement Context Manager”](https://docs.python.org/3/reference/datamodel.html#context-managers). - All objects which [inherit](https://docs.python.org/3/tutorial/classes.html#inheritance) from [IOBase](https://docs.python.org/3/library/io.html#io.IOBase) support this. - [`BufferedIOBase`](https://docs.python.org/3/library/io.html#io.BufferedIOBase), [`BufferedWriter`](https://docs.python.org/3/library/io.html#io.BufferedWriter), and [`GzipFile`](https://docs.python.org/3/library/gzip.html#gzip.GzipFile) all support this. - Ensures `.close()` is called in both exception and regular cases. 2. Ensure [`.close()`](https://docs.python.org/3/library/io.html#io.IOBase.close) is always called which flushes data before closing. 3. If the underlying stream need to be kept open, use [`.detach()`](https://docs.python.org/3/library/io.html#io.BufferedIOBase.detach) Since 3.12 flushing has been necessary in GzipFile (see gh-105808 which was a release blocker), this makes that more visible. Users have been encountering as they upgrade to 3.12 (ex. gh-129726). There are a number of cases of unclosed file-like objects being deleted in CPython libraries and the test suite. This issue includes resolving those cases where the new ResourceWarning is emitted. Co-authored-by: Victor Stinner <[email protected]>
1 parent d12d8c5 commit 93089c0

File tree

3 files changed

+22
-4
lines changed

3 files changed

+22
-4
lines changed

Lib/gzip.py

+15-1
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,11 @@ def __init__(self, filename=None, mode=None,
193193
194194
"""
195195

196+
# Ensure attributes exist at __del__
197+
self.mode = None
198+
self.fileobj = None
199+
self._buffer = None
200+
196201
if mode and ('t' in mode or 'U' in mode):
197202
raise ValueError("Invalid mode: {!r}".format(mode))
198203
if mode and 'b' not in mode:
@@ -368,7 +373,9 @@ def closed(self):
368373

369374
def close(self):
370375
fileobj = self.fileobj
371-
if fileobj is None or self._buffer.closed:
376+
if fileobj is None:
377+
return
378+
if self._buffer is None or self._buffer.closed:
372379
return
373380
try:
374381
if self.mode == WRITE:
@@ -445,6 +452,13 @@ def readline(self, size=-1):
445452
self._check_not_closed()
446453
return self._buffer.readline(size)
447454

455+
def __del__(self):
456+
if self.mode == WRITE and not self.closed:
457+
import warnings
458+
warnings.warn("unclosed GzipFile",
459+
ResourceWarning, source=self, stacklevel=2)
460+
461+
super().__del__()
448462

449463
def _read_exact(fp, n):
450464
'''Read exactly *n* bytes from `fp`

Lib/test/test_gzip.py

+5-3
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
import struct
1010
import sys
1111
import unittest
12+
import warnings
1213
from subprocess import PIPE, Popen
1314
from test.support import catch_unraisable_exception
1415
from test.support import import_helper
@@ -899,9 +900,10 @@ def test_refloop_unraisable(self):
899900
# fileobj would be closed before the GzipFile as the result of a
900901
# reference loop. See issue gh-129726
901902
with catch_unraisable_exception() as cm:
902-
gzip.GzipFile(fileobj=io.BytesIO(), mode="w")
903-
gc.collect()
904-
self.assertIsNone(cm.unraisable)
903+
with self.assertWarns(ResourceWarning):
904+
gzip.GzipFile(fileobj=io.BytesIO(), mode="w")
905+
gc.collect()
906+
self.assertIsNone(cm.unraisable)
905907

906908

907909
class TestOpen(BaseTest):
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Deleting :class:`gzip.GzipFile` before it is closed now emits a
2+
:exc:`ResourceWarning`.

0 commit comments

Comments
 (0)