fix: Remove premature return after closing sparse diff file so Convert executes. #4421

ashwat287 · 2025-12-02T09:21:22Z

Fixes #4418
Ensure diffDisk is converted to the requested format instead of staying sparse.

AkihiroSuda · 2025-12-02T11:05:29Z

pkg/driverutil/disk.go

 			return fmt.Errorf("failed to create sparse diff disk %q: %w", diffDisk, err)
 		}
-		return diffDiskF.Close()
+		diffDiskF.Close()


How did you test this PR?
The diffDisk created in L43 will be overwritten by a new disk converted from baseDisk in L57, no?

Yes diffDisk would get converted in L57 when the base image is ISO.
I have include the tests to verify the conditions when base image is ISO and non-ISO
Can you please clarify on what should be the desired behaviors in each test cases?

When the base disk is an ISO9660 image, the diff disk has to be a new empty disk.
("diff" is misnomer here)

The diff disk when created as a sparse file in L49 is empty and has size 0 bytes.
On debugging, I find that after conversion the diffdisk size becomes equal to diskSizeInBytes.

To keep the diffDisk empty we can either:

not convert the diffDisk and leave it as a sparse file and eventually ignore diskImageFormat when the base disk is ISO. or,

convert the diffDisk to the available formats while keeping diskSizeInBytes=0 in case of ISO base disk. The problem here arises when the disk is being converted to asif format, the following error gets thrown specified size 0 is smaller than the original image size because of

lima/pkg/imgutil/nativeimgutil/nativeimgutil.go

Lines 59 to 61 in 9dbf572

if size != nil && *size < srcImg.Size() {

return fmt.Errorf("specified size %d is smaller than the original image size (%d) of %q", *size, srcImg.Size(), source)

}

norio-nomura

This PR does not fix #4418:

$ limactl start ./hack/test-templates/alpine-iso-9p-writable.yaml --vm-type vz --set .vmOpts.vz.diskimageFormat=\"asif\" --set .mountType=null --tty=false --log-level fatal
$ limactl stop alpine-iso-9p-writable
INFO[0000] Sending SIGINT to hostagent process 62151    
INFO[0000] Waiting for the host agent and the driver processes to shut down 
INFO[0000] [hostagent] Received SIGINT, shutting down the host agent 
INFO[0000] [hostagent] Shutting down the host agent     
INFO[0000] [hostagent] Shutting down VZ                 
INFO[0003] [hostagent] [VZ] - vm state change: stopped  
ERRO[0003] [hostagent] accept tcp 127.0.0.1:58662: use of closed network connection 
INFO[0004] Waiting for the instance to shut down        
INFO[0004] The instance alpine-iso-9p-writable has shut down 
$ diskutil image info ~/.lima/alpine-iso-9p-writable/diffdisk
Image Format: RAW
Format Description: RAW read-write image

	Size Info:
		Empty Bytes: 0
		Min Size Bytes: 512
		Sector Count: 209715200
		Total Bytes: 107374182400

	Encryption Info:
		Is Encrypted: 0

	Partitions
		0:
			content-hint: GUID_partition_scheme
			total-space: 107374182400
			used-space: N/A
			volume-name: N/A
		1:
			content-hint: Microsoft Basic Data
			total-space: 167936
			used-space: N/A
			volume-name: N/A
		2:
			content-hint: EFI
			total-space: 1474560
			used-space: N/A
			volume-name: NO NAME
		3:
			content-hint: Microsoft Basic Data
			total-space: 81270784
			used-space: N/A
			volume-name: N/A
$ limactl --version
limactl version 2.0.2-25-gea17ac7c.m

norio-nomura · 2025-12-04T01:20:04Z

This PR does not fix #4418:

updated console log to add missing --set .mountType=null on limactl start

ashwat287 · 2025-12-06T23:11:11Z

$ limactl start ./hack/test-templates/alpine-iso-9p-writable.yaml --vm-type vz --set .vmOpts.vz.diskImageFormat=\"asif\" --set .mountType=null --tty=false --log-level fatal
$ diskutil image info ~/.lima/alpine-iso-9p-writable/diffdisk                                                                                                                     
Image Format: ASIF
Format Description: Apple sparse image

	Identity Info:
		Stable UUID: F240DA64-A9D1-44B2-841C-1BABD6F3CE17
		UUID: 8C2D97C2-80AE-4545-979D-FC4DE3DCF943

	Size Info:
		Empty Bytes: 107303096320
		Max Size Bytes: 4503599626321920
		Min Size Bytes: 1048576
		Sector Count: 209715200
		Total Bytes: 107374182400

	Encryption Info:
		Is Encrypted: 0

	ASIF Info:
		Bitmapped Entries: 83
		Chunk Size: 1048576
		Dir Pointer Version: 4
		Full Entries: 62
		Is Cache: 0
		Max Sector Count: 8796093022208
		Num Tables: 2
		Uninitialized Entries: 155647
		Unmapped Entries: 102256

	Partitions
		0:
			content-hint: FDisk_partition_scheme
			total-space: 107374182400
			used-space: N/A
			volume-name: N/A
		1:
			content-hint: Linux
			total-space: 107373133824
			used-space: N/A
			volume-name: N/A

norio-nomura

Confirmed that #4418 will be corrected.
I don't know if the call to MakeSparse needs to be left, but I don't think it's a problem.

norio-nomura · 2025-12-07T06:17:11Z

pkg/driverutil/disk.go

 		}

-		err = diskUtil.MakeSparse(ctx, diffDiskF, 0)
+		err = diskUtil.MakeSparse(ctx, diffDiskF, diskSizeInBytes)


Is this MakeSparse still needed?

Nope, just an empty volume is enough. I have removed it in the amended commit.

norio-nomura

LGTM! 👍🏻

AkihiroSuda · 2025-12-11T13:38:24Z

pkg/driverutil/disk.go

-			return fmt.Errorf("failed to create sparse diff disk %q: %w", diffDisk, err)
-		}
-		return diffDiskF.Close()
+		diffDiskF.Close()


err is unchecked

AkihiroSuda · 2025-12-11T13:39:37Z

pkg/driverutil/disk_test.go

+func makeTempInstanceDir(t *testing.T) string {
+	t.Helper()
+	return t.TempDir()
+}


This function seems useless. You can just use t.TempDir.

AkihiroSuda · 2025-12-11T13:41:31Z

pkg/driverutil/disk_test.go

+	assert.NilError(t, os.Remove(path))
+	_, err := os.Stat(path)
+	assert.ErrorIs(t, err, os.ErrNotExist)
+}


This function seems useless, why not just os.Remote it?

os.Remove returning nil will only ensure that that the syscall didn't report an error. But it doesn't really asserts that the file is deleted.
By calling os.Stat and asserting os.ErrNotExist confirms that it is not present in the filesystem.

But it doesn't really asserts that the file is deleted.

Did you actually observe such a behavior? On what filesystem?

No, I haven't observed such behaviour. And yes, now that i have checked again, I feel that the additional assert by os.Stat is redundant here

But it doesn't really asserts that the file is deleted.

Here i meant that before the process ends the the file might get created again after deletion, not that it doesn't get deleted by os.remove at the first place. According to me this can happen if there is any race condition or when the filesystem is buggy, which I feel is a very specific case and doesn't apply here.

I will remove the function removeAndCheck and just keep os.remove

AkihiroSuda · 2025-12-11T13:43:56Z

pkg/driverutil/disk_test.go

+	if runtime.GOOS != "darwin" {
+		return false
+	}
+	out, err := exec.CommandContext(t.Context(), "sw_vers", "-productVersion").Output()


Use

lima/pkg/osutil/osversion_darwin.go

Line 16 in ae90938

var ProductVersion = sync.OnceValues(func() (*semver.Version, error) {

AkihiroSuda · 2025-12-11T13:44:12Z

pkg/imgutil/nativeimgutil/nativeimgutil.go

 	case raw.Type:
-		if err = srcF.Close(); err != nil {
-			return err
-		}


lima/pkg/imgutil/nativeimgutil/nativeimgutil.go

Line 128 in ae90938

if err := makeSparse(destTmpF, srcImg.Size()); err != nil {

srcImg.size() was giving the value -1. internally it was not able to run the syscall os.stat because it was being closed early when the close statement was outside the if statement. This change ensures that the the file is open when trying to get scrImg.Size() and that the value is not -1.

lima/pkg/imgutil/nativeimgutil/nativeimgutil.go

Line 123 in ae90938

defer destTmpF.Close()

and

lima/pkg/imgutil/nativeimgutil/nativeimgutil.go

Line 54 in ae90938

defer srcF.Close()

The above defer statements already take care of closing the file after processing.

…ty volume for diff disk so Convert executes. Ensure diffDisk is converted to the requested format instead of staying sparse. Signed-off-by: ashwat287 <[email protected]>

AkihiroSuda · 2025-12-12T09:24:56Z

pkg/driverutil/disk_test.go

+	assert.Equal(t, detectImageType(t, diff), expectedType)
+}
+
+var ProductVersion = sync.OnceValues(func() (*semver.Version, error) {


Why duplicate

lima/pkg/osutil/osversion_darwin.go

Line 16 in ae90938

var ProductVersion = sync.OnceValues(func() (*semver.Version, error) {

Sorry, I missed that it already exists 😵‍💫

Add pkg/driverutil/disk_test.go covering: - Base image is ISO: EnsureDisk creates/keeps diffdisk and converts to asif and raw; base ISO remains unchanged (content hash and ISO signature). - Base image is non-ISO: EnsureDisk converts diffdisk to raw and asif. Signed-off-by: ashwat287 <[email protected]>

AkihiroSuda reviewed Dec 2, 2025

View reviewed changes

ashwat287 force-pushed the Issue4418 branch 2 times, most recently from efb2779 to ea17ac7 Compare December 2, 2025 17:18

AkihiroSuda requested a review from norio-nomura December 3, 2025 11:31

norio-nomura reviewed Dec 4, 2025

View reviewed changes

ashwat287 force-pushed the Issue4418 branch 5 times, most recently from e7cdb60 to 2951d80 Compare December 6, 2025 22:54

ashwat287 requested a review from norio-nomura December 6, 2025 23:21

norio-nomura approved these changes Dec 7, 2025

View reviewed changes

ashwat287 force-pushed the Issue4418 branch 2 times, most recently from d4948a6 to cde8922 Compare December 7, 2025 11:35

ashwat287 requested a review from norio-nomura December 8, 2025 20:28

norio-nomura approved these changes Dec 9, 2025

View reviewed changes

ashwat287 requested a review from AkihiroSuda December 9, 2025 05:12

AkihiroSuda added this to the v2.0.3 milestone Dec 11, 2025

AkihiroSuda reviewed Dec 11, 2025

View reviewed changes

fix: for ISO base disk remove premature return after creating and emp…

65f9ec3

…ty volume for diff disk so Convert executes. Ensure diffDisk is converted to the requested format instead of staying sparse. Signed-off-by: ashwat287 <[email protected]>

ashwat287 force-pushed the Issue4418 branch from cde8922 to cfa7dd6 Compare December 11, 2025 20:06

ashwat287 requested a review from AkihiroSuda December 11, 2025 20:38

ashwat287 force-pushed the Issue4418 branch from cfa7dd6 to 39fbe4a Compare December 12, 2025 09:20

AkihiroSuda reviewed Dec 12, 2025

View reviewed changes

ashwat287 force-pushed the Issue4418 branch from 39fbe4a to 65b6485 Compare December 12, 2025 09:58

	if size != nil && *size < srcImg.Size() {
	return fmt.Errorf("specified size %d is smaller than the original image size (%d) of %q", *size, srcImg.Size(), source)
	}

fix: Remove premature return after closing sparse diff file so Convert executes. #4421

Are you sure you want to change the base?

fix: Remove premature return after closing sparse diff file so Convert executes. #4421

Uh oh!

Conversation

ashwat287 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwat287 Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwat287 Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norio-nomura left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norio-nomura commented Dec 4, 2025

Uh oh!

ashwat287 commented Dec 6, 2025

Uh oh!

norio-nomura left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

norio-nomura left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwat287 Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwat287 Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashwat287 Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ashwat287 commented Dec 2, 2025 •

edited

Loading

ashwat287 Dec 2, 2025 •

edited

Loading

ashwat287 Dec 3, 2025 •

edited

Loading

norio-nomura left a comment •

edited

Loading

ashwat287 Dec 12, 2025 •

edited

Loading

ashwat287 Dec 11, 2025 •

edited

Loading

ashwat287 Dec 12, 2025 •

edited

Loading