all: test failures with "'/boot/workdir' fid in use" on plan9-arm

[bcmills]

#!watchflakes
post <- builder ~ `^plan9` && `fid in use`

2021-06-02T19:26:46-29f5b8f/plan9-arm
2021-06-01T21:14:02-567ee86/plan9-arm
2021-05-27T19:55:46-cdcd028/plan9-arm
2021-05-19T13:04:45-658b5e6/plan9-arm
2021-05-13T14:52:20-2a61b3c/plan9-arm
2021-05-07T21:01:09-3980c4d/plan9-arm
2021-05-07T02:17:32-d2b0311/plan9-arm
2021-05-06T16:28:56-5f9fe47/plan9-arm
2021-05-04T19:12:24-f03daea/plan9-arm
2021-05-03T21:45:16-42984c4/plan9-arm
2021-04-30T18:06:26-c05d50f/plan9-arm
2021-04-29T23:41:22-a9705e1/plan9-arm
2021-04-29T04:59:52-6afa0ae/plan9-arm
2021-04-28T15:10:38-5c69cb2/plan9-arm
2021-04-23T21:43:02-59ceb98/plan9-arm
2021-04-16T19:20:53-95ed5c3/plan9-arm
2021-04-15T01:54:41-48b7432/plan9-arm
2021-04-14T16:59:08-25b25a9/plan9-arm
2021-04-09T04:03:50-8518aac/plan9-arm
2021-04-05T19:29:17-d5b9dc1/plan9-arm
2021-03-30T16:07:38-c274a7c/plan9-arm
2021-03-17T18:27:22-9b614f5/plan9-arm
2021-03-15T23:49:28-d7cc2f1/plan9-arm
2021-03-15T20:28:34-c236095/plan9-arm
2021-03-13T18:02:24-3224990/plan9-arm
2021-03-10T04:06:13-4d608eb/plan9-arm
2021-03-10T01:46:25-41245ab/plan9-arm
2021-03-08T03:23:21-125eca0/plan9-arm
2021-02-24T17:00:14-adb467f/plan9-arm
2021-02-20T19:33:34-40656f3/plan9-arm
2021-02-17T00:04:02-70c37ee/plan9-arm
2021-02-16T22:18:37-8482559/plan9-arm
2021-02-11T05:13:29-706a59c/plan9-arm
2021-01-25T23:54:36-deaf29a/plan9-arm
2021-01-22T06:33:04-11def3d/plan9-arm
2021-01-21T20:32:52-9f43a9e/plan9-arm
2021-01-21T19:09:28-b186e4d/plan9-arm
2021-01-15T20:36:53-b386c73/plan9-arm
2021-01-08T16:28:15-32afcc9/plan9-arm
2021-01-07T01:55:27-df81a15/plan9-arm
2020-10-20T20:17:14-7f73669/plan9-arm
2020-10-19T18:48:17-ab541a0/plan9-arm
2020-10-13T19:15:53-076a45a/plan9-arm
2020-10-08T08:53:13-542693e/plan9-arm
2020-10-02T14:15:43-2268388/plan9-arm
2020-10-01T02:42:58-b8ec1d5/plan9-arm
2020-09-16T15:04:07-587cf23/plan9-arm
2020-09-08T17:01:02-ae3680b/plan9-arm
2020-08-27T18:28:53-c00b708/plan9-arm
2020-08-26T21:12:37-c769f03/plan9-arm
2020-08-21T13:58:45-9ac8e33/plan9-arm
2020-08-15T05:58:24-c0dded0/plan9-arm
2020-08-12T18:35:22-20254c0/plan9-arm
2020-07-16T19:33:22-f2c07d7/plan9-arm

CC @fhs @millerresearch @0intro

[millerresearch]

The plan9-arm builders are running an ad hoc directory cache on top of the normal fossil file system, to mitigate the performance cost of extravagant file tree walking as discussed in #38537 and #24300. I think this error is flakiness in that cache. I'll look into it.

[bcmills]

@millerresearch, any luck tracking down the cache issue? This is still a fairly frequent cause of flakes on the plan9.-arm builder.

greplogs --dashboard -md -l -e 'fid in use' --since=2021-06-03

2021-11-03T05:00:00-519c0a2/plan9-arm
2021-11-02T23:05:17-714668c-eb63cbd/plan9-arm
2021-11-02T13:43:24-ee08195-4d7bf41/plan9-arm
2021-11-01T15:55:25-e2e910e/plan9-arm
2021-10-30T16:46:47-d1dceaf/plan9-arm
2021-10-30T12:32:06-351c04c-5d6d9f5/plan9-arm
2021-10-29T02:11:42-33c392f/plan9-arm
2021-10-28T17:52:59-3124968/plan9-arm
2021-10-27T20:25:06-bbc0595/plan9-arm
2021-10-27T13:12:49-cfb5321/plan9-arm
2021-10-26T23:12:17-13eccaa/plan9-arm
2021-10-26T01:22:47-f916b54-adfb85b/plan9-arm
2021-10-25T23:28:03-e9eb66d/plan9-arm
2021-10-22T14:05:31-ff8115d/plan9-arm
2021-10-21T17:31:42-392bb06/plan9-arm
2021-10-20T20:39:19-d419a80/plan9-arm
2021-10-19T18:22:59-6c3cd5d/plan9-arm
2021-10-19T17:12:53-404f84d/plan9-arm
2021-10-19T07:45:46-281050f-ee92daa/plan9-arm
2021-10-15T00:16:20-ce04ca3-9e8ed86/plan9-arm
2021-10-14T13:09:28-24e798e/plan9-arm
2021-10-13T18:15:48-40f82f8/plan9-arm
2021-10-13T15:11:16-0454d73/plan9-arm
2021-10-12T21:15:50-3d051ba/plan9-arm
2021-10-12T20:20:41-d032b2b/plan9-arm
2021-10-11T19:20:12-18fa840-65ffee6/plan9-arm
2021-10-07T22:19:58-1ceb723/plan9-arm
2021-10-07T16:38:24-be571a3/plan9-arm
2021-10-01T23:51:16-c129af9/plan9-arm
2021-10-01T21:26:43-0ebff1a-0d65c27/plan9-arm
2021-10-01T19:30:53-13c407c-0d65c27/plan9-arm
2021-09-30T01:32:54-1c35f2a-6656269/plan9-arm
2021-09-28T22:34:40-0df0ca0-b8a6017/plan9-arm
2021-09-28T21:42:57-da36f79-b8a6017/plan9-arm
2021-09-28T19:51:56-e7a2573/plan9-arm
2021-09-28T17:13:30-e301b2f/plan9-arm
2021-09-24T14:52:47-217507e/plan9-arm
2021-09-23T17:47:39-29bf010-2fc7697/plan9-arm
2021-09-22T16:53:49-efaec4e-e128749/plan9-arm
2021-09-21T20:28:50-2fc7df9/plan9-arm
2021-09-21T12:51:45-9cbdc1d/plan9-arm
2021-09-19T02:43:09-2758b04-771b8ea/plan9-arm
2021-09-17T00:31:49-14e812b/plan9-arm
2021-09-13T22:57:34-9a58aa2/plan9-arm
2021-09-13T18:38:15-ac40c98/plan9-arm
2021-09-13T18:02:42-e74e363/plan9-arm
2021-09-09T00:29:35-4c52eac/plan9-arm
2021-09-08T18:48:02-65f0d24/plan9-arm
2021-09-08T18:47:32-37c9552/plan9-arm
2021-09-07T20:37:05-8078355/plan9-arm
2021-09-02T14:35:10-37d4532/plan9-arm
2021-09-01T22:28:21-8373dc3-0bfd6fc/plan9-arm
2021-08-23T03:39:14-d4cc65f-457418b/plan9-arm
2021-08-22T22:30:38-d4cc65f-29d7e54/plan9-arm
2021-08-22T22:04:02-d4cc65f-5d5e50c/plan9-arm
2021-08-20T03:25:36-bf6c7f2-e9e0d1e/plan9-arm
2021-08-19T16:10:32-bf6c7f2-91e2e3b/plan9-arm
2021-08-16T23:32:14-8ff16c1/plan9-arm
2021-08-16T14:27:31-fe489c8/plan9-arm
2021-08-13T20:49:48-bad1fc1/plan9-arm
2021-08-12T16:12:36-749a5cd-095bb79/plan9-arm
2021-08-11T16:51:24-5f5a173-d4c0ed2/plan9-arm

[millerresearch]

@millerresearch, any luck tracking down the cache issue?

No, I'll have another look.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)