Support compiling and fetching from cache in parallel

[luser]

If we implement #30 to let local developers pull from the S3 cache, performance is going to be variable depending on network timings. It'd be nice as a future optimization if we could run a compilation in parallel with a cache fetch, and use whichever result returns first. We might want this to have a heuristic where if it finds that some percentage of the time one strategy wins that it will prefer that strategy. Like if someone is building with a decent CPU and slow network, we might want to limit the number of concurrent fetches from S3 and just run local builds for any other compile requests that come in.

[luser]

As a follow-up to the last sentence in my previous comment, it strikes me that sccache could provide a net speedup even with a fast CPU and slower network access if we ran the build with -jN where N is greater than the number of local CPU cores, but limited sccache such that it would only run enough concurrent compile jobs to saturate the number of CPU cores, allowing others to fetch cache hits in the background. That way someone could saturate their local CPU building some objects locally while simultaneously fetching cache hits for other objects.