MS level for precursor-ion, neutal-loss/gain scans

[caetera]

While working on #197, I found out that internally API uses negative MS level values for all triple quad scans (precursor, neutral loss/gain). For the mzML implementation there is a special CV term for this kind of scans, and, thus I just skip reporting the MS level term for these. In high density formats, such as mzparquet there are no scan types, thus, I am a bit in doubt as to how to report such scans with minimal confusion for the user. I can see a couple of options.

1. Report it "as is", i.e. as negative integer, documenting the values to meaning mapping. Requires changing schema from uint to int.
2. Report these as MS level 0 - i.e. unknown.
3. Report these as MS level 2 - they are similar to MS2, there is a single fragmentation event, however for the neutral loss/gain scans there is no precursor.
4. Introduce a Enum-column for scan type, similar to mzML, i.e. MS scan, MSn scan, neutral loss scan etc.

What do you think?

@ypriverol @lazear

[lazear]

I think 3 or 4 probably make the most sense?

The nice thing about a columnar format is that it is essentially "free" to add columns - downstream users/readers can define the minimal set of columns they need. I'm also not aware (haven't looked) of anyone publicly using mzparquet, so it's okay to play fast and loose at this point I think.

I had some earlier versions of the format I played around with that included a cvParam column, but had some issues with out-of-the-box readers not supporting map columns

[ypriverol]

I like option 4, with some kind of CvParam that explains that this is a negative scan, similar to what mzML does. @mobiusklein has been playing a lot with parquet and RAW files, it would be nice to have his input.

[mobiusklein]

I also think option 4 is the best choice, recording a 2 in the MS level column (see https://github.com/ProteoWizard/pwiz/blob/master/pwiz/data/vendor_readers/Thermo/SpectrumList_Thermo.cpp#L233-L254). If you encode the categorical value as a string, either a name, a CURIE, or a group encoding the same information, are capable of compactly representing those states with RLE dictionaries. As it is, if you use a string of any sort, the only cost compared to using an integer comes when you try to evaluate a predicate on the column, but even then, it's essentially trivial. Depending upon how the file is encoded, you might have issues with Polars and how it treats string columns with dictionaries as Categorical types.

The nice thing about a group is it naturally extends to storing a value, but some tradeoff between logical tidiness and happy path ergonomics is made there, and then trade again to decide how to represent the value.

I do not know whether the format you're writing here is expected to be read with indices, or just scanned over.

[caetera]

Thank you all for your input. It seems option 4 is the winner. I will extend the schema with the scan_type<string> column. Should it be a qualified cvParam name, for example MSn spectrum, or the cvParam id, for example MS:1000580? The former is easier for humans, but the latter is probably easier for automatic parsers.

[caetera]

Implemented in 5bac912 as qualified cvParam name.