Permit namelist character delimiters to default to quotes

[marshallward]

Currently there is a somewhat paradoxical issue related to namelist support, where a namelist produced by WRITE() and using the default DELIM value of None does not conform to the namelist specification when it contains a character array.

A namelist as described in F2018 requires that all character arrays be delimited with single or double quotes (13.11.3.3 p7):

When the next effective item is of type character, the input form consists of a sequence of zero or more rep-chars whose kind type parameter is implied by the kind of the corresponding list item, delimited by apostrophes or quotes.

However, the default value of DELIM=None will produce a namelist without delimiters (i.e. space-delimited). From 13.11.4.2 p1:

Values in namelist output records are edited as for list-directed output (13.10.4).

If we go back to 13.10.4, it highlights three forms of character array output based on DELIM, with NONE being the default. In other words, the default is incompatible with the namelist format.

Note 1 of 13.11.4.2 clarifies this point:

Namelist output records produced with a DELIM= specifier with a value of NONE and which contain acharacter sequence might not be acceptable as namelist input records.

In practice, this is often not a problem, and a robust parser can usually resolve the non-delimited strings if whitespace can act as a delimiter. But it becomes a more problematic, if not impossible, if the string contains a lexical token, such as &, =, or /. For example, strings containing paths are almost guaranteed to cause problems.

To summarize, a namelist containing character arrays when written with DELIM=NONE does not conform to the namelist specification, and a compiler can not read its own namelist if DELIM is not set to either QUOTE or APOSTROPHE.

Currently, gfortran ignores this requirement and defaults to using quote (") delimiters. Intel Fortran does not, and will produce a namelist that it cannot read.

---

While this particular problem can be avoided by requiring the programmer to use DELIM for all namelist output (e.g. DELIM='QUOTE'), it will inevitably cause problems for the less experience developer, and could lead to output with errors.

I'd like to propose that this problem be address in some way. Two possible solutions:

• Require that that DELIM be set for a namelist input, and DELIM='QUOTE' or DELIM=APOSTROPHE) when writing a namelist. If WRITEis called with aNMLargument and withoutDELIM`, then it is an error.

• Follow GFortran and silently use DELIM='QUOTE' (or APOSTROPHE) when the output is a namelist.

There may be others possible solutions.

This is not a problem that needs to be solved, since users could just be trained to use DELIM when writing namelists. But I believe that this change would be an improvement in usability and would help to avoid future errors.

---

This was prompted by the following discussion on the Intel forums:

https://software.intel.com/en-us/forums/intel-fortran-compiler/topic/831685#comment-1947391

[FortranFan]

@marshallward, I totally agree.

I personally find NAMELIST a very useful facility in Fortran even if it is generally underutilized and maligned at times. I also think the support toward NAMELIST in derived-type IO (so happy for that!) can also help with user-friendly serialization and deserialization of object instances of 'classes' (derived types) in Fortran, so anything that nourishes or otherwise keeps fresh the NAMELIST facility is a good thing in my opinion.

Your proposal looks valuable, it should be relatively easy to add to the standard if there is will for the same.

[certik]

@marshallward thanks a lot for reporting this. I agree, I think we should fix it. Let's try to write a proposal for this and let's get the committee to consider this at the next meeting.

[marshallward]

Thanks for the positive feedback, @FortranFan @certik . My impression from past interactions with J3 members is that there is little interest in doing anything related to namelists, so this is encouraging to hear.

I'll defer to more experienced members here on how to properly write and submit a proposal.

[milancurcic]

I'll defer to more experienced members here on how to properly write and submit a proposal.

Please, please, at least start a draft (submit via PR). You already have substantial material in this original post of this thread. It will significantly lower the bar for others to add contributions to the proposal once there's something concrete there. See a current early proposal draft as an example:

https://github.com/j3-fortran/fortran_proposals/blob/master/proposals/namespace_modules/19-246.txt

I will put an effort soon toward a proposal template which will help streamline the bootstrapping of a proposal.

[marshallward]

Sure, in that case I'm happy to put something together. I'll look over some of the proposals and will ping for feedback when it's ready.

[certik]

Thanks @marshallward. I would like the wide community to create proposals, and then the J3 members can help refine it and provide feedback. Such a process scales well with people.

[marshallward]

I've drafted most of a proposal here:

https://github.com/marshallward/fortran_proposals/blob/namelist_delim/proposals/namelist_delimiters/namelist_proposal.rst

I am not entirely sure how it is supposed to look, and I did deviate a bit from the example. For example, I don't have a "use cases" section, since I was suggesting a different interpretation of existing rules, rather than a new feature. I also would expect that the final section is to be removed, I mostly left those as my own personal notes.

I can either send it as a PR or suggestions can be made on my local branch, whatever is easier.

[certik]

@marshallward thanks! Send a PR please to this repository.

I would like to have a template here about what a proposal should look like so that the committee can consider it. In the meantime, create a PR and let's discuss it.

[marshallward]

I've created a PR here: #94

If you'd like to close this issue and move discussion there, that's fine with me.

[zjibben]

I shared @marshallward 's #154 proposal with the J3/JoR subgroup. Dan gave me very detailed feedback summarizing their conclusions:

1. Changing the default behavior of namelist will change the behavior of existing programs. This may be done only for a very important reason, and a convenience issue, such as a default setting, likely does not rise to that level.

2. The default will also affect list-directed transfers, which is likely unintended, and will be surprising to list-directed users. This also argues that a very good reason will be required to overcome the desire not to break programs.

Note that one may set the delim value via the OPEN statement or the specific transfer (read/write print) statement in question. The ease of setting the desired behavior per connection or per transfer argues that we not proceed with a change like this one. As a changeable mode, delim may be set and reset as needed via the initial OPEN and (re)OPEN statements.

Also, we have added several features included in the last several revisions of the standard to help input/output of csv records. Specifically, the unlimited format item allows writing "item [separator item] ..." style records. Including a g data edit descriptor in the unlimited format item can treat different types (perhaps when a derived type has had a new component added, for example). The new f202x SPLIT intrinsic operates similarly to strtok(), and helps decode csv records read into a (possibly long) character entity.

These are my interpretation of JoR's conclusions and not J3's, nor WG5's. I say 'likely' above because it is a committee decision, and I do not know with certainty what the outcome of votes would be.

[marshallward]

Thanks very much for providing the feedback.

I'm sympathetic to point 1, both options would change default behavior, and perhaps that is an automatic veto.

I think my only response would be to point 2. I deliberately did not intend for list-directed output to be affected here, and that DELIM would default to different values depending on whether or not NML= has been set in a WRITE() statement. If the NML= is absent, and the output is list-directed, then the default DELIM from OPEN() (or perhaps WRITE()) would be respected.

So I feel that there ought to be a solution which does not affect list-directed input. In fact, given that GFortran is already doing this, it is proof that there is a way to implement this which does not affect list-directed output.

In other words, I am requesting that the non-compliant GFortran behavior become the standard behavior.

Not sure if there's any constructive way to relay this feedback, so perhaps I am just talking to the air. But thanks very much for presenting the proposal and sending the feedback, I really do appreciate it.

[certik]

@marshallward is your new proposal (make GFortran behavior the default) different to what you submitted above? If so, let's write a new proposal and we will present it at the next meeting.