Preprocessor support

[LKedward]

This issue is to ask whether fpm will have any built-in support for preprocessing and how this might look.

I bring this up since I noticed that stdlib is listed in #17 under 'Pure Fortran', however it requires the fypp preprocessor to build from repo source.

[everythingfunctional]

We've had discussions about it. I believe we decided we want to settle on a preprocessor, and just always use it. I'd have to go find that discussion, but I believe we settled on fypp.

[certik]

Yes, we definitely want fpm to apply a preprocessor. We also talked about file extensions, and it seems most people would prefer to just stick with .f90, and fpm would apply the preprocessor appropriately (via a compiler option or otherwise).

We probably should support both cpp and fypp. For fypp we should further provide a fast C++ implementation, so that we don't have to depend on Python.

[milancurcic]

👍 on both cpp and fypp. cpp is de facto standard and many Fortran projects rely on it. fypp we have established earlier that it is more powerful than cpp and thus useful for generation of specific procedures like those in stdlib.

For fypp we should further provide a fast C++ implementation, so that we don't have to depend on Python.

I agree, although this is a non-issue until fpm itself is Fortran or C++. Python ships out of the box on most systems.

[certik]

I agree, although this is a non-issue until fpm itself is Fortran or C++.

Actually it's an issue for distributing fpm, as we cannot easily integrate fypp into the fpm single binary, so we now have to ship it along side fpm somehow, etc.

However, since we will eventually use Conda for the non Fortran dependencies, then fypp can just be installed using Conda / Mamba, and then indeed it should become a non-issue.

[urbanjost]

So is there any interest in a Fortran-based version of fpp? Without macro support I think it would be simple to convert
pfpp to look like fpp(1), albeit there are varients of fpp(1) to be sure. Adding macros would be considerably more effort. Adding basic templating based on looping or cacheing and substitution would probably be easier than macros. It would not be as powerful as some but not looking to recreate m4(1). But for basic if/else/elseif/endif preprocessing it works well. Combined with fpm setting a few variables in a standard way like OSTYPE and COMPILER and COMPILER_VERSION it could ship with fpm and only require a Fortran compiler. I am thinking it would just handle user-end pre-processing, not expand the code on the backend so it would handle "cpp"-like functions and assume the files were already expanded via fypp or whatever before-hand so that all that remained were .F and .F90 files with #if/#else/#elif/#endif/#ifdef/#ifndef directives.

[ivan-pi]

I definitely think having a Fortran based preprocessor that can be tweaked as desired is appealing. Have you compared your preprocessor to the list of behaviors compiled by the flang developers: https://github.com/llvm/llvm-project/blob/master/flang/docs/Preprocessing.md?

If anyone is interested I also uploaded the Sun Microsystems Fortran preprocessor to GitHub: https://github.com/ivan-pi/fdfpp
The original code can be found on netlib. I have made no attempts to run it.

[urbanjost]

Making pfpp support macros and some of the other issues there is more effort than it is worth without stronger interest I suppose. Interesting list. The main problem seemed to be to emulate cpp without the issues that cpp has, and then some added
a few features to fix other issues with pre-processing Fortran like resultant line length and others did not and so on. I had run across a few issues with fpp commands not being the same but did not know it varied that much. Looks like I would have hit more if I had not started using my own.

[LKedward]

+1 for a Fortran-based preprocessor, however as pointed out by Ivan, there is notoriously poor portability due to variation among Fortran preprocessor implementations. This is the primary reason for me avoiding preprocessors with Fortran. With that said I am quite impressed by what can be achieved with fypp.

I wonder whether there is interest in developing a community-agreed standard for preprocessing Fortran. Failure of previous standardisation efforts shouldn't preclude this.

[ivan-pi]

I am also quite satisfied by what can be done with fypp (even if the syntax is quite verbose). However, I am not sure if Ondrej's suggestion to have a Fortran / C++ version of fypp is feasible. At least some elements of the fypp preprocessor language are tightly coupled to Python, e.g. one can call Python functions directly.

Certainly, fpm is an excellent place to experiment with new preprocessing constructs. As long as the intermediate (standard-conforming) ".f90" files are recoverable one can use it portably across compilers. On a personal level however, I will rather direct my efforts in other areas first.

[ivan-pi]

Could a plugin based approach like suggested in #211 also be used to implement support of various preprocessors (this issue, #308, #469)?

A list of Fortran preprocessors can be found at the Fortran Wiki. My feeling is that overall the opinions are too divided to settle on any single one of them.

The cpp/fpp preprocessors which are currently the de-facto standard are too limited for some tasks, but still used heavily in practice. The younger "tech-savvy" Fortran users prefer fypp as a more complete solution. On the other hand, in recent discussions at Discourse several users expressed their irritation about having to install Python to use fypp.

Preprocessor plugins could be offered as fpm dependency packages. Then everyone could just use the preprocessor he likes best.

The Intel Fortran compiler offers something similar with the -fpp-name compiler flag. This allows user to supply their own custom preprocessors, by using a command like this (I haven't tested it):

ifort -fpp-name=fypp -Qoption,fpp,"--line-length=132" [..]

where the quoted text are the options passed to fypp. The -D (define) and -I (include directory) options are forwarded automatically. The compiler then spawns a sub-command with the following signature:

fypp [[-D<define>]..] [[-I<include>]..] --line-length=132 [..]

Output from the preprocessor goes to stdout and is captured for any further processing.

I know @certik has suggested in a few places we embed a preprocessor with fpm, but I was wondering lately whether an extensible plugin model fits the diversity of Fortran preprocessing needs better. One negative aspect of this approach, is that instead of joining forces to build a single reliable preprocessor that covers most needs, we'd essentially remain with the current pool of (flawed) tools (no offense meant to Fortran preprocessor developers).