PVXS calls errlogInit early enough that it is not possible to override in an st.cmd

[simon-ess]

Describe the bug
Long messages to the errorlog get truncated if you run an ioc using softIocPVX (or something similar).

Ordinarily, you can call

errlogInit2 2048 2047

(or some such large number) early on in an IOC startup script to override this. However, PVXS has a logging call that calls errlogInit() early on which prevents you from overwriting the buffer size.

To Reproduce
Steps to reproduce the behavior:

1. Create an IOC that will produce a longer-than-256 character error log
2. Run it
3. oops

Expected behavior
It should be possible to override the error log messages size to avoid this issue

Information (please complete the following):

• PVXS Version or Git commit ID: 1.3.3
• EPICS Base Version: 7.0.8.1
• libevent Version: n/a
• EPICS_HOST_ARCH: n/a
• Compiler version: n/a

The versions above likely do not matter however.

[mdavidsaver]

softIocPVX (or something similar).

Tricky for softIocPVX. Quite a bit happens during pvxsBaseRegistrar(), including reading $PVXS_LOG. If "something similar" is a regular IOC executable, then my answer is to call errlogInit2() before <ioc>_registerRecordDeviceDrivers().

[simon-ess]

Unfortunately as may not surprise you, we do use softIocPVX as our IOC executable at ESS. We would prefer not to patch PVXS, however...

[mdavidsaver]

Unfortunately as may not surprise you

Actually, I am a little surprised. Maintaining a separate main() do not seem like much effort, and would give you control in situations like this. Not a big deal though.

We would prefer not to patch PVXS, however...

What do you think that patching might look like?

Presently pvxsBaseRegistrar() constructs the Server object. This is done to allow support modules to add Source and SharedPV objects to it prior to iocInit, at which point this Server starts actually serving. The process of creating the Server involves interpretation of configuration and allocation of resource (eg. sockets) which can fail. So logging from the registratar is unavoidable with the present design.

In an st.cmd script, the set of operations which need to precede *_registerRecordDeviceDrivers() is relatively small. Mostly epicsEnvSet(), which can be externalized.

Besides errlogInit*(), can you think of other operations/iocsh commands which need to happen before registrars run?

I can see a couple of ways forward. Broadly, to teach softMain.cpp to call errlogInit* prior to running the registrars. Or to add an option to defer the dbLoadDatabase("softIocPVX.dbd") and softIocPVX_registerRecordDeviceDrivers() call to the user iocsh script, which could then call errlogInit beforehand.

[mdavidsaver]

Besides errlogInit*(), can you think of other operations/iocsh commands which need to happen before registrars run?

Another class of operations which could happen before registrars are run are global variable changes via. var().

[simon-ess]

One possible addition would be the option to run specific commands, e.g. something like

softIocPVX -c "errlogInit2 2048 2087" -c "var foo=bar" -c "epicsEnvSet something else" st.cmd

We currently use this in our wrapper script that runs softIocPVX, mostly for debugging, but this would also work.

I think that or allowing the option to not run *_registerRecordDeviceDrivers seems best. It's true that the list of commands that might need to run before that point is small, but it is good to have that option open.

[simon-ess]

Anyhow, regarding our hesitation to patching things, this is more because we are trying to move away as much as possible from custom patches on community code. Currently we are aiming to use EPICS base and PVXS unmodified (essentially) as a part of a conda distribution, and to have all of our modifications occuring from require onwards. If we want to patch PVXS then this breaks this approach.

That said, we could do as you suggest and include a custom executable. That is a real option.

[anjohnson]

I just created a Codeathon issue for errlog epics-base #747 which might be helpful in the long run.

At APS we always import community sources into a local Git repository to allow for local changes (including build configuration settings) and to decouple ourselves from having to wait for upstream to merge or fix issues that urgently need resolving. We always try to push our local changes upstream and work with them to minimize the differences, but fixing the machine comes first.

Keeping what CVS called a "vendor branch" in our Git repo for the imported community releases makes pulling in new versions from upstream and merging them into our version trivial in most cases (I have a script for that). If we did make local changes Git shows us exactly where they conflict with a new upstream version when we merge that.

[anderslindho]

At APS we always import community sources into a local Git repository to allow for local changes (including build configuration settings) and to decouple ourselves from having to wait for upstream to merge or fix issues that urgently need resolving. We always try to push our local changes upstream and work with them to minimize the differences, but fixing the machine comes first.

Keeping what CVS called a "vendor branch" in our Git repo for the imported community releases makes pulling in new versions from upstream and merging them into our version trivial in most cases (I have a script for that). If we did make local changes Git shows us exactly where they conflict with a new upstream version when we merge that.

FWIW we do something similar at ESS but without importing or forking: we instead have a so-called wrapper repository containing our build configs, patches, and other site/ESS specific modifications.

But with our approach as well as with yours, we would need to create a commit or patch onto the PVXS source, which is what we want to avoid doing (but might do if we do not find better options in time), especially given that we maintain the PVXS build on conda-forge, which we would prefer to keep as close to upstream as possible.