A bunch of VxWorks issues

[ompiteam]

An enterprising Mercury employee (Ken Cain) has been noodling around with getting OMPI to compile on vxworks. After talking extensively with him at a conference, he sent a list of current issues that he is having:

---

Hello Jeff,

At the OFA reception tonight you asked me to send the list of porting issues I've seen so far with OMPI for !VxWorks PPC. It's just a raw list that reflects a work in progress, sorry for the messiness...

-Ken

1. configure issues with "checking prefix for global symbol labels"

1a. !VxWorks assembler (CCAS=asppc) generates a.out by default (vs. conftest.o that we need subsequently)

there is this fragment to determine the way to assemble conftest.s:

if test "$CC" = "$CCAS" ; then
    ompi_assemble="$CCAS $CCASFLAGS -c conftest.s >conftest.out 2>&1"
else
    ompi_assemble="$CCAS $CCASFLAGS conftest.s >conftest.out 2>&1"
fi

The subsequent link fails because conftest.o does not exist:

   ompi_link="$CC $CFLAGS conftest_c.$OBJEXT conftest.$OBJEXT -o conftest > conftest.link 2>&1"

To work around the problem, I did not set CCAS. This gives me the first
invocation that includes the -c argument to CC=ccppc, generating
conftest.o output.

1b. linker fails because LDFLAGS are not passed

The same linker command line caused problems because $CFLAGS were passed
to the linker

   ompi_link="$CC $CFLAGS conftest_c.$OBJEXT conftest.$OBJEXT -o conftest > conftest.link 2>&1"

In my environment, I set CC/CFLAGS/LDFLAGS as follows:

CC=ccppc

CFLAGS=-ggdb3 -std=c99 -pedantic -mrtp -msoft-float -mstrict-align
-mregnames -fno-builtin -fexceptions'

LDFLAGS=-mrtp -msoft-float -Wl,--start-group -Wl,--end-group
-L/amd/raptor/root/opt/WindRiver/vxworks-6.3/target/usr/lib/ppc/PPC32/sfcommon

The linker flags are not passed because the ompi_link

[xp-kcain1:build_vxworks]  ccppc -ggdb3 -std=c99 -pedantic -mrtp -msoft-float -mstrict-align -mregnames -fno-builtin -fexceptions -o hello hello.c
/amd/raptor/root/opt/WindRiver/gnu/3.4.4-vxworks-6.3/x86-linux2/bin/../lib/gcc/powerpc-wrs-vxworks/3.4.4/../../../../powerpc-wrs-vxworks/bin/ld: 
cannot find -lc_internal
collect2: ld returned 1 exit status

1. OPAL atomics asm.c:

int versus int32_t (refer to email with Brian Barrett

1. OPAL event/event.c: sys/time.h and timercmp() macros not defined by !VxWorks refer to workaround in event.c using #ifdef MCS_VXWORKS
2. OPAL event/event.c: pipe() syscall not found

workaround:

#ifdef HAVE_UNISTD_H
#include <unistd.h>
#ifdef MCS_VXWORKS
#include <ioLib.h>      /* for pipe() */
#endif
#endif

1. OPAL event/signal.c

static sig_atomic_t opal_evsigcaught[NSIG];

NSIG is not defined, but _NSIGS is

In Linux, NSIG is defined with -D__USE_MISC

So I added this code fragment to signal.c:

/* VxWorks signal.h defines _NSIGS, not NSIG */
#ifdef MCS_VXWORKS
#define NSIG (_NSIGS+1)
#endif

1. OPAL event/signal.c: no socketpair()

workaround: use pipe():

#ifdef HAVE_UNISTD_H
#include <unistd.h>
#ifdef MCS_VXWORKS
#include <ioLib.h>      /* for pipe() */
#endif
#endif

and later in void opal_evsignal_init(sigset_t *evsigmask)

#ifdef MCS_VXWORKS
        if (pipe(ev_signal_pair) == -1)
                event_err(1, "%s: pipe", __func__);
#else
    if (socketpair(AF_UNIX, SOCK_STREAM, 0, ev_signal_pair) == -1)
        event_err(1, "%s: socketpair", __func__);
#endif

1. OPAL util/basename.c: #if HAVE_DIRNAME problem

../../../opal/util/basename.c:23:5: warning: "HAVE_DIRNAME" is not defined
../../../opal/util/basename.c: In function `opal_dirname':

problem: HAVE_DIRNAME is not defined in opal_config.h so the #if HAVE_DIRNAME will fail at preprocessor/compile time

workaround:

change #if HAVE_DIRNAME to #if defined(HAVE_DIRNAME)

1. OPAL util/basename.c: strncopy_s and _strdup

../../../opal/util/basename.c: In function `opal_dirname':
../../../opal/util/basename.c:153: error: implicit declaration of
function `strncpy_s'
../../../opal/util/basename.c:160: error: implicit declaration of
function `_strdup'

#ifdef MCS_VXWORKS
        strncpy( ret, filename, p - filename);
#else
                strncpy_s( ret, (p - filename + 1), filename, p - filename );
#endif
#ifdef MCS_VXWORKS
    return strdup(".");
#else
    return _strdup(".");
#endif

1. opal/util/if.c: socket() prototype not found in vxworks headers

#ifdef HAVE_SYS_SOCKET_H
#include <sys/socket.h>
#ifdef MCS_VXWORKS
#include <sockLib.h>
#endif
#endif

1. opal/util/if.c: ioctl()

#ifdef HAVE_SYS_IOCTL_H
#include <sys/ioctl.h>
#ifdef MCS_VXWORKS
#include <ioLib.h>
#endif
#endif

1. opal/util/os_path.c: MAXPATHLEN change to PATH_MAX

#ifdef MCS_VXWORKS
    if (total_length > PATH_MAX) {  /* path length is too long - reject
it */
        return(NULL);
#else
    if (total_length > MAXPATHLEN) {  /* path length is too long -
reject it */
        return(NULL);
#endif

1. opal/util/output.c: gethostname()

#include <hostLib.h>

1. opal/util/output.c: MAXPATHLEN

same fix as os_path.c above

1. opal/util/output.c: closelog/openlog/syslog

manually turned off HAVE_SYSLOG_H in opal_config.h, then got a patch from Jeff Squyres that avoids syslog

1. opal/util/opal_pty.c

complains about mismatched prototype of opal_openpty() between this source file and opal_pty.h

workaround: manually edit build_vxworks_ppc/opal/include/opal_config.h, use the following line (change 1 to 0):

#define OMPI_ENABLE_PTY_SUPPORT 0

1. opal/util/stacktrace.c

FPE_FLTINV not present in signal.h

workaround: edit opal_config.h to turn off

OMPI_WANT_PRETTY_PRINT_STACKTRACE (this can be explicitly configured out
but I don't want to reconfigure because I hacked item 15 above)

1. opal/mca/base/mca_base_open.c

gethostname() -- same as opal/util/output.c, must include hostLib.h

1. opal_progress.c

from opal/event/event.h (that I modified earlier)

cannot find #include <sys/_timeradd.h>

It is in opal/event/compat/sys

workaround: change event.h to include the definitions that are present in _timeradd.h instead of including it.

1. Link errors for opal_wrapper

strcasecmp
strncasecmp

I rolled my own in mca_base_open.c (temporary fix, since we may come across this problem elsewhere in the code).

1. dss_internal.h uses a type 'uint'

Not sure if it's depending on something in the headers, or something it
defined on its own.

I changed it to be just like the header I found somewhere under Linux /usr/include:

#ifdef MCS_VXWORKS
typedef unsigned int uint;
#endif

1. struct iovec definition needed

orte/mca/iof/base/iof_base_fragment.h:45: warning: array type has incomplete element type

#ifdef MCS_VXWORKS
#include <net/uio.h>
#endif

not sure if this is right, or if I should include something like <netBufLib.h> or <ioLib.h>

1. iof_base_setup.c

struct termios not understood

can only find termios.h header in 'diab' area and I'm not using that compiler.

a variable usepty is set to 0 already when OMPI_ENABLE_PTY_SUPPORT is 0.
So, why are we compiling this fragment of code at all? I hacked the file
so that the struct termios code will not get compiled.

1. oob_base_send/recv.c, oob_base_send/recv_nb.c. struct iovec not known.

#ifdef MCS_VXWORKS
#include <net/uio.h>
#endif

1. orte/mca/rmgr/base/rmgr_base_check_context.c:58: error:

`MAXHOSTNAMELEN' undeclared (first use in this function)

#ifdef MCS_VXWORKS
#define MAXHOSTNAMELEN 64
#endif

1. orte/mca/rmgr/base/rmgr_base_check_context.c:58:
   gethostname()

#ifdef MCS_VXWORKS
#include <hostLib.h>
#endif

1. Compile problem

orte/mca/iof/proxy/iof_proxy.h:135: warning: array type has incomplete element type
../../../../../orte/mca/iof/proxy/iof_proxy.h:135: error: field `proxy_iov' has incomplete type

#ifdef MCS_VXWORKS
#include <net/uio.h>
#endif

1. Compile problem

/orte/mca/iof/svc/iof_svc.h:147: warning: array type has incomplete element type
../../../../../orte/mca/iof/svc/iof_svc.h:147: error: field `svc_iov' has incomplete type

#ifdef MCS_VXWORKS
#include <net/uio.h>
#endif

1. Compile problem

../../../../../orte/mca/oob/tcp/oob_tcp_msg.h:66: warning: array type has incomplete element type
../../../../../orte/mca/oob/tcp/oob_tcp_msg.h:66: error: field `msg_iov' has incomplete type
../../../../../orte/mca/oob/tcp/oob_tcp_msg.h: In function `mca_oob_tcp_msg_iov_alloc':
../../../../../orte/mca/oob/tcp/oob_tcp_msg.h:196: error: invalid application of `sizeof' to incomplete type `iovec'

1. Compile problem

../../../../../orte/mca/oob/tcp/oob_tcp.c:344: error: implicit declaration of function `accept'
../../../../../orte/mca/oob/tcp/oob_tcp.c: In function `mca_oob_tcp_create_listen':
../../../../../orte/mca/oob/tcp/oob_tcp.c:383: error: implicit declaration of function `socket'
../../../../../orte/mca/oob/tcp/oob_tcp.c:399: error: implicit declaration of function `bind'
../../../../../orte/mca/oob/tcp/oob_tcp.c:407: error: implicit declaration of function `getsockname'
../../../../../orte/mca/oob/tcp/oob_tcp.c:415: error: implicit declaration of function `listen'
../../../../../orte/mca/oob/tcp/oob_tcp.c: In function `mca_oob_tcp_listen_thread':
../../../../../orte/mca/oob/tcp/oob_tcp.c:459: error: implicit declaration of function `bzero'
../../../../../orte/mca/oob/tcp/oob_tcp.c: In function `mca_oob_tcp_recv_probe':
../../../../../orte/mca/oob/tcp/oob_tcp.c:696: error: implicit declaration of function `send'
../../../../../orte/mca/oob/tcp/oob_tcp.c: In function `mca_oob_tcp_recv_handler':
../../../../../orte/mca/oob/tcp/oob_tcp.c:795: error: implicit declaration of function `recv'
../../../../../orte/mca/oob/tcp/oob_tcp.c: In function `mca_oob_tcp_init':
../../../../../orte/mca/oob/tcp/oob_tcp.c:1087: error: implicit declaration of function `usleep'

This gets rid of most (except bzero and usleep)

#ifdef MCS_VXWORKS
#include <sockLib.h>
#endif

Trying to reconfigure the package so CFLAGS will not include -pedantic.
This is because $WIND_HOME/vxworks-6.3/target/h/string.h has protos for
bzero, but only when #if _EXTENSION_WRS is true. So turn off
-ansi/-pedantic gets this? In my dreams?

[ompiteam]

Imported from trac issue 1010. Created by jsquyres on 2007-05-01T10:16:21, last modified: 2008-06-24T11:50:36

[ompiteam]

Trac comment by brbarret on 2007-07-09 23:47:02:

(In [15320]) Fix a bunch of issues exposed by Ken Cain in getting Open MPI to work with
VxWorks. Still some issues remaining, I'm sure.

Refs https://svn.open-mpi.org/trac/ompi/ticket/1010

[ompiteam]

Trac comment by jsquyres on 2008-06-24 11:50:36:

These won't all be fixed in time for v1.3 without an externally-submitted patch.

[rhc54]

No longer supported