bpo-31904 : Add support for VxWorks RTOS #4184

kuhlenough · 2017-10-31T03:13:37Z

This pull request enables cpython to cross-build on VxWorks, it is obsolete and only provided for reference of the scope of changes..
Smaller more manageable PRs against the current master as requested on python-dev will follow.

All doesn't include shared if static build Hardwire asscii encoding for VxWorks default to UTF-8 (like Android) Semicolon as path separator If build is configure'd for static, propigate to Makefile Add detection for oddly named VxWorks OPENSSL and HASH libraries Fix some warnings with unused functions and macro re-def Extra include in faulthandler.c Avoid duplicate macro definition Setup.py complies all VxWorks compatible modules Update some extension modules to compile on VxWorks VxWorks does not have "timezone" and daylight "constants" Limited signal fields in VxWorks undef DATE TIME for VxWorks

kuhlenough · 2017-10-31T03:56:57Z

Travis failures appear completely unrelated to this pull request?

…ries if configured for static build ( i.e. ./configure --enable-shared=no )

(needs additional automake support)

merwok · 2017-11-03T18:37:00Z

setup.py

@@ -516,7 +524,7 @@ def detect_modules(self):
            add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
            add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
        # only change this for cross builds for 3.3, issues on Mageia
-        if cross_compiling:
+        if ( cross_compiling and not host_platform == 'vxworks'):


The extra space after the paren here is probably the source of the patchcheck warning on TravisCI.

serhiy-storchaka · 2017-11-07T07:25:28Z

Modules/mmapmodule.c

@@ -1463,9 +1463,9 @@ PyInit_mmap(void)
 #endif

    setint(dict, "PAGESIZE", (long)my_getpagesize());
-
+#ifndef __VXWORKS__


I suggest to define my_getallocationgranularity to my_getpagesize above like on Unix.

Your correct this is not required.

serhiy-storchaka · 2017-11-07T07:32:29Z

Modules/posixmodule.c

@@ -174,6 +174,7 @@ corresponding Unix manual entries for more information on calls.");
 #define HAVE_FSYNC      1
 #define fsync _commit
 #else
+#ifndef __VXWORKS__


Are all of this function absent? It looks suspicious that all of them are grouped alphabetically.

A couple are present in some configurations of VxWorks, but following comment is dated.
"Unix functions that the configure script doesn't check for"
The automake configure script finds all these without issue. So a better change would be to simply remove all these and let automake do it's job. I'm just not prepared to regression test the 1/2 dozen other OS referenced in this section.

serhiy-storchaka · 2017-11-07T07:34:54Z

Modules/socketmodule.c

@@ -181,6 +181,12 @@ if_indextoname(index) -- return the corresponding interface name\n\
 # endif
 #endif

+#ifdef __VXWORKS__
+# include <ipcom_sock2.h>
+# define gethostbyaddr_r( a1, a2, a3, a4, a5, a6, a7 )  ipcom_gethostbyaddr_r( a1, a2, a3, a4, a5, a6, a7 )


Too long line. Why not just define gethostbyaddr_r to ipcom_gethostbyaddr_r?

# define gethostbyaddr_r ipcom_gethostbyaddr_r

serhiy-storchaka · 2017-11-07T07:36:22Z

Modules/socketmodule.c

@@ -532,7 +538,7 @@ set_error(void)
    return PyErr_SetFromErrno(PyExc_OSError);
 }

-
+#ifndef __VXWORKS__


Is HAVE_HSTRERROR defined?

yes, but VxWorks has no h_errno also

serhiy-storchaka · 2017-11-07T07:49:39Z

setup.py

@@ -851,20 +871,31 @@ def detect_modules(self):
                               ['/usr/kerberos/include'])
            if krb5_h:
                ssl_incs += krb5_h
-        ssl_libs = find_library_file(self.compiler, 'ssl',lib_dirs,
+        if host_platform == 'vxworks':


The code below is mostly duplicated. The only differences are few arguments. Wouldn't be easy to just set platform depending local variables for these arguments? The value for libraries also is used below.

(for Travis checkpatch)

Conflicts: Modules/faulthandler.c Modules/mmapmodule.c

Conflicts: Modules/mmapmodule.c

serhiy-storchaka · 2017-11-09T15:04:15Z

Modules/socketmodule.c

@@ -525,7 +532,7 @@ set_error(void)
    return PyErr_SetFromErrno(PyExc_OSError);
 }

-
+/*#ifndef __VXWORKS__ */


If this no longer needed, remove it.

Modules/socketmodule.c

serhiy-storchaka · 2017-11-09T15:12:56Z

Modules/timemodule.c

@@ -1421,13 +1421,15 @@ PyInit_timezone(PyObject *m) {
 #if defined(HAVE_TZNAME) && !defined(__GLIBC__) && !defined(__CYGWIN__)
    PyObject *otz0, *otz1;
    tzset();
+#ifndef __VXWORKS__
    PyModule_AddIntConstant(m, "timezone", timezone);


Does timezone not exist? What about _timezone?

No timezone, no _timezone, you have to use tzset/tzget

serhiy-storchaka · 2017-11-09T15:22:41Z

configure

@@ -2926,7 +2926,7 @@ $as_echo_n "checking for python interpreter for cross build... " >&6; }
 	    if $interp -c "import sys;sys.exit(not '.'.join(str(n) for n in sys.version_info[:2]) == '$PACKAGE_VERSION')"; then
 	        break
 	    fi
-            interp=


Why this line is removed? Is configure generated from configure.ac? What version of autoconf you have used?

I haven't checked in a regenerated configure yet. What version do you use?

The last stable version is 2.69.

serhiy-storchaka · 2017-11-09T15:23:55Z

setup.py

@@ -503,7 +511,7 @@ def add_gcc_paths(self):

    def detect_math_libs(self):
        # Check for MacOS X, which doesn't need libm.a at all
-        if host_platform == 'darwin':
+        if (host_platform == 'darwin' or host_platform == 'vxworks'):


Parenthesis are redundant.

serhiy-storchaka · 2017-11-09T15:24:08Z

setup.py

@@ -516,7 +524,7 @@ def detect_modules(self):
            add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
            add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
        # only change this for cross builds for 3.3, issues on Mageia
-        if cross_compiling:
+        if (cross_compiling and not host_platform == 'vxworks'):


Parenthesis are redundant.

serhiy-storchaka · 2017-11-11T11:01:50Z

configure

@@ -2926,7 +2926,7 @@ $as_echo_n "checking for python interpreter for cross build... " >&6; }
 	    if $interp -c "import sys;sys.exit(not '.'.join(str(n) for n in sys.version_info[:2]) == '$PACKAGE_VERSION')"; then
 	        break
 	    fi
-            interp=


The last stable version is 2.69.

serhiy-storchaka · 2017-11-11T11:02:58Z

configure.ac

-	*-*-cygwin*)
-		_host_cpu=
-		;;
+        *-*-cygwin*)


Seems this is not indented as above lines.

Remove test code

gvanrossum · 2018-04-16T21:05:56Z

Yes, we'll clean this up the cruff and squash some commits.
I have no expectation that this pull will be merged until some discussion on long term support is agreed upon

Who is "we" here? Are you just a happy VxWorks user or are you working for the org or company that owns it (assume there is such an org or company)?

This reverts commit 7f38637. # Conflicts: # Doc/tools/templates/indexsidebar.html

kuhlenough · 2018-04-16T23:20:06Z

Wind River owns VxWorks. ( https://www.windriver.com/products/vxworks/ It really is the most pervasive OS you've never heard of. ) I'm a manager at Wind River, as side project I contribute VxWorks support to open source projects. I work with the VxWorks Product Manager. The product manager, Michel, wants "official" python support. In reply to bpo-31904 bug/enhancement Victor (vstinner) mentioned we should have a buildbot to run against pull requests as part of any official support. So I hired an intern, Oscar @ooosssososos, to work on that. He's also been working on getting the test suite to pass with VxWorks. ( His work term ends soon, so we just merged his work into my branch and it seems we got some unwanted commits with that :( )

On my side I'd like to get some additional POSIX support into VxWorks, e.g. proper handling of fcntl( ,F_SETFD, FD_CLOEXEC) and that effort may take awhile.

This reverts commit 18463dd. # Conflicts: # Include/pystate.h

This reverts commit 0ad214f.

This reverts commit dd55e39. # Conflicts: # Lib/test/test_support.py

This reverts commit 18463dd.

nascheme · 2019-01-10T19:13:51Z

Lib/multiprocessing/util.py

+        import subprocess
+        try:
+            return subprocess.Popen(args).pid
+        except:


This is not good. First, 'try' is bare 'except' is bad news, hides bugs. Second, you are doing the 'vxworks' test on every function call, for every platform. Instead you could have something like:

def spawnv_passfds(...): ... if 'vxworks' in sys.platform: # a comment explaining why this is needed def spawnv_passfds(...): # implementation of vxworks special version

nascheme · 2019-01-10T19:15:17Z

Lib/platform.py

@@ -753,7 +753,7 @@ def _syscmd_uname(option, default=''):
        return default
    try:
        f = os.popen('uname %s 2> %s' % (option, DEV_NULL))
-    except (AttributeError, OSError):
+    except (AttributeError, OSError, ValueError):


Looks suspicious. Does os.popen() now return different exceptions? I suspect you should fix os.popen() to trap ValueError and return OSError instead. Otherwise everyone who uses os.popen() has to deal with the new exception case.

nascheme · 2019-01-10T19:16:38Z

Lib/posixpath.py

+                 # way to find if a path is absolute. V7COR-3074, F7233
+        if not isinstance(s, str):
+            s = s.decode(sys.getdefaultencoding())
+        return bool(_vxwapi.isAbs(s))


Again, you are doing the _vxworks test on every function call for every platform. Not a huge deal but isabs() might be called a lot. Instead, define a new isabs() on vxworks.

nascheme · 2019-01-10T19:17:42Z

Lib/posixpath.py

+                ind = rest.find(':')
+                ind = 0 if ind < 0 else ind
+                path = rest[:ind]
+                rest = rest[ind:]


Again, runtime testing of _vxworks. Better to define a special _joinrealpath() function at module import time.

nascheme · 2019-01-10T19:21:50Z

Lib/test/test_netrc.py

@@ -138,7 +138,7 @@ def test_file_not_found_in_home(self):
    def test_file_not_found_explicit(self):
        self.assertRaises(FileNotFoundError, netrc.netrc,
                          file='unlikely_netrc')
-
+    @unittest.skipIf('vxworks' in sys.platform, 'VxWorks does not require or have a HOME directory')
    def test_home_not_set(self):


Sprinkling verbose tests like this throughout the code is not the way. Instead, you should define a global flag at the top of the module, named based on the feature that is not on VxWorks. E.g.

_HAVE_HOME = 'vxworks' not in sys.platform

Then, for the test functions:

@unittest.skipIf(not _HAVE_HOME)

Better yet perhaps is to add those flags to a central module, e.g. platform. Doing it this way makes it more clear what is going on. Also, if a new platform is added, the tests for platform specific features becomes easier to understand.

nascheme · 2019-01-10T19:23:59Z

Modules/_multiprocessing/multiprocessing.h

+#  endif
+#  ifndef FALSE
+#    define FALSE 0
+#  endif


Doesn't look correct to me. multiprocessing requires TRUE = 1. I think you should 'undef' the existing TRUE/FALSE and define what multiprocessing expects.

nascheme · 2019-01-10T19:29:15Z

The code quality in this PR is pretty far from the standard needed to get merged into cpython. The number of changes needed to support VxWorks doesn't look too extreme. So, if we do want to support it, I think it can be done without too much pain. Probably we should decide if we are able to support the new platform before putting a lot of effort into polishing the code and getting it to the quality sufficient for merging.

vstinner · 2019-03-27T15:12:17Z

This PR is just too big, it's not possible to review it. I close it.

You should continue the effort that you already started to split such giant PR into "atomic" changes for fix one issue per PR.

kuhlenough · 2019-03-27T17:55:10Z

NP. thanks fro your support Victor

kuhlenough added 5 commits September 15, 2017 11:03

patch configure for VxWorks

10d4405

Any version of python should be OK for cross build.

9d53ef2

undef DATE TIME for VxWorks

db2e0e8

limited signal feilds in VxWorks

cb3aa97

the-knights-who-say-ni added the CLA signed label Oct 31, 2017

bedevere-bot added the awaiting review label Oct 31, 2017

What's new file for bpo-30914

9a92b63

kuhlenough added 4 commits October 31, 2017 12:38

**install** and **commoninstall** targets do not include shared libra…

e997ecc

…ries if configured for static build ( i.e. ./configure --enable-shared=no )

revert Makefile.pre.in static build changes

7faa281

(needs additional automake support)

Merge branch 'master' of https://github.com/kuhlenough/cpython

98d3bdf

EOL whitespace removal

249794c

merwok reviewed Nov 3, 2017

View reviewed changes

kuhlenough added 2 commits November 3, 2017 15:46

Remove space

8bab325

revert change

21457f7

serhiy-storchaka reviewed Nov 7, 2017

View reviewed changes

kuhlenough added 8 commits November 8, 2017 20:36

Merge branch 'master' of https://github.com/kuhlenough/cpython

b6c9012

whitespace

809b73a

(for Travis checkpatch)

Merge remote-tracking branch 'upstream/master'

847515f

Conflicts: Modules/faulthandler.c Modules/mmapmodule.c

pull review changes

4abb38d

Merge branch 'master' of https://github.com/kuhlenough/cpython

08d4b40

Conflicts: Modules/mmapmodule.c

VxWorks has no h_errno reworked

c8c322c

Ignor readelf cross-compile error with VxWorks

fd303e0

ssl detections differences with varibles

93dca1b

serhiy-storchaka reviewed Nov 9, 2017

View reviewed changes

kuhlenough added 3 commits November 9, 2017 15:57

fix indent with patchcheck

288e627

add VxWorks support for os_cpu_count, spawnv, spawnve

c768168

Merge branch 'master' of https://github.com/kuhlenough/cpython

e7bf7e3

serhiy-storchaka reviewed Nov 11, 2017

View reviewed changes

ooosssososos added 4 commits April 16, 2018 16:26

fixed the random problem in vxworks seeding

df4b6ff

change modificitation to skip

39e4d4e

remove #endif

f2682d6

Merge pull request #4 from kuhlenough/oscar

ddde264

Remove test code

Revert "Update Doc build for split off of 3.7 branch"

d0c89e4

This reverts commit 7f38637. # Conflicts: # Doc/tools/templates/indexsidebar.html

terryjreedy removed their request for review April 16, 2018 21:55

Merge branch 'master' of https://github.com/kuhlenough/cpython

69ff954

the-knights-who-say-ni added CLA signed and removed CLA not signed labels Apr 17, 2018

kuhlenough added 3 commits April 17, 2018 10:08

Revert "Fix static linking extensions"

d5127ac

This reverts commit 18463dd. # Conflicts: # Include/pystate.h

Revert "Fix Windows Build"

d0ecc40

This reverts commit 0ad214f.

Revert "Add more extensions to the test"

43aeefc

This reverts commit dd55e39. # Conflicts: # Lib/test/test_support.py

skrah removed their request for review April 17, 2018 14:44

kuhlenough added 3 commits April 17, 2018 10:50

revert linefeed changes

0b620c4

Revert "Fix static linking extensions"

79f497a

This reverts commit 18463dd.

clean-up cruff

cbcfdf9

gpshead removed their request for review September 10, 2018 23:33

nascheme reviewed Jan 10, 2019

View reviewed changes

vstinner closed this Mar 27, 2019

kuhlenough mannequin mentioned this pull request Jan 12, 2024

Python should support VxWorks RTOS #76085

Closed

Uh oh!

bpo-31904 : Add support for VxWorks RTOS #4184

bpo-31904 : Add support for VxWorks RTOS #4184

Uh oh!

Conversation

kuhlenough commented Oct 31, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kuhlenough commented Oct 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gvanrossum commented Apr 16, 2018

Uh oh!

kuhlenough commented Apr 16, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nascheme commented Jan 10, 2019

Uh oh!

vstinner commented Mar 27, 2019

Uh oh!

kuhlenough commented Mar 27, 2019

Uh oh!

Uh oh!

kuhlenough commented Oct 31, 2017 •

edited

Loading