Skip to content

Client segfaults when threads are used #78

Closed
@criccomini

Description

@criccomini

As part of #40, we started noticing that our tests were failing with a segfault in the Travis CI container, but only for Python 3.4.

We were unable to reproduce this locally on our laptops, but with Docker, and using a Travis CI container, we can. Here are the steps to trigger a segfault:

# Start Docker
docker run -it quay.io/travisci/travis-python /bin/bash

Once you're in your Docker container, run these commands:

sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
sudo apt-get install python3.4
pip install pip==7.1.0
pip install virtualenv==13.1.2
virtualenv -p /usr/bin/python3.4 /test
source /test/bin/activate
cd ~
git clone --depth=50 https://github.com/confluentinc/confluent-kafka-python.git
cd confluent-kafka-python/
git fetch origin pull/40/head:40
git checkout 40
export LD_LIBRARY_PATH=$PWD/tmp-build/lib
bash tools/bootstrap-librdkafka.sh v0.9.2 tmp-build
pip install pytest-timeout
pip install -v --global-option=build_ext --global-option="-Itmp-build/include/:/opt/python/3.4.2/include/python3.4m" --global-option="-Ltmp-build/lib" . .[avro]
git checkout 8148a79026d7324c34a24a03ba9043ebe2f931fd
py.test -v --timeout 20 --ignore=tmp-build --import-mode append

This will show a segfault before the tests finish.

Furthermore, I'm able to trigger this on master without any of our changes. After the above commands, run:

git checkout master

Then apply this diff:

diff --git a/tests/test_threads.py b/tests/test_threads.py
index 09f6467..e4f76e8 100644
--- a/tests/test_threads.py
+++ b/tests/test_threads.py
@@ -64,6 +64,35 @@ def test_thread_safety():
 
     print('Done')
 
+import time
+class myThread (threading.Thread):
+    def __init__(self, threadID, name, counter):
+        threading.Thread.__init__(self)
+        self.threadID = threadID
+        self.name = name
+        self.counter = counter
+    def run(self):
+        print("Starting " + self.name)
+        print_time(self.name, self.counter, 5)
+        print("Exiting " + self.name)
+
+def print_time(threadName, delay, counter):
+    while counter:
+        if exitFlag:
+            threadName.exit()
+        time.sleep(delay)
+        print("%s: %s" % (threadName, time.ctime(time.time())))
+        counter -= 1
+
+def test_foo():
+    # Create new threads
+    thread1 = myThread(1, "Thread-1", 1)
+    thread2 = myThread(2, "Thread-2", 2)
+
+    # Start new Threads
+    thread1.start()
+    thread2.start()
+

It's just some example of Python threading that I found on Stack Overflow. After the diff is applied, run the same py.test command again. It will again segfault.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions