Skip to content

WebSphere Liberty deployment tests timeout on Concourse when using Java 10 #13141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wilkinsona opened this issue May 11, 2018 · 1 comment
Closed

WebSphere Liberty deployment tests timeout on Concourse when using Java 10 #13141

wilkinsona opened this issue May 11, 2018 · 1 comment
Labels
type: task A general task
Milestone
2.0.3

Comments

@wilkinsona
Copy link
Member

wilkinsona commented May 11, 2018 •
edited
Loading

Looking at messages.log from the server, it appears that the tests have completed and the test application has been undeployed. Liberty is then failing to stop:

[5/11/18, 7:45:12:156 UTC] 0000003f SystemOut                                                    O 2018-05-11 07:45:12.156  INFO 935 --- [cutor-thread-26] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 39 ms
[5/11/18, 7:45:12:156 UTC] 0000003f com.ibm.ws.webcontainer.servlet                              I SRVE0242I: [bootapp] [/bootapp] [dispatcherServlet]: Initialization successful.
[5/11/18, 7:45:12:170 UTC] 0000003f SystemOut                                                    O 2018-05-11 07:45:12.169  INFO 935 --- [cutor-thread-26] com.ibm.ws.webcontainer.servlet          : SRVE0242I: [bootapp] [/bootapp] [dispatcherServlet]: Initialization successful.
[5/11/18, 7:45:21:781 UTC] 00000012 com.ibm.ws.kernel.launch.internal.ServerCommandListener      A CWWKE0055I: Server shutdown requested on Friday, May 11, 2018 at 7:45 AM. The server defaultServer is shutting down.
[5/11/18, 7:45:21:786 UTC] 0000007c com.ibm.ws.runtime.update.internal.RuntimeUpdateManagerImpl  A CWWKE1100I: Waiting for up to 30 seconds for the server to quiesce.
[5/11/18, 7:45:21:834 UTC] 0000007d com.ibm.ws.tcpchannel.internal.TCPChannel                    I CWWKO0220I: TCP Channel defaultHttpEndpoint-ssl has stopped listening for requests on host localhost  (IPv4: 127.0.0.1) port 9443.
[5/11/18, 7:45:21:838 UTC] 0000007d com.ibm.ws.tcpchannel.internal.TCPChannel                    I CWWKO0220I: TCP Channel defaultHttpEndpoint has stopped listening for requests on host localhost  (IPv4: 127.0.0.1) port 38148.
[5/11/18, 7:45:21:838 UTC] 0000007d com.ibm.ws.http.internal.VirtualHostImpl                     A CWWKT0017I: Web application removed (default_host): http://localhost:38148/bootapp/
[5/11/18, 7:45:21:839 UTC] 0000007d com.ibm.ws.http.internal.VirtualHostImpl                     A CWWKT0017I: Web application removed (default_host): http://localhost:38148/cargocpc/
[5/11/18, 7:45:51:795 UTC] 0000007c com.ibm.ws.runtime.update.internal.RuntimeUpdateManagerImpl  W CWWKE1102W: The quiesce operation did not complete. The server will now stop.

I've tried to get a thread dump from the server process but have been unable to do so. kill -3 doesn't produce anything in any of Liberty's log files (assuming that standard out is redirected somewhere), and jstack times out. I can get a thread dump from the process that's issued the stop command, but it doesn't appear to tell us much:

2018-05-11 08:31:58
Full thread dump OpenJDK 64-Bit Server VM (10.0.1+10-Debian-3 mixed mode):

Threads class SMR info:
_java_thread_list=0x00007f44b40028e0, length=12, elements={
0x00007f453c013000, 0x00007f453c2b9000, 0x00007f453c2bb000, 0x00007f453c2ce000,
0x00007f453c2d0000, 0x00007f453c2d2000, 0x00007f453c2d3800, 0x00007f453c2d5800,
0x00007f453c2d7000, 0x00007f453c370800, 0x00007f453c3e2000, 0x00007f44b4001000
}

"main" #1 prio=5 os_prio=0 tid=0x00007f453c013000 nid=0x565 runnable  [0x00007f4543ea8000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.FileDispatcherImpl.read0([email protected]/Native Method)
	at sun.nio.ch.SocketDispatcher.read([email protected]/SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer([email protected]/IOUtil.java:283)
	at sun.nio.ch.IOUtil.read([email protected]/IOUtil.java:250)
	at sun.nio.ch.IOUtil.read([email protected]/IOUtil.java:226)
	at sun.nio.ch.SocketChannelImpl.read([email protected]/SocketChannelImpl.java:382)
	- locked <0x000000065eadbbd8> (a java.lang.Object)
	at com.ibm.ws.kernel.boot.internal.ServerCommand.read(ServerCommand.java:74)
	at com.ibm.ws.kernel.boot.internal.commands.ServerCommandClient.write(ServerCommandClient.java:123)
	at com.ibm.ws.kernel.boot.internal.commands.ServerCommandClient.stopServer(ServerCommandClient.java:237)
	at com.ibm.ws.kernel.boot.internal.commands.ProcessControlHelper.stop(ProcessControlHelper.java:97)
	at com.ibm.ws.kernel.boot.Launcher.handleActions(Launcher.java:265)
	at com.ibm.ws.kernel.boot.Launcher.createPlatform(Launcher.java:121)
	at com.ibm.ws.kernel.boot.cmdline.EnvCheck.main(EnvCheck.java:59)
	at com.ibm.ws.kernel.boot.cmdline.EnvCheck.main(EnvCheck.java:35)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f453c2b9000 nid=0x583 waiting on condition  [0x00007f44e8970000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList([email protected]/Native Method)
	at java.lang.ref.Reference.processPendingReferences([email protected]/Reference.java:174)
	at java.lang.ref.Reference.access$000([email protected]/Reference.java:44)
	at java.lang.ref.Reference$ReferenceHandler.run([email protected]/Reference.java:138)

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f453c2bb000 nid=0x584 in Object.wait()  [0x00007f44e886f000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <0x000000065ed09480> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:151)
	- waiting to re-lock in wait() <0x000000065ed09480> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:172)
	at java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:216)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f453c2ce000 nid=0x585 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f453c2d0000 nid=0x586 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f453c2d2000 nid=0x587 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007f453c2d3800 nid=0x588 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x00007f453c2d5800 nid=0x589 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #9 daemon prio=9 os_prio=0 tid=0x00007f453c2d7000 nid=0x58a runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 tid=0x00007f453c370800 nid=0x58b in Object.wait()  [0x00007f449fefd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <0x000000065eb082a8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:151)
	- waiting to re-lock in wait() <0x000000065eb082a8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run([email protected]/CleanerImpl.java:148)
	at java.lang.Thread.run([email protected]/Thread.java:844)
	at jdk.internal.misc.InnocuousThread.run([email protected]/InnocuousThread.java:134)

"Service Thread" #11 daemon prio=9 os_prio=0 tid=0x00007f453c3e2000 nid=0x58c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" #12 daemon prio=9 os_prio=0 tid=0x00007f44b4001000 nid=0x11b5 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 tid=0x00007f453c2af000 nid=0x57f runnable

"GC Thread#0" os_prio=0 tid=0x00007f453c029800 nid=0x566 runnable

"GC Thread#1" os_prio=0 tid=0x00007f453c02b000 nid=0x567 runnable

"GC Thread#2" os_prio=0 tid=0x00007f453c02c800 nid=0x568 runnable

"GC Thread#3" os_prio=0 tid=0x00007f453c02e800 nid=0x569 runnable

"GC Thread#4" os_prio=0 tid=0x00007f453c030000 nid=0x56a runnable

"GC Thread#5" os_prio=0 tid=0x00007f453c032000 nid=0x56b runnable

"GC Thread#6" os_prio=0 tid=0x00007f453c033800 nid=0x56c runnable

"GC Thread#7" os_prio=0 tid=0x00007f453c035000 nid=0x56d runnable

"G1 Main Marker" os_prio=0 tid=0x00007f453c0a9000 nid=0x56e runnable

"G1 Conc#0" os_prio=0 tid=0x00007f453c0ab000 nid=0x56f runnable

"G1 Conc#1" os_prio=0 tid=0x00007f453c0ac800 nid=0x570 runnable

"G1 Refine#0" os_prio=0 tid=0x00007f453c214800 nid=0x571 runnable

"G1 Refine#1" os_prio=0 tid=0x00007f453c216800 nid=0x572 runnable

"G1 Refine#2" os_prio=0 tid=0x00007f453c218000 nid=0x573 runnable

"G1 Refine#3" os_prio=0 tid=0x00007f453c21a000 nid=0x574 runnable

"G1 Refine#4" os_prio=0 tid=0x00007f453c21b800 nid=0x575 runnable

"G1 Refine#5" os_prio=0 tid=0x00007f453c21d800 nid=0x576 runnable

"G1 Refine#6" os_prio=0 tid=0x00007f453c21f000 nid=0x577 runnable

"G1 Refine#7" os_prio=0 tid=0x00007f453c221000 nid=0x578 runnable

"G1 Young RemSet Sampling" os_prio=0 tid=0x00007f453c222800 nid=0x579 runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007f453c3e4000 nid=0x58d waiting on condition

JNI global references: 14
@wilkinsona
Copy link
Member Author

There's an open issue for Java 9 support so it looks like we're being overly ambitious here.

@wilkinsona wilkinsona added this to the 2.0.3 milestone May 11, 2018
@wilkinsona wilkinsona added the type: task A general task label May 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: task A general task
Projects
None yet
Milestone
2.0.3
Development

No branches or pull requests
1 participant
@wilkinsona