VERIFIED SOLUTION i

Resolve java.lang.OutOfMemoryError: unable to create new native thread when creating/submitting workItem in EngageOne

Product Feature: API (Web Service, SDI, etc.)

 

Issue

Creating/submitting a workItem fails in EngageOne Server with the following error:

ERROR | org.apache.activemq.thread.TaskRunnerFactory] Error in thread 'ActiveMQ BrokerService[localhost] Task-301'
java.lang.OutOfMemoryError: unable to create new native thread

Cause

Some Linux distributions have default limits on concurrent threads which can be run by a single user or system. During the start of the composition bundle, the system tries to run more threads than is allowed by the operating system and as it is not possible to start them, the following error is returned in the composition-services.log file:

[2017-11-08 14:48:11,615 | ERROR | org.apache.activemq.thread.TaskRunnerFactory] Error in thread 'ActiveMQ BrokerService[localhost] Task-301'
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method) ~[na:1.8.0_131]
at java.lang.Thread.start(Thread.java:717) ~[na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) ~[na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018) ~[na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) ~[na:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]

Resolution

UPDATED: November 23, 2017
To fix the issue, you need to do the following:

1. Stop the composition bundle.
2. Check the current limit by running the following command:
ulimit –u
3. To create extra process capacity on your system, edit the limits.conf file, located in /etc/security/limits.conf. where it will be necessary to add or modify lines for nproc, for example:
engageOneProcessUser soft nproc 4096
engageOneProcessUser hard nproc 4096
where ‘engageOneProcessUser’ is the Linux user running the EngageOne composition service.
Both soft and hard limits need to be adjusted.
4. Once you modify the limits, start the composition bundle and check whether there are still errors java.lang.OutOfMemoryError: unable to create new native thread.  If the problem still occurs, report this to software.support@pb.com
.