Hash: SHA1

             AUSCERT External Security Bulletin Redistribution

                 Memory leak in WAS 8.5.x J2C PoolManager
                               23 April 2014


        AusCERT Security Bulletin Summary

Product:           IBM WebSphere Application Server
Publisher:         IBM
Operating System:  z/OS
Impact/Access:     Denial of Service -- Remote/Unauthenticated
Resolution:        Patch/Upgrade

Original Bulletin: 

- --------------------------BEGIN INCLUDED TEXT--------------------

Memory leak in WAS 8.5.x J2C PoolManager

Document information

More support for:
WebSphere Application Server
Out of Memory

Software version:
8.5, 8.5.5

Operating system(s):
AIX, HP-UX, IBM i, Linux, Mac OS, Solaris, Windows, i5/OS, z/OS

Reference #:

Modified date:

Flash (Alert)


High Garbage Collection overhead and possible Out Of Memory condition when 
using the MQ JMS Resource adapter on WebSphere Application Server V8.5.x


A memory leak in the J2C PoolManager can potentially lead to an OutOfMemory 
(OOM) condition.

Symptoms may include high CPU usage and the following FFDC:
FFDC Exception:java.lang.Exception
SourceId:com.ibm.ejs.j2c.PoolManager$2 ProbeId:50 Reporter:com.ibm.ejs.j2c.PoolManager$@ab1234ef
java.lang.Exception: WSThreadLocal: instance count = 200 ; potential memory leak; verify usage.

Heapanalyzer might show a leak suspect with a stack trace similar to this:

A Linked List Data Structure Detected.                              
1,981,662,352 bytes (47.28 %) of Java heap is used by 44 instances of   java/util/concurrent/ConcurrentSkipListMap$Index                

Contains the following object:- 44 instances of java/util/concurrent/ConcurrentSkipListMap$Node holding 5,366,608 bytes                                          
TotalSize [ObjectSize] NumberOfChildObject(249,874) ObjectName Address    
|- 1,981,966,928 [24] 1 java/util/concurrent/ConcurrentSkipListMap$HeadIndex 0xf7766298                                                

   |- 1,981,966,904 [24] 2 java/util/concurrent/ConcurrentSkipListMap$HeadIndex 0xf180b1b8                                                

      |- 1,981,966,544 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0xffa75200                                                

         |- 1,453,166,232 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0xffa751e8                      
         |- 528,679,664 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0x1055a3258                      
         |- 120,624 [24] 3 java/util/concurrent/ConcurrentSkipListMap

This memory leak specifically manifests itself when applications use the JMS 
Queue Connection Factory where each JMS Managed Connection has an associated 
JMS Connection pool and JMS Session pool. When the Managed connection is 
destroyed due to unused or aged timeouts or the connection is stale, then the 
associated JMS Session pool will be stopped/destroyed and the reaper alarms 
associated with the PoolManager should also be cancelled.

The Session pool is being stopped, however, the PoolManager instances 
registered in the alarms never get destroyed and the alarms are repeatedly 
created and cancelled for every reap cycle. This causes the PoolManager 
objects and its associated reaper alarm objects to stay on the heap forever, 
potentially leading to OOM conditions.

This memory leak problem is addressed by APAR PI14746 which is targetted for 
inclusion in WebSphere Application Server fixpack v8.5.5.3.

Cross reference information 
Segment               Product                                 Platform   Version
Application Servers   WebSphere Application Server for z/OS   z/OS       8.5.5, 8.5

- --------------------------END INCLUDED TEXT--------------------

You have received this e-mail bulletin as a result of your organisation's
registration with AusCERT. The mailing list you are subscribed to is
maintained within your organisation, so if you do not wish to continue
receiving these bulletins you should contact your local IT manager. If
you do not know who that is, please send an email to auscert@auscert.org.au
and we will forward your request to the appropriate person.

NOTE: Third Party Rights
This security bulletin is provided as a service to AusCERT's members.  As
AusCERT did not write the document quoted above, AusCERT has had no control
over its content. The decision to follow or act on information or advice
contained in this security bulletin is the responsibility of each user or
organisation, and should be considered in accordance with your organisation's
site policies and procedures. AusCERT takes no responsibility for consequences
which may arise from following or acting on information or advice contained in
this security bulletin.

NOTE: This is only the original release of the security bulletin.  It may
not be updated when updates to the original are made.  If downloading at
a later date, it is recommended that the bulletin is retrieved directly
from the author's website to ensure that the information is still current.

Contact information for the authors of the original document is included
in the Security Bulletin above.  If you have any questions or need further
information, please contact them directly.

Previous advisories and external security bulletins can be retrieved from:


Australian Computer Emergency Response Team
The University of Queensland
Qld 4072

Internet Email: auscert@auscert.org.au
Facsimile:      (07) 3365 7031
Telephone:      (07) 3365 4417 (International: +61 7 3365 4417)
                AusCERT personnel answer during Queensland business hours
                which are GMT+10:00 (AEST).
                On call after hours for member emergencies only.
Comment: http://www.auscert.org.au/render.html?it=1967