Problem Statement
We have a java web service deployed on the WebSphere Application Server 7. Everything was fine till one day when the testing team observed OutOfMemory error in the log file. The web service was developed using Spring MyBatis. The initial code analysis did not show any leak suspects in the code. The OutOfMemory error was coming intermittently since its first occurrence.
Heap Analysis
After the initial code analysis it was more or less clear to us that the culprit for OutOfMemory errors could be spring or MyBatis. While search the internet for heap analyzer tools we came across IBM HeapAnalyzer. It is a nice tool to analyze heap dumps generated by Web Sphere Application Server. For more information on this tool as well as to download its jar please visit
https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=4544bafe-c7a2-455f-9d43-eb866ea60091
HeapAnalyzer helps in identifying the memory leak areas by analyzing the heap dumps. The ‘heap leak suspect’ feature shows you the top 3 to 5 java objects consuming maximum memory. This helps in identifying the part of code which would be causing the out of memory issue. WebSphere Application Server by default generates heap dump whenever an OutOfMemory error is encountered.
The Culprit
When we ran HeapAnalyzer on the heap dump generated for our applications we found that more than 90% of memory is occupied by an ArrayList inside MyBatis DefaultResultHandler. On doing further analysis we found the issue with the default implementation of DefaultResultHandler provided by MyBatis. The default implementation of ResultHandler creates an internal array to store the select query result before returning the result to invoker. In our case some select queries were returning more than 500k of big size records. Now when the DefaultResultHandler is trying to store all these in the ArrayList, the heap was getting exhausted and OutOfMemory error was thrown.
Solution Details
The solution we implemented to overcome the issue was to override the DefaultResultHandler. The idea was to divide the result of select query into small chunks. We override the DefaultResultHandler to load 50000 records in the internal ArrayList and then invoke a callback operation to process those records. When one set of records is processed the ArrayList is emptied and new set of records are stored in it. This way we can restrict the number of objects in memory at any given point of time and avoid OutOfMemory errors.
Below is the sample code for custom ResultHandler that does the job explained above.
public class CustomResultHandler implements ResultHandler {
private ArrayList<HashMap<String,String>> recordList;
private CallbackInterface callbackClass;
private int MAX_COUNT= 100;
public CustomResultHandler() {
super();
this.recordList = new ArrayList<HashMap<String,String>>();
}
@Override
public void handleResult(ResultContext resultContext) {
recordList.add((HashMap<String, String>) resultContext.getResultObject());
if(recordList.size() == MAX_COUNT) {
System.out.println("Invoking Callback method");
callbackClass.processData(recordList);
recordList.clear();
}
}
public ArrayList<HashMap<String,String>> getRecordList() {
return recordList;
}
public void setRecordList(ArrayList<HashMap<String,String>> recordList) {
this.recordList = recordList;
}
public Object getCallbackClass() {
return callbackClass;
}
public void setCallbackClass(CallbackInterface callbackClass) {
this.callbackClass = callbackClass;
}
}
Here the CustomResultHandler implements the ResultHandler interface defined by MyBatis. Its handleResult method is invoked for each record returned from the database by select query.
We need to define a callback interface and implementation class used in the above code. A simple callback interface could be defined as below:
public interface CallbackInterface {
public void processData(ArrayList<HashMap<String,String>> recordList);
}
And the corresponding implementation class could be defined as below:
public class CallbackImpl implements CallbackInterface {
@Override
public void processData(ArrayList<HashMap<String, String>> recordList) {
System.out.println("Callback operation invoked for list size: " + recordList.size());
}
}
Next we need to bind the custom result handler with the mapper. For this we have to use sqlSession.select() methods, to which we pass the result handler to be invoked for each record in the ResultSet returned by the select query.
public ArrayList<HashMap<String, String>> getAllUsers(CustomResultHandler objCustomResultHandler) {
//set the custom result handler in sql session
sqlSession.select("com.memoryleak.poc.persistence.UserMapper.getAllUsers", objCustomResultHandler);
return userMapper.getAllUsers(objCustomResultHandler);
}
Now when getAllUsers is invoked an instance of CustomResultHandler is passed. This enables the custom result handler be invoked by MyBatis instead of DefaultResultHandler. A simple way to invoke this service method is like below:
public static void main(String[] args) {
ApplicationContext context = new ClassPathXmlApplicationContext("Beans.xml");
UserService userService = (UserService) context.getBean("userService");
CustomResultHandler objResultHandler = new CustomResultHandler();
CallbackInterface objCallback = new CallbackImpl();
objResultHandler.setCallbackClass(objCallback);
System.out.println("Users List: " + userService.getAllUsers(objResultHandler));
}
If you configure and run the attached code you will see that processData of CallbackImpl is invoked after every 100 records. We can optimize the record set by changing the MAX_COUNT value in CustomResutlHandler.
References
