Bye bye blogger!

It's been a long time since I blogged last. As I endeavor to blog more often, I've switched myself to wordpress site. Please follow me here. Thank you!.

Getting around Tomcat out-of-memory issues

“Server running out of memory” is one of the common causes for webserver crashing experienced by most of the Web Application Admins, yet it’s really easy for anyone to determine it happens due to java heap space limitation (JVM memory allocation).  This could lead to false conclusions such as application needing more heap space or application should be clustered/load balanced to provide high availability. On the other hand, when the service is running out of memory it could appear as a memory leak within the application and eventually it can end up with development team testing it.

This post covers simple steps to fine-tune your system memory according to your application requirement, prior to deciding on any infrastructure changes that would be considered.

OutOfMemoryError

JVM throws this error when there is no space in memory for JVM to continue working. This happens mostly while you try to create more objects when there is no space in the heap to allocate as needed.

There are two main causes for this error.

1.  Java heap space limitation.
2.  Java perm gen limitation

Java heap limitation

This is the most commonly known fact related to the out-of-memory issue. You can tweak heap space settings using –Xmx (Maximum heap size) and –Xms (Initial heap size) parameters. In general, if you have more memory the less frequent it needs to be garbage collected. So you could expect applications to run faster when there is more memory available. But from GC’s perspective it will have to work on a larger memory than usual. It’s just like for a floor cleaner you have a bigger floor to clean than he usually does. This in turn can decrease overall performance of your application so you should tweak these settings consciously.

Example :
    -Xmx1536m
    -Xms1024m

Java Permanent Generation limitation

This is where JVM store meta data information about classes, methods (on Eden heap and Tenured heap) and other java primitives. By default this value is set to 64m. Most of the time it’s not sufficient for many applications. So it’s really important to change this setting and provide some space for perm gen to work on. Unlike xmx and xms, you have to use “=” to define this setting. Also you can use additional JVM flags “-XX:+CMSClassUnloadingEnabled” and “-XX:+CMSPermGenSweepingEnabled”. This will enforce sweeping through the permanent generation to release unused references to some extent.

Example :
    -XX:PermSize=64M
    -XX:MaxPermSize=256m
    -XX:+CMSPermGenSweepingEnabled
    -XX:+CMSClassUnloadingEnabled

Generally many Application Admins are aware of heap space settings. Yet there are many instances where perm gen settings have gone unnoticed. It’s really important to fine-tune your application for permanent generation heap as it could be a good reason for your out-of-memory errors

Apache Solr Install Notes

I started working on full text searching solution for our map application. No surprise, Apache Solr is the selected solution and here are some quick notes for you to get started with it.

Part 1 - Basic Setup

1. Download Solr from: http://www.apache.org/dyn/closer.cgi/lucene/solr/

2. Unzip to: C:\Program Files\Apache Software Foundation\Solr (ensure that you rename the solr download to Solr)

3. Stop Tomcat

4. Copy C:\Program Files\Apache Software Foundation\Solr\dist\apache-solr-1.4.1.war to: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps

5 Rename: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\apache-solr-1.4.1.war to Solr.war

6. Open: C:\Program Files\Apache Software Foundation\Tomcat 6.0\bin\tomcat6w.exe goto Java tab and add Java options: -Dsolr.solr.home=C:\Program Files\Apache Software Foundation\Solr\example\solr

7. Start Tomcat and go to: http://localhost:8080/Solr/admin/

Part 2 - Configuring and running the DataImportHandler 

1. Added the following to file: C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\solrconfig.xml

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">

    <lst name="defaults">

      <str name="config">data-config.xml</str>

    </lst>

</requestHandler>

2. Downloaded sqljdbc.jar and sqljdbc4.jar into C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\solr\WEB-INF\lib

3.Create data-config.xml in C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\

   Add to data-config.xml: 

<dataConfig>

 <dataSource type="JdbcDataSource"

  driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"

  url="jdbc:sqlserver://localhost\\sqlserver08;databaseName=Business Objects"

  user="sa" password="sqlserver08" />

 <document>

  <entity name="address"

   query="select OBJECTID,HOUSENO,StreetName,Suburb from Address">

   <field column="OBJECTID" name="id" />

   <field column="HOUSENO" name="house_number" />

   <field column="StreetName" name="street" />

   <field column="Suburb" name="Suburb" />

  </entity>

 </document>

</dataConfig> 

4. Added to schema.xml (C:\Program Files\Apache Software Foundation\Solr\example\solr\conf) in:

   <field name="house_number" type="string" indexed="true" stored="true"/>

   <field name="street" type="string" indexed="true" stored="true"/>

   <field name="Suburb" type="string" indexed="true" stored="true"/>

5. Run: http://localhost:8080/Solr/dataimport?command=full-import 

    You can use the interface at :  http://localhost:8080/Solr/admin/dataimport.jsp?handler=/dataimport

6. Test: http://localhost:8080/Solr/select?q=TYTUS or use http://localhost:8080/Solr/admin/form.jsp

Part 3 - Querying 

http://localhost:8088/solr/db/select/?q=house_number:41&version=2.2&start=0&rows=50&indent=on

Faceting:  http://localhost:8080/Solr/select?indent=on&version=2.2&q=house_number:41&facet=true&facet.field=Suburb

You can facet on multiple fields by appending &facet.field=<field name> The key to faceting is giving all your documents a category or type.

Test Drive ...

I just posted some of the new pics taken with my new DSLR. Follow me on flicker. Watch out for more actions ;)

To be or not to be GWT?

With the current project that I’m working on, I had to work out some GWT stuff to get specific UI level requirements done. In fact I’m not 100% convinced the way it was done that started pinching me to explore more about it.

This is the first time I touched GWT in real live project development. Being java developer for a long time, I have never come across this much of complexity at the UI level especially when you have been dealing with latest technologies and tools. Basically this system has several layers of wrappers before it really feed with back end calls. Also it contains an in-house written configurable UI Binder that uses xml file to configure the UI. According to my understanding this was done to make reusable and scalable UI level components. But I’m not sure if a software house should invest this much of resources to make front end reusable and scalable as these front end tools can be easily replaced by a new technology that is yet to be arrived in near future. As of now, with GWT 2.0 there is a UI Binder feature which allows you to configure your UI in xml files. In fact I found out that smart-client offers you same kind of configurable UI. So all effort they put to make UI configurable is in vain with this kind of releases. Another disadvantage I see is the expertise required to develop the front end. Front end development supposed to be something different so that it requires different mindset with creativity. Obviously GWT development requires experienced java developers, not the creative designers :). So are we doing Swing development for the web? I really feel someone should have considered writing a tool to convert swing applications in to GWT type of front end applications which could have added more value, considering the power it brings to the web applications. There are number of cool features that you can enjoy with GWT based front end, providing an end to the all time headache "browser compatibility".

In my readings, I came across Flex which has similar kind of UI dynamic binding scripts but something easier to deal with. There are so many pros and cons with both GWT and Flex. There is a good comparison here
- More readings.
- Loading speed comparison

ESB competition,This is how it began...

"Ultra ESB", another class product from a Sri Lankan has been released. They've published the results of the Round 4 benchmark testing. Its interesting to wait and see how vendors such as Mule, BEA, WSO2 reacts on this. http://esbperformance.org/wiki/ESB_Performance_Testing

Another update from Chrome

http://www.infoq.com/news/2010/02/Web-SQL-Database