Apache Solr Install Notes

I started working on full text searching solution for our map application. No surprise, Apache Solr is the selected solution and here are some quick notes for you to get started with it.

Part 1 - Basic Setup

1. Download Solr from: http://www.apache.org/dyn/closer.cgi/lucene/solr/

2. Unzip to: C:\Program Files\Apache Software Foundation\Solr (ensure that you rename the solr download to Solr)

3. Stop Tomcat

4. Copy C:\Program Files\Apache Software Foundation\Solr\dist\apache-solr-1.4.1.war to: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps

5 Rename: C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\apache-solr-1.4.1.war to Solr.war

6. Open: C:\Program Files\Apache Software Foundation\Tomcat 6.0\bin\tomcat6w.exe goto Java tab and add Java options: -Dsolr.solr.home=C:\Program Files\Apache Software Foundation\Solr\example\solr

7. Start Tomcat and go to: http://localhost:8080/Solr/admin/

Part 2 - Configuring and running the DataImportHandler 

1. Added the following to file: C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\solrconfig.xml

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">

    <lst name="defaults">

      <str name="config">data-config.xml</str>

    </lst>

</requestHandler>

2. Downloaded sqljdbc.jar and sqljdbc4.jar into C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\solr\WEB-INF\lib

3.Create data-config.xml in C:\Program Files\Apache Software Foundation\Solr\example\solr\conf\

   Add to data-config.xml: 

<dataConfig>

 <dataSource type="JdbcDataSource"

  driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"

  url="jdbc:sqlserver://localhost\\sqlserver08;databaseName=Business Objects"

  user="sa" password="sqlserver08" />

 <document>

  <entity name="address"

   query="select OBJECTID,HOUSENO,StreetName,Suburb from Address">

   <field column="OBJECTID" name="id" />

   <field column="HOUSENO" name="house_number" />

   <field column="StreetName" name="street" />

   <field column="Suburb" name="Suburb" />

  </entity>

 </document>

</dataConfig> 

4. Added to schema.xml (C:\Program Files\Apache Software Foundation\Solr\example\solr\conf) in:

   <field name="house_number" type="string" indexed="true" stored="true"/>

   <field name="street" type="string" indexed="true" stored="true"/>

   <field name="Suburb" type="string" indexed="true" stored="true"/>

5. Run: http://localhost:8080/Solr/dataimport?command=full-import 

    You can use the interface at :  http://localhost:8080/Solr/admin/dataimport.jsp?handler=/dataimport

6. Test: http://localhost:8080/Solr/select?q=TYTUS or use http://localhost:8080/Solr/admin/form.jsp

Part 3 - Querying 

http://localhost:8088/solr/db/select/?q=house_number:41&version=2.2&start=0&rows=50&indent=on

Faceting:  http://localhost:8080/Solr/select?indent=on&version=2.2&q=house_number:41&facet=true&facet.field=Suburb

You can facet on multiple fields by appending &facet.field=<field name> The key to faceting is giving all your documents a category or type.

Test Drive ...

I just posted some of the new pics taken with my new DSLR. Follow me on flicker. Watch out for more actions ;)

To be or not to be GWT?

With the current project that I’m working on, I had to work out some GWT stuff to get specific UI level requirements done. In fact I’m not 100% convinced the way it was done that started pinching me to explore more about it.

This is the first time I touched GWT in real live project development. Being java developer for a long time, I have never come across this much of complexity at the UI level especially when you have been dealing with latest technologies and tools. Basically this system has several layers of wrappers before it really feed with back end calls. Also it contains an in-house written configurable UI Binder that uses xml file to configure the UI. According to my understanding this was done to make reusable and scalable UI level components. But I’m not sure if a software house should invest this much of resources to make front end reusable and scalable as these front end tools can be easily replaced by a new technology that is yet to be arrived in near future. As of now, with GWT 2.0 there is a UI Binder feature which allows you to configure your UI in xml files. In fact I found out that smart-client offers you same kind of configurable UI. So all effort they put to make UI configurable is in vain with this kind of releases. Another disadvantage I see is the expertise required to develop the front end. Front end development supposed to be something different so that it requires different mindset with creativity. Obviously GWT development requires experienced java developers, not the creative designers :). So are we doing Swing development for the web? I really feel someone should have considered writing a tool to convert swing applications in to GWT type of front end applications which could have added more value, considering the power it brings to the web applications. There are number of cool features that you can enjoy with GWT based front end, providing an end to the all time headache "browser compatibility".

In my readings, I came across Flex which has similar kind of UI dynamic binding scripts but something easier to deal with. There are so many pros and cons with both GWT and Flex. There is a good comparison here
- More readings.
- Loading speed comparison

ESB competition,This is how it began...

"Ultra ESB", another class product from a Sri Lankan has been released. They've published the results of the Round 4 benchmark testing. Its interesting to wait and see how vendors such as Mule, BEA, WSO2 reacts on this. http://esbperformance.org/wiki/ESB_Performance_Testing

Another update from Chrome

http://www.infoq.com/news/2010/02/Web-SQL-Database

Lets make the web a better place

Speed of the web matters everyone who browse over the internet. Internet has come a long way to reach the level of speed that possess today. Still it demands a better and better speed to make possible many great things that are restricted due to slow internet speed. Google has put a step forward to solve this for the world. It’s a great effort, as people who build the web we all should contribute. Please refer: http://code.google.com/speed/articles/

Caching Service

While I was working on my last assignment, there was a need for implementing good caching solution for the company. This has been a long time headache as we did not utilize proper caching mechanism to cache our frequently used data. Only single component that we were using had many draw backs and it was so much restricted cache reservation details only. The highly coupled nature of that component sometime made trouble shooting a nightmare for developers. As I was assigned to work on new searching component, with my refactoring cycles I had to remove most of the highly coupled back-end logics ultimately existing component became obsolete. Therefore I started working on new component and designed it in such a way where we can use it as a service.

This Caching Service has been developed to facilitate data caching needs required not only by our system, but also any JEE based system. When talking about Caching service, a framework is essential to build and govern the infrastructure of it. Therefore I came up with a framework that accommodates existing caching needs with extensibility in mind.

This is a very powerful framework that can be used to implement new caching mechanisms from the scratch or it can be used as an interface for 3rd party caching engine (MemCache, JbossCache, OSCache … etc). It has a user friendly monitoring interface with functionality rich including the flushing results, flush contains by systems connected, view particular requests, delete particular request … etc.

The beauty of this is it has a central control over all systems connected to caching engine and you can govern them individually. You can have multiple systems connected to multiple caching engines, but you can govern and monitor them in a central location. You are free to add new caching solutions written by your own or you can plug new caching engines that use the same set of operations.

I have an idea of sharing something same as this framework to the open source community let’s see what kind of improved version I can come up with.