Enabling SOLR search engine for Madisons Store

Enabling SOLR search engine for Madisons Store

I have been working on this article for some time and had to face multiple issues, here are some steps if you want to get going and explore WCS Search (Apache SOLR) which was introduced in FEP1, I have tested this with Feature Pack 4 installation.

You may follow these steps to enable SOLR for Madisons sample store which has been published as a standalone B2C store, you can do the same with Madisons extended sites store as well.

Step 1

Enable Search feature by following instruction from this link

http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.developer.doc/tasks/tsdsearchbuilddeployfirst.htm

I happened to enable features while upgrading my toolkit to Feature pack 3 and hence did not follow these steps.

You can Enable Store enhacements by using following command

c:\IBM\WCDE_ENT70\bin>enableFeature.bat -DfeatureName=store-enhancements

Ensure you are able to view following output from the URL, this is an indication of successful enablement of SOLR search engine, SOLR engine will initialize the very first time you hit this url, this can be noticed in application server log.

http://localhost/solr/Default/select?q=*%3A*

Step 2

Publish the started store included in feature pack to test advanced functionality, for instance Madisons ConsumerDirect Store can be used for testing new features.

2. 1 Publish Madisons Store Archive

Depending on your feature pack version you need to publish the Madisons store in following sequence


For Feature Pack 1 name for the store is Madisons.sar
For Feature Pack 2 and above the store is Madisons-FEP.sar

Publish as shows in the screenshot below

2. 2 Publish MadisonsEnhancements Store Archive

In this sample I have published Madisons-FEP.sar as a normal store (not an extended sites store), hence while publishing MadisonsEnhancements SAR file I have selected 

Marketing Store Identifier, catalog and customer facing store identifier as "Madisons-FEP" as shown in the screen shot below.

Login to CMC to validate search functionality is visible for the new Store.

Now validate search functionality for the new store.

Optional Steps

I have noticed that the store publish process in previous step will accomplish Step 3, 4 and 5, hence skip these steps is SOLR index was setup in previous step. 

You should notice something similar in store publish logs in Step 2

[5/13/12 20:13:21:447 CDT] 000001dd task          I com.ibm.commerce.search.indexsetup.execution.task.ConfigWCforSolrCatalogEntrySayHelloTask performExecute

**************************************************************

Started configuring WebSphere Commerce for Solr cores

**************************************************************

Master Catalog Id: 10351

Index Type: CatalogEntry

Languages: [en_US]

Step 3

3. 1 Retrieve Master catalog ID

-- For normal store (not an extended sites store)

select * from catalog where IDENTIFIER='MadisonsFEP';

-- For extended sites store

select * from storeent where IDENTIFIER='MadisonsFEP';

--Use the storeent_id as the store_id in the following SQL to find the Catalog Asset store ID of this Extended Site store

select * from storerel where store_id=11001 and streltyp_id=-4 and relatedstore_id not in (11001);

-Get the master catalog ID

select * from storecat where storeent_id=YYYYYY and mastercatalog='1'

3. 2 Stop the test server if it is running

Step 4

Setting up the WebSphere Commerce search index structure for a specific master catalog locally

C:\IBM\WCDE_ENT70\components\foundation\subcomponents\search\bin\setupSearchIndex.bat -masterCatalogId 10001

C:\IBM\WCDE_ENT70\components\foundation\subcomponents\search\log\wc-search-index-setup.log

Should create pre-process XML in C:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10001\Cloudscape

Step 5

Preprocessing the WebSphere Commerce search index data

c:\IBM\WCDE_ENT70\bin>di-preprocess.bat C:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10001\Cloudscape

review C:\IBM\WCDE_ENT70\logs\wc-dataimport-preprocess.log

you should see following message towards the end of log file


Program exiting with exit code: 0.
Data import pre-processing completed successfully with no errors.

Step 6

Build Search Index

Restart the WebSphere Commerce search server after preprocessing is complete
and hit following URL from browser


Ensure you can visit SOLR over http by checking following URL in your browser

http://localhost/solr/Default/select?q=*%3A*

Start the test server, this time around you would notice lot of logging related to SOLR search engine in your start up log

[12/16/11 11:41:27:935 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader locateSolrHome No /solr/home in JNDI

[12/16/11 11:41:27:937 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader locateSolrHome using system property solr.solr.home: C:/IBM/WCDE_E~1/search/solr/home

[12/16/11 11:41:27:938 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1/search/solr/home\'

[12/16/11 11:41:28:045 CST] 00000011 CoreContainer I org.apache.solr.core.CoreContainer load loading shared library: C:\IBM\WCDE_E~1\search\solr\home\lib

[12/16/11 11:41:28:078 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1\search\solr\home\default\'

[12/16/11 11:41:28:119 CST] 00000011 SolrConfig    I org.apache.solr.core.SolrConfig initLibs Adding specified lib dirs to ClassLoader

[12/16/11 11:41:28:622 CST] 00000011 Config        I org.apache.solr.core.SolrConfig <init> Loaded SolrConfig: solrconfig.xml

[12/16/11 11:41:28:631 CST] 00000011 IndexSchema   I org.apache.solr.schema.IndexSchema readSchema Reading Solr Schema

[12/16/11 11:41:28:651 CST] 00000011 IndexSchema   I org.apache.solr.schema.IndexSchema readSchema Schema name=example

[12/16/11 11:41:32:055 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1\search\solr\home\MC_10051\en_US\CatalogEntry\'

[12/16/11 11:41:32:199 CST] 00000011 SolrConfig    I org.apache.solr.core.SolrConfig initLibs Adding specified lib dirs to ClassLoader

di-buildindex.bat -masterCatalogId masterCatalogId

c:\IBM\WCDE_ENT70\bin>di-buildindex.bat -masterCatalogId 10001

You should notice something similar at the end of this command execution..

May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode
INFO:

-----------------------------------------------

May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode(int)
INFO:
Program exiting with exit code: 0.
Data import process completed successfully with no errors.

May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode
INFO:
-----------------------------------------------


May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logEndDateAndTime
INFO: Data import process ended:Fri May 11 20:33:43 CDT 2012
May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logEndDateAndTime
INFO: Data import process completed in 81.565 seconds.

Reference Links

Setting up WebSphere Commerce search on a starter store
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.starterstores.doc/refs/rsdfepfeatroadmap.htm

Setting up WebSphere Commerce search

http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.developer.doc/concepts/csddeploysearchtop.htm

WebSphere commerce search in starter stores

http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.starterstores.doc/concepts/csmsearch.htm