Monday, October 28, 2013

Enabling SOLR search engine for Madisons Store

Enabling SOLR search engine for Madisons Store





I have been working on this article for some time and had to face multiple issues, here are some steps if you want to get going and explore WCS Search (Apache SOLR) which was introduced in FEP1, I have tested this with Feature Pack 4 installation.

You may follow these steps to enable SOLR for Madisons sample store which has been published as a standalone B2C store, you can do the same with Madisons extended sites store as well.
Step 1

Enable Search feature by following instruction from this link
I happened to enable features while upgrading my toolkit to Feature pack 3 and hence did not follow these steps.

You can Enable Store enhacements by using following command
c:\IBM\WCDE_ENT70\bin>enableFeature.bat -DfeatureName=store-enhancements
Ensure you are able to view following output from the URL, this is an indication of successful enablement of SOLR search engine, SOLR engine will initialize the very first time you hit this url, this can be noticed in application server log.

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2GRC8PXXcijzu5gg3aiHfX1UnUdb6De6ZSK8FirTfbpqJ_1fK_ggMzR18EJOBTRbp7rPtpUH5wdSAV80LO3MCcVfqqFol_ZmRgPji-PFoBUO3Yg3hPr8LdNNSlJnwXt85HwhA_ucFh2uM/s640/solr.png




Step 2
Publish the started store included in feature pack to test advanced functionality, for instance Madisons ConsumerDirect Store can be used for testing new features.

2. 1 Publish Madisons Store Archive
Depending on your feature pack version you need to publish the Madisons store in following sequence


For Feature Pack 1 name for the store is Madisons.sar
For Feature Pack 2 and above the store is Madisons-FEP.sar

Publish as shows in the screenshot below
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-xkrITtaIVGFwMFFTqIeJbpyJtrf_RJexRwbyqZjuTIgypxEbynLTFUVwoMHW_YGO8VX7feZVkd3pdxctN6gTU5zDaRZ1hv_-3sIcU83ZL7fZQe8bMM2Hwi-zUHbGdsx0ajgg6tZOZj3g/s640/Madisons1.png

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi285gFlJLfOPs8FO1BJxKk0jpJydwuvLJyiKqkQeRtaci_ZdkPYm5Jdc4ypq7s9Oht2gGX9lxH8II__mJ1OSBgU02fFjHtA7PnfLm1Q0Kc1Du70o9K3darjjzhCjLwJEJ5v-BHTiWYPwbp/s640/Madisons2.png






















2. 2 Publish MadisonsEnhancements Store Archive
In this sample I have published Madisons-FEP.sar as a normal store (not an extended sites store), hence while publishing MadisonsEnhancements SAR file I have selected 
Marketing Store Identifier, catalog and customer facing store identifier as "Madisons-FEP" as shown in the screen shot below.

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhj2tZ3tuP_JStipQeMx7gB33QJDcXVmJSBple2cgZ1jgU343iRoOt2P1gSIL3avFNmbyp6JF6BX-FGa1BLt06jFFZ90SOZUC6M5OsnOErL2VO3DnKNMeMaYu_NY7KL-VZMvqZH7WIIwynq/s640/store_selection.png




















Login to CMC to validate search functionality is visible for the new Store.
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGSEwY6WOm7bYZxOBTCssIuXbDIuQPkBaZVBBvaB42rBruX-yvAiKo5Ji7kDRQvdvjT84Wyvto1SAl7rd6nqSaJ9A1SzYvnvHMf73aRZ3skceqXwySsJGl1b9usETDFqRWtpM-LIucH1DY/s640/cmc_after.png























Now validate search functionality for the new store.
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEV3HsvcBpkU_4dMoL78UKtarN24NKyzxMjzQbOg6tjtg6neam2N79ZzXsQntwDU6m6clv0JBinT7pXLxVyHULEqA8vrnInCDKbC8IN-IxsWII-16vehhtliS9X1bQJqSOouiscUqjxIxj/s640/Madisons3.png



















Optional Steps
I have noticed that the store publish process in previous step will accomplish Step 3, 4 and 5, hence skip these steps is SOLR index was setup in previous step. 

You should notice something similar in store publish logs in Step 2

[5/13/12 20:13:21:447 CDT] 000001dd task          I com.ibm.commerce.search.indexsetup.execution.task.ConfigWCforSolrCatalogEntrySayHelloTask performExecute

**************************************************************
Started configuring WebSphere Commerce for Solr cores
**************************************************************
Master Catalog Id: 10351
Index Type: CatalogEntry
Languages: [en_US]
Step 3

3. 1 Retrieve Master catalog ID
-- For normal store (not an extended sites store)
select * from catalog where IDENTIFIER='MadisonsFEP';

-- For extended sites store
select * from storeent where IDENTIFIER='MadisonsFEP';
--Use the storeent_id as the store_id in the following SQL to find the Catalog Asset store ID of this Extended Site store
select * from storerel where store_id=11001 and streltyp_id=-4 and relatedstore_id not in (11001);
-Get the master catalog ID
select * from storecat where storeent_id=YYYYYY and mastercatalog='1'


https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ6p6TXC3j67iB-POq0yaTq8BYeBqXF27WpnKW5tN2GT4UHW7tOrkMJ6PJVln0VIVUJ4-mCMu9gOEUThbjKdJlbgXGfF0n-lb2-NEI_q4dJ1RlWaZ_fPWHIZ9wNnPJ9Jh5SDq08woPiUl2/s640/storeid.png


3. 2 Stop the test server if it is running


Step 4
Setting up the WebSphere Commerce search index structure for a specific master catalog locally

C:\IBM\WCDE_ENT70\components\foundation\subcomponents\search\bin\setupSearchIndex.bat -masterCatalogId 10001


C:\IBM\WCDE_ENT70\components\foundation\subcomponents\search\log\wc-search-index-setup.log
Should create pre-process XML in C:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10001\Cloudscape


Step 5
Preprocessing the WebSphere Commerce search index data

c:\IBM\WCDE_ENT70\bin>di-preprocess.bat C:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10001\Cloudscape

review C:\IBM\WCDE_ENT70\logs\wc-dataimport-preprocess.log
you should see following message towards the end of log file


Program exiting with exit code: 0.
Data import pre-processing completed successfully with no errors.



Step 6
Build Search Index

Restart the WebSphere Commerce search server after preprocessing is complete
and hit following URL from browser


Ensure you can visit SOLR over http by checking following URL in your browser

Start the test server, this time around you would notice lot of logging related to SOLR search engine in your start up log

[12/16/11 11:41:27:935 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader locateSolrHome No /solr/home in JNDI
[12/16/11 11:41:27:937 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader locateSolrHome using system property solr.solr.home: C:/IBM/WCDE_E~1/search/solr/home
[12/16/11 11:41:27:938 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1/search/solr/home\'
[12/16/11 11:41:28:045 CST] 00000011 CoreContainer I org.apache.solr.core.CoreContainer load loading shared library: C:\IBM\WCDE_E~1\search\solr\home\lib
[12/16/11 11:41:28:078 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1\search\solr\home\default\'
[12/16/11 11:41:28:119 CST] 00000011 SolrConfig    I org.apache.solr.core.SolrConfig initLibs Adding specified lib dirs to ClassLoader
[12/16/11 11:41:28:622 CST] 00000011 Config        I org.apache.solr.core.SolrConfig <init> Loaded SolrConfig: solrconfig.xml
[12/16/11 11:41:28:631 CST] 00000011 IndexSchema   I org.apache.solr.schema.IndexSchema readSchema Reading Solr Schema
[12/16/11 11:41:28:651 CST] 00000011 IndexSchema   I org.apache.solr.schema.IndexSchema readSchema Schema name=example

[12/16/11 11:41:32:055 CST] 00000011 SolrResourceL I org.apache.solr.core.SolrResourceLoader <init> Solr home set to 'C:\IBM\WCDE_E~1\search\solr\home\MC_10051\en_US\CatalogEntry\'
[12/16/11 11:41:32:199 CST] 00000011 SolrConfig    I org.apache.solr.core.SolrConfig initLibs Adding specified lib dirs to ClassLoader

di-buildindex.bat -masterCatalogId masterCatalogId

c:\IBM\WCDE_ENT70\bin>di-buildindex.bat -masterCatalogId 10001

You should notice something similar at the end of this command execution..


May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode
INFO:

-----------------------------------------------

May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode(int)
INFO:
Program exiting with exit code: 0.
Data import process completed successfully with no errors.

May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logExitCode
INFO:
-----------------------------------------------


May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logEndDateAndTime
INFO: Data import process ended:Fri May 11 20:33:43 CDT 2012
May 11, 2012 8:33:43 PM com.ibm.commerce.foundation.dataimport.process.DataImpor
tProcessorMain logEndDateAndTime
INFO: Data import process completed in 81.565 seconds.


Reference Links

Setting up WebSphere Commerce search

WebSphere commerce search in starter stores


3 comments:

  1. Thanks Tarunam for listing these steps

    ReplyDelete
  2. Thanks for sharing, nice post! Post really provice useful information! I would like to share more information on the same. Nextbrick, Inc is a technology services firm that provides consulting services. For over 5 years, NextBrick has worked with onboardings, performance tuning, relevancy tuning and general Apache Solr Consulting Services and Lucidworks Fusion support for enterprise search and e-commerce search solutions for Fortune 500 companies.

    ReplyDelete
  3. Your Post really provice useful information! I would like to share more information on the same. Nextbrick, Inc is a technology services firm that provides consulting services. For over 5 years, NextBrick has worked with onboardings, performance tuning, relevancy tuning and general Apache Solr support Services and Lucidworks Fusion support for enterprise search and e-commerce search solutions for Fortune 500 companies.

    ReplyDelete