SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye
Issue Details (XML | Word | Printable)

Key: CMP-767
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Shay Banon
Reporter: Adam Lane
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Compass

Emulated BLOB locators must come from a ResultSet with only one table selected, and all primary keys selected

Created: 02/Nov/08 07:58 PM   Updated: 02/Nov/08 08:45 PM
Component/s: Compass::Core
Affects Version/s: 2.1.0 GA
Fix Version/s: None

Environment: mysql 5.0.67, spring and hibernate.


 Description  « Hide
I am trying to use JDBC storage instead of my current file storage and getting the "emulated blob error" in the stack below when I try to create the index.

org.compass.core.engine.SearchEngineException: Failed to create index for sub index [product]; nested exception is org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [select name_, 'value_' as x, size_ from gpsindex_product where name_ = ?]; nested exception is java.sql.SQLException: Emulated BLOB locators must come from a ResultSet with only one table selected, and all primary keys selected

I dont know what the deal is because there is only one table in the sql and the primary key name_ is selected.
I have been trying hard to do my due diligence and figure out what the problem is here and it seems i am doing everything correct according to all the docs and forum threads i have read.

I am using compass.engine.store.jdbc.dialect org.apache.lucene.store.jdbc.dialect.MySQLDialect but I have tried innodb one with the same error.

I am using spring and hibernateGPS but I have emulateLocators=true in the string. (If I don't put emulateLocators I get the ArrayIndexOutOfBoundsException.)
Here is my connection url:
<driver type="com.p6spy.engine.spy.P6SpyDriver">
<url>
jdbc:mysql://localhost/savwebd?emulateLocators=true&autoReconnect=true&useCursorFetch=true&zeroDateTimeBehavior=convertToNull&jdbcCompliantTruncation=false
</url>
<user>root</user>
<password></password>
</driver>

STACK:

org.compass.core.engine.SearchEngineException: Failed to create index for sub index [product]; nested exception is org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [select name_, 'value_' as x, size_ from gpsindex_product where name_ = ?]; nested exception is java.sql.SQLException: Emulated BLOB locators must come from a ResultSet with only one table selected, and all primary keys selected
org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [select name_, 'value_' as x, size_ from gpsindex_product where name_ = ?]; nested exception is java.sql.SQLException: Emulated BLOB locators must come from a ResultSet with only one table selected, and all primary keys selected
java.sql.SQLException: Emulated BLOB locators must come from a ResultSet with only one table selected, and all primary keys selected
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:910)
at com.mysql.jdbc.BlobFromLocator.notEnoughInformationInQuery(BlobFromLocator.java:155)
at com.mysql.jdbc.BlobFromLocator.<init>(BlobFromLocator.java:117)
at com.mysql.jdbc.ResultSet.getNativeBlob(ResultSet.java:3204)
at com.mysql.jdbc.ResultSet.getBlob(ResultSet.java:1437)
at com.mysql.jdbc.ResultSet.getBlob(ResultSet.java:1452)
at com.p6spy.engine.spy.P6ResultSet.getBlob(P6ResultSet.java:658)
at com.p6spy.engine.spy.P6ResultSet.getBlob(P6ResultSet.java:654)
at org.apache.lucene.store.jdbc.index.FetchOnBufferReadJdbcIndexInput$1.execute(FetchOnBufferReadJdbcIndexInput.java:90)
at org.apache.lucene.store.jdbc.support.JdbcTemplate.executeSelect(JdbcTemplate.java:112)
at org.apache.lucene.store.jdbc.index.FetchOnBufferReadJdbcIndexInput.refill(FetchOnBufferReadJdbcIndexInput.java:58)
at org.apache.lucene.store.ConfigurableBufferedIndexInput.readByte(ConfigurableBufferedIndexInput.java:27)
at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:36)
at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:221)
at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:175)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1093)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:698)
at org.compass.core.lucene.engine.store.DefaultLuceneSearchEngineStore.createIndex(DefaultLuceneSearchEngineStore.java:493)
at org.compass.core.lucene.engine.store.DefaultLuceneSearchEngineStore.cleanIndex(DefaultLuceneSearchEngineStore.java:549)
at org.compass.core.lucene.engine.store.DefaultLuceneSearchEngineStore.cleanIndex(DefaultLuceneSearchEngineStore.java:539)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$3.doInTransaction(DefaultLuceneSearchEngineIndexManager.java:133)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$3.doInTransaction(DefaultLuceneSearchEngineIndexManager.java:130)
at org.compass.core.impl.DefaultCompass$CompassTransactionContext.execute(DefaultCompass.java:349)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager.cleanIndex(DefaultLuceneSearchEngineIndexManager.java:129)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager.cleanIndex(DefaultLuceneSearchEngineIndexManager.java:124)
at org.compass.gps.impl.SingleCompassGps.doIndex(SingleCompassGps.java:128)
at org.compass.gps.impl.AbstractCompassGps.index(AbstractCompassGps.java:154)
at org.compass.gps.impl.AbstractCompassGps.index(AbstractCompassGps.java:128)
at estalea.search.CompassSearchManager.refreshIndexes(CompassSearchManager.java:401)
at estalea.dealmonkey.search.controller.IndexFormController.onSubmit(IndexFormController.java:37)
at org.springframework.web.servlet.mvc.SimpleFormController.onSubmit(SimpleFormController.java:409)
at org.springframework.web.servlet.mvc.SimpleFormController.onSubmit(SimpleFormController.java:381)
at org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
at org.springframework.web.servlet.mvc.CancellableFormController.processFormSubmission(CancellableFormController.java:140)
at org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:265)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:165)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at estalea.savings.seo.SavingsFrameworkFilter.doFilter(SavingsFrameworkFilter.java:152)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at estalea.base.threadcontext.ThreadContextFilter.doFilterInternal(ThreadContextFilter.java:48)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at estalea.accountmanagement.user.filter.AbstractSessionFilter.doFilterInternal(AbstractSessionFilter.java:44)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at estalea.accountmanagement.user.filter.AbstractSessionFilter.doFilterInternal(AbstractSessionFilter.java:44)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:359)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:371)
at org.springframework.security.ui.ExceptionTranslationFilter.doFilterHttp(ExceptionTranslationFilter.java:101)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:371)
at estalea.accountmanagement.security.authentication.DeactivateableRememberMeProcessingFilter.doFilterHttp(DeactivateableRememberMeProcessingFilter.java:39)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:371)
at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:268)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:371)
at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:371)
at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:174)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at estalea.management.controller.ManagementFilter.doFilter(ManagementFilter.java:217)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at com.caucho.filters.GzipFilter.doFilter(GzipFilter.java:208)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:268)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:602)
at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:690)
at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:612)
at java.lang.Thread.run(Thread.java:613)

Here is my configuration settings:

<bean id="dm_databaseLockFactoryProvider" class="estalea.search.DatabaseLockFactoryProvider">
<constructor-arg index="0"><ref bean="base_baseJdbcDao"/></constructor-arg>
</bean>

<bean id="compass" class="org.compass.spring.LocalCompassBean">
<property name="resourceLocations">
<list>
<value>classpath:estalea/dealmonkey/deal/compass/deal.cpm.xml</value>
<value>classpath:estalea/dealmonkey/merchant/compass/dmmerchant.cpm.xml</value>
<value>classpath:estalea/dealmonkey/category/compass/category.cpm.xml</value>
<value>classpath:estalea/dealmonkey/brand/compass/brand.cpm.xml</value>
<value>classpath:estalea/dealmonkey/product/compass/product.cpm.xml</value>
</list>
</property>
<!--
<property name="settings">
<map>
<entry key="compass.engine.store.lockFactory.type" value-ref="dm_databaseLockFactoryProvider"></entry>
</map>
</property>-->
<property name="compassSettings">
<props>
<prop key="compass.transaction.lockTimeout">$dm{dm.lock.timeout}</prop>
<prop key="compass.engine.optimizer.schedule">$dm{dm.search.optimizer.schedule}</prop>
<prop key="compass.engine.optimizer.schedule.period">180000</prop>
<prop key="compass.engine.optimizer.schedule.fixedRate">false</prop>
<!-- <prop key="compass.engine.store.lockFactory.type"><ref bean="dm_lockFactory"/></prop> -->
<!-<prop key="compass.engine.store.lockFactory.type">nativefs</prop>->

<!-- Lucene defaults to these are 10 but increasing can improve indexing performance -->
<prop key="compass.engine.mergeFactor">50</prop>
<prop key="compass.engine.maxBufferedDocs">50</prop>

<prop key="compass.engine.analyzer.default.type">Snowball</prop>
<prop key="compass.engine.analyzer.default.name">English</prop>

<!-- Each of these is done on each object. There is no event per type unfortunately -->
<prop key="compass.event.preCreate.deal.type">estalea.dealmonkey.deal.DealIndexFilter</prop>
<prop key="compass.event.preCreate.category.type">estalea.dealmonkey.category.CategoryIndexFilter</prop>
<prop key="compass.event.preCreate.brand.type">estalea.dealmonkey.brand.BrandIndexFilter</prop>
<prop key="compass.event.preCreate.merchant.type">estalea.dealmonkey.merchant.MerchantIndexFilter</prop>
<prop key="compass.event.preCreate.product.type">estalea.dealmonkey.product.ProductIndexFilter</prop>

<prop key="compass.event.preSave.deal.type">estalea.dealmonkey.deal.DealIndexFilter</prop>
<prop key="compass.event.preSave.category.type">estalea.dealmonkey.category.CategoryIndexFilter</prop>
<prop key="compass.event.preSave.brand.type">estalea.dealmonkey.brand.BrandIndexFilter</prop>
<prop key="compass.event.preSave.merchant.type">estalea.dealmonkey.merchant.MerchantIndexFilter</prop>
<prop key="compass.event.preSave.product.type">estalea.dealmonkey.product.ProductIndexFilter</prop>

<prop key="compass.engine.connection">jdbc://</prop>
<prop key="compass.transaction.factory">org.compass.spring.transaction.SpringSyncTransactionFactory</prop>

<prop key="compass.engine.store.jdbc.dialect">org.apache.lucene.store.jdbc.dialect.MySQLDialect
</prop>
<prop key="compass.engine.store.jdbc.managed">true</prop>
<prop key="compass.transaction.commitBeforeCompletion">false</prop>
<prop key="compass.engine.useCompoundFile">false</prop>

</props>
</property>
<property name="transactionManager">
<ref bean="transactionManager" />
</property>

<property name="dataSource">
<ref bean="dataSource"/>
</property>

</bean>

<bean id="hibernateEntityIndexInfo" class="org.compass.gps.device.hibernate.HibernateEntityIndexInfo">
<constructor-arg index="0"><value>deal</value></constructor-arg>
<constructor-arg index="1">
<value>from estalea.dealmonkey.deal.Deal d where d.status = 'ACTIVE' AND d.merchant.status = 'ACTIVE'</value>
</constructor-arg>
</bean>

<bean id="hibernateGpsDevice" class="org.compass.gps.device.hibernate.HibernateGpsDevice">
<property name="name"><value>hibernateDevice</value></property>
<property name="sessionFactory"><ref bean="sessionFactory" /></property>
<property name="fetchCount" value="1000" />
<property name="indexEntityInfo"><ref local="hibernateEntityIndexInfo"/></property>
<property name="nativeExtractor"><bean class="org.compass.spring.device.hibernate.SpringNativeHibernateExtractor" /></property>
</bean>

<bean id="compassGps" class="org.compass.gps.impl.SingleCompassGps" init-method="start" destroy-method="stop">
<property name="compass"><ref bean="compass" /></property>
<property name="gpsDevices">
<list>
<bean class="org.compass.spring.device.SpringSyncTransactionGpsDeviceWrapper">
<property name="gpsDevice" ref="hibernateGpsDevice" />
</bean>
</list>
</property>
</bean>

<!-- Don't wrap this in a transaction because Compass has its own transaction scheme and wrapping this guy in a
transaction will cause it to deadlock when building an index -->
<bean id="dm_defSearchManager" class="estalea.search.CompassSearchManager">
<property name="compass"><ref local="compass"/></property>
<property name="compassGps"><ref local="compassGps"/></property>
<property name="maxSearchResults"><value>$dm{dm.max.search.results}</value></property>
<property name="dao"><ref bean="base_baseDao"/></property>
</bean>



 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Adam Lane added a comment - 02/Nov/08 08:45 PM
You will see that i have commitBeforeCompletion set to false even though the docs say it should be true for JDBC.
If I set it to true i get this message...

2008-11-02 22:39:54,260 ERROR org.compass.spring.transaction.SpringSyncTransaction : Exception occured when sync with transaction
org.compass.core.engine.SearchEngineException: Search engine transaction already committed while trying to perform an operation

Not sure what the deal is but seems like a separate issue. Hopefully setting it to false in this case is ok since it gets me past that error.
I dont want to open up two problems in one thread...I only mention it because you may see that setting set to false and the docs say it should be true for jdbc.