[#CMP-704] "Collection not processed by flush" with Compass::GPS and Hibernate

Compass

"Collection not processed by flush" with Compass::GPS and Hibernate

Created: 06/Sep/08 03:37 AM Updated: 04/Dec/09 04:00 PM

Component/s:

Compass::Core

Affects Version/s:

2.1.0 M2

Fix Version/s:

2.1.0 M3

File Attachments:

processedinflush.tar (60 kB)

Description

« Hide

This issue has been around for a while, I will can add references later but I have limited time right now.

I don't really like the post-commit event listeners option for reasons previously mentioned, so I must actually fix the original issue. There are two options I see:

1. set the processed flag of the new collections that Compass instantiates as part of the indexing process.

This seems a bit naughty, but then again it's simple, clean and self contained.

2. For those collections that are not already initialised, merge the old value (in the current Lucene document/Compass Resource) with the Resouce being created during the mirroring process.

This is IMHO the "right way" since Lucene and DBs are fundamentally different in that Lucene doesn't have an "update" operation in CRUD terms. Therefore - when "updating" a search engine index document - we need to do more.

But it is also much more work since the marshalling code (the Class marshaller specifically - sorry cannot remember the same) doesn't currently appear to allow for partial marshalling/merging - it just does every property.

I have attached a patch (it's a TAR file, not patch file format) for Compass trunk in order to fix the issue with solution #1; it includes a test case that reproduces the problem when the fix is not applied.

I would be very grateful if they could be committed

All

Comments

Change History

Sort Order:

[ Permlink | « Hide ]

Shay Banon added a comment - 10/Sep/08 01:10 PM

Hi Maurice,

Thanks for the effort!. I took a look at the changes you posted, and I think its a great way to solve this. Two questions before I commit:

1. In the finally clause, there is a HashSet that is created there. I don't think it is really needed, right?
2. Should the same logic exists within postInsert as well?

[ Permlink | « Hide ]

Maurice Nicholson added a comment - 10/Sep/08 03:08 PM

Hi Shay,

1. yes you're right, that is a completely pointless HashSet!!

2. Good point - I will try and create a test case for it and if I can find the same probem with insert also then that answers the question

[ Permlink | « Hide ]

Maurice Nicholson added a comment - 11/Sep/08 02:24 PM

I tried to create the same failure for inserts...

I had to alter the domain model slightly to add one more level of depth between the object I was inserting and the collection I expected the exception to occur for, according to my understanding of the problem.

What happened was that the exception is NOT thrown.

I debugged and found that even though at the end of Compass's HibernateEventListener#onPostInsert there have been additional CollectionEntrys created - presumably due the the indexing process - they do not raise the error, even though they have processed=false. Something in Hibernate sets the processed flag to true later on and everything is fine.

So AFAICT the same code logic is not required for postInsert.

[ Permlink | « Hide ]

Shay Banon added a comment - 11/Sep/08 03:28 PM

Fixed. Thanks!.

[ Permlink | « Hide ]

Martin Kovacik added a comment - 03/Dec/08 03:25 PM

According to http://forum.compass-project.org/thread.jspa?messageID=296707 this issue is still unresolved. In 2.1.0 I still get the collection was not processed by flush() error.

[ Permlink | « Hide ]

Marcus Ilgner added a comment - 04/Dec/09 04:00 PM

I just came across this issue, too. Strangely enough, it only seems to occur after switching the project from a standalone Spring container to a JBoss-based project. There's some discussion on the Hibernate bugtracker (issues HHH-1725 and HHH-2763 mainly) and it seems like other libraries (like Hibernate Search) suffer from the same problem.