Re: Multi-core support for indexing multiple servers

2013-11-12 Thread Liu Bo
As far as I know about magento, it's DB schema is designed for extensible property storage and relationships between db tables are kind of complex. Product has its attribute sets and properties which are stored in different tables. Configurable product may have different attribute values for each

Re: Multi-core support for indexing multiple servers

2013-11-12 Thread Robert Veliz
I have two sources/servers--one of them is Magento. Since Magento has a more or less out of the box integration with Solr, my thought was to run Solr server from the Magento instance and then use DIH to get/merge content from the other source/server. Seem feasible/appropriate? I spec'd it out a

Re: Multi-core support for indexing multiple servers

2013-11-11 Thread Liu Bo
like Erick said, merge data from different datasource could be very difficult, SolrJ is much easier to use but may need another application to do handle index process if you don't want to extends solr much. I eventually end up with a customized request handler which use SolrWriter from DIH package

Re: Multi-core support for indexing multiple servers

2013-11-08 Thread Erick Erickson
Yep, you can define multiple data sources for use with DIH. Combining data from those multiple sources into a single index can be a bit tricky with DIH, personally I tend to prefer SolrJ, but that's mostly personal preference, especially if I want to get some parallelism going on. But whatever wo

Re: Multi-core support for indexing multiple servers

2013-11-07 Thread manju16832003
Eric, Just a question :-), wouldn't it be easy to use DIH to pull data from multiple data sources. I do use DIH to do that comfortably. I have three data sources - MySQL - URLDataSource that returns XML from an .NET application - URLDataSource that connects to an API and return XML Here is par

Re: Multi-core support for indexing multiple servers

2013-11-07 Thread Erick Erickson
Rob: What I think you're missing is that you are responsible for pulling the data from your separate sources and pushing it to solr via an update command. You can do this in SolrJ, PHP, or any other package that supports a Solr client. You simply address your requests (both update and query) to th

Re: Multi-core support for indexing multiple servers

2013-11-07 Thread manju16832003
Hi Rob, mlti-core approach is different. You could have two cares lets say marketing-core [Has its own schema.xml and data-config.xml] magento-core [Has its own schema.xml and data-config.xml] each core have their own schema.xml and data-config.xml If you go by multi-core approach I guess you won'

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Rob Veliz
I've been reading about Solarium--definitely useful. Could you elaborate here: If you are planning a single master index, that's not multicore. Having more than one document type in a single index is possible, they just have to overlap on at least one field - whatever field is the uniqueKey for

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Shawn Heisey
On 11/7/2013 12:07 AM, Rob Veliz wrote: > Great feedback, thanks. So the multi-core structure I have then is a > single Solr server set up, essentially hosted by one domain owner (but to > be used by both). My question is how does that Solr server connect to the > 2 Web applications to create the

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Rob Veliz
Great feedback, thanks. So the multi-core structure I have then is a single Solr server set up, essentially hosted by one domain owner (but to be used by both). My question is how does that Solr server connect to the 2 Web applications to create the 1 master index (to be used when searching on ei

Re: Multi-core support for indexing multiple servers

2013-11-06 Thread Shawn Heisey
On 11/6/2013 11:38 PM, Rob Veliz wrote: > Trying to find specific information to support the following scenario: > > - I have one site running on one server with marketing content, blog, etc. > I want to index. > - I have another site running on Magento on a different server with > ecommerce conte