Data Import Handler - reading GET

[Kiran J]

Hi,

In data import handler, I can read the "clean" query parameter using
${dih.request.clean} and pass it on to the queries. Is it possible to read
any query parameter from the URL ? for eg ${foo} ?

Thanks

DIH dataimport.properties Zulu time

[Kiran J]

Hi

Is it possible to set up the data import handler so that it keeps track of
the last imported time in Zulu time and not local time ?

Its not very clear from the documentation how to do it or if it is even
possible to do it.

Ref:

http://wiki.apache.org/solr/DataImportHandler#Configuring_The_Property_Writer


Thanks

Re: DIH dataimport.properties Zulu time

[Kiran J]

Thank you for the response. This works if I invoke start.jar with java. In
my usecase however, I need to invoke start.jar directly (consoleless
service so that the user cannot close it accidentally). It doesnt pickup
user.timezone property when done this way. Is it possible to do this using
the tag below somehow. I tried setting locale="UTC" and it didnt work.





On Tue, Mar 25, 2014 at 7:45 PM, Gora Mohanty  wrote:

> On 26 March 2014 02:44, Kiran J  wrote:
> >
> > Hi
> >
> > Is it possible to set up the data import handler so that it keeps track
> of
> > the last imported time in Zulu time and not local time ?
> [...]
>
> Start your JVM with the desired timezone, e.g.,
> java -Duser.timezone=UTC -jar start.jar
>
> Regards,
> Gora
>

Re: DIH dataimport.properties Zulu time

[Kiran J]

I figured it out. I use SQL Server, so this is my solution :



In TSQL, this can be converted to a UTC date time using :

CONVERT(datetimeoffset, '${dih.last_index_time}', 127)

Refs:

http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
http://msdn.microsoft.com/en-us/library/ms187928.aspx




On Thu, Mar 27, 2014 at 2:17 PM, Kiran J  wrote:

> Thank you for the response. This works if I invoke start.jar with java. In
> my usecase however, I need to invoke start.jar directly (consoleless
> service so that the user cannot close it accidentally). It doesnt pickup
> user.timezone property when done this way. Is it possible to do this using
> the tag below somehow. I tried setting locale="UTC" and it didnt work.
>
>  type="SimplePropertiesWriter" directory="data" filename="my_dih.properties" 
> locale="en_US" />
>
>
>
> On Tue, Mar 25, 2014 at 7:45 PM, Gora Mohanty  wrote:
>
>> On 26 March 2014 02:44, Kiran J  wrote:
>> >
>> > Hi
>> >
>> > Is it possible to set up the data import handler so that it keeps track
>> of
>> > the last imported time in Zulu time and not local time ?
>> [...]
>>
>> Start your JVM with the desired timezone, e.g.,
>> java -Duser.timezone=UTC -jar start.jar
>>
>> Regards,
>> Gora
>>
>
>

Example for DIH data source through query string

[Kiran J]

Hi,

I want to dynamically specify the data source in the URL when invoking data
import handler. I'm looking at this :

http://wiki.apache.org/solr/DataImportHandler#solrconfigdatasource

/home/username/data-config.xml   com.mysql.jdbc.Driver jdbc:mysql://localhost/dbname db_username db_password  



Can anyone give me a good example ?

ie http://localhost:8983/solr/dataimport?datasource=

Your help is much appreciated.

Thanks

Re: Example for DIH data source through query string

[Kiran J]

Thank you Alex.


On Mon, Jul 15, 2013 at 12:37 PM, Alexandre Rafalovitch
wrote:

> I don't think you can get there from here.
>
> But you can specify config file on a query line. If you only have a couple
> of configurations, you could have them in different files and switch that
> way.
>
> Regards,
>Alex.
>
> Personal website: http://www.outerthoughts.com/
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> - Time is the quality of nature that keeps events from happening all at
> once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)
>
>
> On Mon, Jul 15, 2013 at 2:56 PM, Kiran J  wrote:
>
> > Hi,
> >
> > I want to dynamically specify the data source in the URL when invoking
> data
> > import handler. I'm looking at this :
> >
> > http://wiki.apache.org/solr/DataImportHandler#solrconfigdatasource
> >
> >> class="org.apache.solr.handler.dataimport.DataImportHandler"> > name="defaults">   > name="config">/home/username/data-config.xml   > name="datasource">  > name="driver">com.mysql.jdbc.Driver  > name="url">jdbc:mysql://localhost/dbname  > name="user">db_username  > name="password">db_password  
> > 
> >
> >
> > Can anyone give me a good example ?
> >
> > ie http://localhost:8983/solr/dataimport?datasource=
> >
> > Your help is much appreciated.
> >
> > Thanks
> >
>

Solr Thrift APIs

[Kiran J]

Hi,

Is it possible to access Solr through thrift APIs ?

Thanks

Multi dimensional spatial search

[Kiran J]

Hi,

Does Solr support multi dimensional spatial search ?

http://en.wikipedia.org/wiki/K-d_tree

Thanks

Re: Multi dimensional spatial search

[Kiran J]

Thank you for the excellent explanation David.

My use case is in the signal processing area. I have a wave that is in time
domain & it is converted to frequency domain on 8 different bands (FFT) ie,
an 8D point. The question for me is "If I have a set of waves (8D points)
in the database and I have a lookup wave, what is the best match ?"




On Sat, May 11, 2013 at 10:42 PM, David Smiley (@MITRE.org) <
dsmi...@mitre.org> wrote:

> Hi Kiran.
>
> The often-forgotten PointType field type can be configured to hold a
> variable number of dimensions.  See the "dimension" attribute of the field
> type's configuration in the example schema.  This field type is really just
> a kind of a macro field type for a configurable number of numeric fields.
> To do a range search you could do:  myPointField:[x1,y1,z1 TO x2,y2,z2]  (3
> dimensional).  This works identically to AND'ing together a series of range
> searches on number fields you explicitly configure.  If the space you want
> to search isn't a simple set or ranges on the dimensions, then you might be
> stuck or at least forced to code a solution, but the scalability of any
> solution is very unlikely to be very scalable (i.e. if you have a ton of
> data this may be a problem).  There are some function-queries (AKA
> ValueSources) that compute special distance calculations in N-dimensional
> space:
> http://wiki.apache.org/solr/FunctionQuery#distdist(), hsine() and
> sqedist() can take a PointType or you can configure each numeric field
> individually and reference them as seen on the wiki.  One key limitation of
> PointType (and LatLonType) is that it does not support multi-valued fields
> (no multiple values for any dimension per document).
>
> You linked to info on "K-D Trees".  Lucene internally at its essence has
> *one* index scheme, which is a sorted list of bytes (often tokenized words
> in text but can be any bytes) that map to a list of document ids.  There
> are
> a class of trees in computer science called a "Trie" AKA "PrefixTree" that
> are fundamentally based on just sorted keys.
> http://en.wikipedia.org/wiki/Trie  Lucene's single-dimensional numeric
> range
> fields are in fact tries; they can store a variable number of
> single-dimensional values per document.  For 2-dimensional spatial, there
> is
> the SpatialPrefixTree abstraction with Geohash and QuadTree
> implementations.
> These support not just indexing Points but any Spatial4j shape, which is
> approximated to the grid.  Supporting an N-dimensional Trie would require
> writing a custom SpatialPrefixTree.  The spatial
> RecursivePrefixTreeStrategy
> has spatial search algorithms that use a SpatialPrefixTree but makes no
> assumptions about the dimensionality of the tree or related shapes, so no
> change there (pretty cool, I think).  You would need to implement some
> N-dimensional Spatial4j shapes. An n-dimensional Point -- trivial.  A
> multi-dimensional range shape (i.e. a square or cube or ...) is not hard.
> Anything more complex in N dimensions is going to get hard fast.  Finally,
> there needs to be a way to parse this shape, which for a custom hack could
> simply be a Solr QParser but longer term would be Spatial4j's not yet
> finished extensible WKT parser.
>
> That was probably more info than you care for but someone else may read
> this
> and find it interesting.
>
> Kiran, I'm curious, what do you want an n-dimensional field for?
>
> p.s. A new SpatialPrefixTree implementation is slated to be developed this
> summer as part of the Google Summer of Code (GSOC).  I hadn't planned to
> add
> N-dimensionality to the feature list.  It could be a stretch-goal maybe.
> https://issues.apache.org/jira/browse/LUCENE-4922
>
> ~ David
>
>
>
> Kiran Jayakumar wrote
> > Hi,
> >
> > Does Solr support multi dimensional spatial search ?
> >
> > http://en.wikipedia.org/wiki/K-d_tree
> >
> > Thanks
>
>
>
>
>
> -
>  Author:
> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Multi-dimensional-spatial-search-tp4062515p4062646.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

DIH Delete with Full Import

[Kiran J]

Hi,

I'm using this configuration:

http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport

The wiki says: "In this case it means obviously that in case you also want
to use deletedPkQuery then when running the delta-import command is still
necessary."

In this link: http://wiki.apache.org/solr/DataImportHandler



   -

   *postImportDeleteQuery* : after full-import this will be used to cleanup
   the index . This is honored only on an entity that is an immediate
   sub-child of  Solr1.4 .

Is it possible for me to use full-import and postimportdeletequery ? I have
table that has the UUIDs of all the records that need to be deleted. Can I
define something like postImportDeleteQuery = "Select Id from
delete_log_table". Can someone provide me an example ?

Any help is much appreciated.

Thank you.

Re: DIH Delete with Full Import

[Kiran J]

Thank you Ahmet.

I figured it out. I had to define a separate entity which takes care of
deletes.





On Wed, Feb 13, 2013 at 1:32 AM, Ahmet Arslan  wrote:

> > define something like postImportDeleteQuery = "Select Id
> > from
> > delete_log_table". Can someone provide me an example ?
>
> postImportDeleteQuery and preImportDeleteQuery queries are lucene/solr
> queries. For example I am using the following:
>
> preImportDeleteQuery="document_type:(photo OR news OR video OR audio)"
>

Re: Start solr from different folder / .Net code

[Kiran J]

Thanks guys.

I ended up setting the working folder in ProcessStartInfo.WorkingDirectory
& it works fine.

Batch file though it works, I am not able to kill the Solr process from
code if I need to.


On Sun, Feb 17, 2013 at 1:12 PM, d_k  wrote:

> Might as well do:
> cd /d H:\downloads\apache-solr-3.6.0\example
>
> or add solr and java to the PATH environment variable
>
> On Sat, Feb 16, 2013 at 11:28 AM, 林辉林灯  wrote:
> > cd  H:\downloads\apache-solr-3.6.0\example
> > H:
> > java -jar start.jar
> >
> >
> >
> > save up codes as a bat file，then，start the bat
> > will be ok
> >
> >
> > -- Original --
> > From:  "Kiran J";
> > Date:  Sat, Feb 16, 2013 06:55 AM
> > To:  "solr-user";
> >
> > Subject:  Start solr from different folder / .Net code
> >
> >
> >
> > Hi everyone,
> >
> > How can I start Solr from a different folder in Windows ? I tried
> >
> > *java -cp "c:\\start.jar" -jar start.jar*
> >
> > I get this error:
> >
> > *Unable to access jarfile start.jar*
> > *
> > *
> > I need to be able to start Solr from .Net. I am able to do it if I wrap
> it
> > in a batch file by first CDing to that folder. Is there any other method
> to
> > do this ?
> > *
> > *
> > Any help is much appreciated.
> >
> > Thanks
>

Filter query not null or in list

[Kiran J]

Hi everyone,

I have a group field which restricts the permission for each user. A user
can belong to multiple groups. A document can belong to only Group (ie) non
multi valued. There are some documents which are unrestricted, hence group
id is null. How can I use the filter for a given user so that it includes
results from both Group=NULL and Group=(X or Y or Z) ? I try something like
this, but doesnt work:

-Group:[* TO *] OR Group:(X OR Y OR Z)

Note that the Group is a UUID field. Is it possible to assign a default
UUID value ?

Any help is much appreciated.

Thanks
Kiran

Re: Filter query not null or in list

[Kiran J]

Thank you Jack, that works.

Kiran

On Thu, Sep 27, 2012 at 5:18 PM, Jack Krupansky wrote:

> Add a "*:*" before the negative query.
>
> (*:* -Group:[* TO *]) OR Group:(X OR Y OR Z)
>
> -- Jack Krupansky
>
> -----Original Message- From: Kiran J Sent: Thursday, September 27,
> 2012 8:07 PM To: solr-user@lucene.apache.org Subject: Filter query not
> null or in list
> Hi everyone,
>
> I have a group field which restricts the permission for each user. A user
> can belong to multiple groups. A document can belong to only Group (ie) non
> multi valued. There are some documents which are unrestricted, hence group
> id is null. How can I use the filter for a given user so that it includes
> results from both Group=NULL and Group=(X or Y or Z) ? I try something like
> this, but doesnt work:
>
> -Group:[* TO *] OR Group:(X OR Y OR Z)
>
> Note that the Group is a UUID field. Is it possible to assign a default
> UUID value ?
>
> Any help is much appreciated.
>
> Thanks
> Kiran
>

DIH scheduling

[Kiran J]

Hi everyone,

Does Solr have out of the box data import handler scheduling ? This link
looks like I need to run an additional JAR.

http://wiki.apache.org/solr/DataImportHandler?highlight=%28%28DataImportHandler%29%29#Scheduling

I need to invoke the import from .Net environment, so I'd like to avoid any
non-Solr code. Any help is much appreciated.

Thanks
Kiran