Hi Jacob,
I just updated the code such that it should now be possible to send in
multiple values as literals, as in an HTML form that looks like:
<form enctype="multipart/form-data" action="/solr/update/extract"
method="POST">
<input name="ext.literal.features" value="solr"/>
<input name="ext.literal.features" value="cool"/>
<input name="ext.def.fl" value="text"/>
Choose a file to upload: <input name="file" type="file" /><br />
<input type="submit" value="Upload File" />
</form>
Cheers,
Grant
On Dec 12, 2008, at 11:53 PM, Jacob Singh wrote:
Hi Grant,
Thanks for the quick response. My Colleague looked into the code a
bit, and I did as well, here is what I see (my Java sucks):
http://svn.apache.org/repos/asf/lucene/solr/trunk/contrib/extraction/src/main/java/org/apache/solr/handler/extraction/SolrContentHandler.java
//handle the literals from the params
Iterator<String> paramNames = params.getParameterNamesIterator();
while (paramNames.hasNext()) {
String name = paramNames.next();
if (name.startsWith(LITERALS_PREFIX)) {
String fieldName = name.substring(LITERALS_PREFIX.length());
//no need to map names here, since they are literals from the
user
SchemaField schFld = schema.getFieldOrNull(fieldName);
if (schFld != null) {
String value = params.get(name);
boost = getBoost(fieldName);
//no need to transform here, b/c we can assume the user sent
it in correctly
document.addField(fieldName, value, boost);
} else {
handleUndeclaredField(fieldName);
}
}
}
I don't know the solr source quite well enough to know if
document.addField() can take a struct in the form of some serialized
string, but how can I pass a multi-valued field via a
file-upload/multi-part POST?
One idea is that as one of the POST fields, I could add an XML payload
as could be parsed by the XML handler, and then we could instantiate
it, pass in the doc by reference, and get its multivalue fields all
populated nicely. But this perhaps isn't a fantastic solution, I'm
really not much of a Java programmer at all, would love to hear your
expert opinion on how to solve this.
Best,
J
On Fri, Dec 12, 2008 at 6:40 PM, Grant Ingersoll
<gsing...@apache.org> wrote:
Hmmm, I think I see the disconnect, but I'm not sure. Sending to
the ERH
(ExtractingReqHandler) is not an XML command at all, it's a file-
upload/
multi-part encoding. I think you will need an API that does
something like:
(Just making this up, this is not real code)
File file = new File(fileToIndex)
resp = solr.addFile(file, params);
----
Where params contains the literals, captures, etc. Then, in your
API you
need to do whatever PHP does to send that file as a multipart file
(I think
you can also POST it, too, but that has some downsides as described
on the
wiki)
I'll try to whip up some SolrJ sample code, as I know others have
asked for
that.
-Grant
On Dec 12, 2008, at 5:34 AM, Jacob Singh wrote:
Hi Grant,
Happy to.
Currently we are sending over documents by building a big XML file
of
all of the fields of that document. Something like this:
$document = new Apache_Solr_Document();
$document->id = apachesolr_document_id($node->nid);
$document->title = $node->title;
$document->body = strip_tags($text);
$document->type = $node->type;
foreach ($categories as $cat) {
$document->setMultiValue('category', $cat);
}
The PHP Client library then takes all of this, and builds it into an
XML payload which we POST over to Solr.
When we implement rich file handling, I see these instructions:
-----------------------------
Literals
To add in your own metadata, pass in the literal parameter along
with the
file:
curl
http://localhost:8983/solr/update/extract?ext.idx.attr=true
\&ext.def.fl=text\&ext.map.div=foo_t\&ext.capture=div
\&ext.boost.foo_t=3\&ext.literal.blah_i=1
-F "tutori...@tutorial.pdf"
-----------------------------
So it seems we can:
a). Refactor the class to not generate XML, but rather to build post
headers for each field. We would like to avoid this.
b) Instead, I was hoping we could send the XML payload with all the
literal fields defined (like id, type, etc), and the post fields
required for the file content and the field it belongs to in one
reqeust
Since my understanding is that docs in Solr are immutable, there
is no:
c). Send the file contents over, give it an ID, and then send over
the
rest of the fields and merge into that ID.
If the unfortunate answer is a, then how do we deal with multi-value
fields? I don't know how to format them given the ext.literal
format
above.
Thanks for your help and awesome contributions!
-Jacob
On Fri, Dec 12, 2008 at 4:52 AM, Grant Ingersoll <gsing...@apache.org
>
wrote:
On Dec 10, 2008, at 10:21 PM, Jacob Singh wrote:
Hey folks,
I'm looking at implementing ExtractingRequestHandler in the
Apache_Solr_PHP
library, and I'm wondering what we can do about adding meta-data.
I saw the docs, which suggests you use different post headers to
pass
field
values along with ext.literal. Is there anyway to use the
XmlUpdateHandler
instead along with a document? I'm not sure how this would work,
perhaps it
would require 2 trips, perhaps the XML would be in the post
"content"
and
the file in something else? The thing is we would need to
refactor the
class pretty heavily in this case when indexing RichDocs and we
were
hoping
to avoid it.
I'm not sure I follow how the XmlUpdateHandler plays in, can you
explain
a little more? My PHP is weak, but maybe some code will help...
Thanks,
Jacob
--
+1 510 277-0891 (o)
+91 9999 33 7458 (m)
web: http://pajamadesign.com
Skype: pajamadesign
Yahoo: jacobsingh
AIM: jacobsingh
gTalk: jacobsi...@gmail.com
--
+1 510 277-0891 (o)
+91 9999 33 7458 (m)
web: http://pajamadesign.com
Skype: pajamadesign
Yahoo: jacobsingh
AIM: jacobsingh
gTalk: jacobsi...@gmail.com
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
--
+1 510 277-0891 (o)
+91 9999 33 7458 (m)
web: http://pajamadesign.com
Skype: pajamadesign
Yahoo: jacobsingh
AIM: jacobsingh
gTalk: jacobsi...@gmail.com
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
