[
https://issues.apache.org/jira/browse/TIKA-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661807#comment-16661807
]
ASF GitHub Bot commented on TIKA-2762:
--------------------------------------
lewismc commented on issue #251: TIKA-2762 Capture short fields (<150 chars) in
EnviParserHeader Metadata
URL: https://github.com/apache/tika/pull/251#issuecomment-432532425
Hi @tballison I think it's an excellent idea. I've prefixed any key with
```envi.``` and also aligned the replacement of space ```" "``` with period
```.```
Additionally, I am still having issues actually detecting the envi header
files as such, they seem to be detected as .txt files unless additional header
information is provided. This justifies the additional tests I added to
detection logic.
An example as follows
**Using tika-app**
Incorrectly detected and minimal metadata extracted
```
java -jar tika-app-2.0.0-SNAPSHOT.jar
/usr/local/pycoal_data/avng.jpl.nasa.gov/AVNG_2015_data_distribution/L2/ang20150420t182050_rfl_v1e/ang20150420t182050_corr_v1e_img.hdr
Oct 23, 2018 11:24:00 PM
org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.
Oct 23, 2018 11:24:00 PM
org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
<?xml version="1.0" encoding="UTF-8"?><html
xmlns="http://www.w3.org/1999/xhtml";>
<head>
<meta name="X-Parsed-By" content="org.apache.tika.parser.DefaultParser"/>
<meta name="X-Parsed-By" content="org.apache.tika.parser.txt.TXTParser"/>
<meta name="Content-Encoding" content="ISO-8859-1"/>
<meta name="resourceName" content="ang20150420t182050_corr_v1e_img.hdr"/>
<meta name="Content-Length" content="20731"/>
<meta name="Content-Type" content="text/plain; charset=ISO-8859-1"/>
<title/>
</head>
```
**Using tika-server with additional header information**
Correct metadata extracted
```
curl -T ang20150420t182050_corr_v1e_img.hdr http://localhost:9998/meta
--header "Content-Type: application/envi.hdr"
"envi.byte.order","0"
"X-Parsed-By","org.apache.tika.parser.DefaultParser","org.apache.tika.parser.envi.EnviHeaderParser"
"envi.sensor.type","Unknown"
"envi.bands","432"
"envi.header.offset","0"
"language","en"
"envi.interleave","bil"
"envi.wavelength.units","Nanometers"
"envi.lines","14674"
"envi.file.type","ENVI Standard"
"envi.samples","739"
"Content-Encoding","ISO-8859-1"
"envi.data.type","4"
"Content-Type","application/envi.hdr"
```
**Using tika-server with NO additional header information**
Again, same result as tika-app, detection logic is not working...
```
curl -T ang20150420t182050_corr_v1e_img.hdr http://localhost:9998/meta
"X-Parsed-By","org.apache.tika.parser.DefaultParser","org.apache.tika.parser.txt.TXTParser"
"Content-Encoding","ISO-8859-1"
"language","en"
"Content-Type","text/plain; charset=ISO-8859-1"
```
There is something up with detection logic for application/envi.hdr files
with .hdr file suffix even though this information is included within
tika-mimetypes.xml
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Capture short fields (<150 chars) in EnviParserHeader Metadata
> --------------------------------------------------------------
>
> Key: TIKA-2762
> URL: https://issues.apache.org/jira/browse/TIKA-2762
> Project: Tika
> Issue Type: Improvement
> Components: parser
> Affects Versions: 1.19.1
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.20
>
>
> I have always wanted to capture more metadata for the EnviHeader files. Right
> now everything is shoved into the records content and I think we could
> improve it.
> I've implemented a rudimentary parser improvement with essentially captures
> any reasonably sized lines items (<150 chars) which can then be populated up
> to Metadata level making faceted search over ENVI .hdr documents a much
> easier task.
> PR coming up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
