[
https://issues.apache.org/jira/browse/PIG-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-3879:
----------------------------
Attachment: PIG-3879-1.patch
Seems DBStorage is quite old and need some major revamp. In multiquery, it does
not commit the transaction since the commit code is in OutputCommitter and
PreparedStatement ps does not shared across DBStorage and OutputCommitter (ps
== null inside OutputCommitter). Attach a fix which works in multiquery. I move
the transaction commit code to RecordWriter. Still have one issue, if the job
fail, transaction will get committed.
> Script fails with multiple STORE statements unless multiquery is off
> --------------------------------------------------------------------
>
> Key: PIG-3879
> URL: https://issues.apache.org/jira/browse/PIG-3879
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Taylor Finnell
> Attachments: PIG-3879-1.patch
>
>
> We are having an issue with multiple store statements in the same script. The
> STORE fails to insert the records into the database. The script is roughly as
> follows...
> {code}
> A = LOAD '...' USING CSVLoader ...;
> STORE A INTO '/tmp/A-unused' USING DBStorage (org.postgresql.Driver, ...,
> INSERT INTO ....);
> B = FOREACH A GENERATE X, Y, CONCAT(X, Y) as Z;
> STORE B INTO '/tmp/B-unused' USING DBStorage (org.postgresql.Driver, ...,
> INSERT INTO ....);
> {code}
> Both DBStorage calls insert into different tables in the same database.
> When the script is run both A, B are stored into their /tmp/ locations.
> However, the data never makes it into the database. We found two ways to get
> the data to make it into the database. The first, was to add a DUMP B command
> after the assignment of B. The second was to execute the script with the -M
> flag.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
