WangHL created MAPREDUCE-7477:
---------------------------------
Summary: compare some Map-Reduce Framework Counters to point out
or warn user there are issues with the results
Key: MAPREDUCE-7477
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7477
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mapreduce-client
Reporter: WangHL
Our project aims to ensure data uniformity, Map results use
System.currentTimeMillis()%numOfRed decide which Reduce task to send to.
In the case of unstable key output from repeated Map tasks, if a Map task is
rerun due to various reasons, it may cause result data omission or duplication.
I want compare some Map-Reduce Framework Counters(MAP_OUTPUT_RECORDS,
COMBINER_INPUT_RECORDS, COMBINER_OUPUT_REDORDS, REDUCE_INPUT_RECORDS), to point
out or warn user there are issues with the results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
