Hi, Yes using Map/Reduce its possible . 1. In Mapper, Read the words in line and make pair with calculated distance. Output key of mapper will be the this word pair and value will be distance
2.In Reducer, Just sum the distance for each pair. On Wed, Mar 28, 2012 at 7:42 AM, Praveen Kumar K J V S < [email protected]> wrote: > Hi All, > > I have a file in HDFS spanning across many blocks. Say the file has many > words in it from W1, W2 , W3 ...Wn. > > I want to find the edit distance between all pairs of words. Is this is > possible in Mapreduce. > > For example I have 2 blocks for the file: And an edit distance function > which returns value between 0 to 1(0 means two words are very close t0 each > other). Say edit distance between even and odd words is 1 > > Block 1 has the words W1, W2, W3, W4 > Block 2 has the words W1, W2, W5, W6 > > Now is there a way to find all pairs edit distance. If so how? > > Thanks, > Praveen > -- https://github.com/zinnia-phatak-dev/Nectar
