Re: [Beowulf] Generation of strings MPI fashion..

2016-10-09 Thread Tim Cutts
Sent from my iPhone > On 9 Oct 2016, at 11:05 pm, John Hearns wrote: > > This sounds interesting. > > I would question this step though, as it seems intrinically a bottlneck: >> Removes duplicate lines: >> $ sort filename.txt | uniq > > First off - do you need a sorted list? If not do not p

Re: [Beowulf] Generation of strings MPI fashion..

2016-10-09 Thread John Hearns
Thinking about this even more, of course the ASCII character set is just a mapping of an integer number to a character. So all you need do is generate sequential sets of numbers from 1 to 255 of the desired length of string, and computers are good at this sort of thing. Which of course takes all

Re: [Beowulf] Generation of strings MPI fashion..

2016-10-09 Thread John Hearns
This sounds interesting. I would question this step though, as it seems intrinically a bottlneck: > Removes duplicate lines: > $ sort filename.txt | uniq First off - do you need a sorted list? If not do not perform the sort... Secondly - how efficient is the Unix 'uniq' utility? How about, ever