I am using parallel + wget to download a list of urls.
I would like to append multiple urls at the end of wget for each wget
process.
parallel will append a 'carriage return' after each argument, so all urls
are appended with a %0D by wget.
Is there are anyway to get around this?
Thanks,
I do not understand what is not working. Can you provide an example
showing the problem?
/Ole
On Mon, Mar 10, 2014 at 4:46 PM, Raingo Lee wrote:
> I am using parallel + wget to download a list of urls.
>
> I would like to append multiple urls at the end of wget for each wget
> process.
>
> paral
I got to know what's going on.
My urllist file was coming from Windows, so the line breaks are /r/n
However, when "parallel" read each line in urllist, the /n was striped, but
/r was kept. In the end, the argument lists for each wget becomes, "wget
url1\r url2\r url3\r"
I don't know whether it's
2014-03-10 20:12 GMT+01:00 Raingo Lee :
> However, when "parallel" read each line in urllist, the /n was striped,
> but /r was kept. In the end, the argument lists for each wget becomes,
> "wget url1\r url2\r url3\r"
>
If you see any \r and that is a problem, just remove them with "tr".
cat urll