Hi,
Thanks for your answer. After I've read your emails, I decided to clear
completely my mapper method to see If I can disable the output of the
mapper class at all, but it seems it did not work
So, here is my mapper method:
@Override
public void map(ByteBuffer key, SortedMap<ByteBuffer, IColumn> columns,
Context context)
throws IOException, InterruptedException
{
}
when I execute hadoop fs -ls, I still see many small output files as
following:
-rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:44
/user/mmigdiso/output/part-m-00034
-rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
/user/mmigdiso/output/part-m-00037
-rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
/user/mmigdiso/output/part-m-00039
-rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
/user/mmigdiso/output/part-m-00040
-rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
/user/mmigdiso/output/part-m-00042
Do you know If I have to put something special to the context to specify
the "empty" output?
Regards
Murat
On Mon, Jun 4, 2012 at 2:38 PM, Devaraj k <[email protected]> wrote:
> Hi Murat,
>
> As Praveenesh explained, you can control the map outputs as you want.
>
> map() function will be called for each input i.e map() function invokes
> multiple times with different inputs in the same mapper. You can check by
> having the logs in the map function what is happening in it.
>
>
> Thanks
> Devaraj
>
> ________________________________________
> From: praveenesh kumar [[email protected]]
> Sent: Monday, June 04, 2012 5:57 PM
> To: [email protected]
> Subject: Re: What happens when I do not output anything from my mapper
>
> You can control your map outputs based on any condition you want. I have
> done that - it worked for me.
> It could be your code problem that its not working for you.
> Can you please share your map code or cross-check whether your conditions
> are correct ?
>
> Regards,
> Praveenesh
>
> On Mon, Jun 4, 2012 at 5:52 PM, murat migdisoglu <
> [email protected]
> > wrote:
>
> > Hi,
> > I have a small application where I have only mapper class defined(no
> > reducer, no combiner).
> > Within the mapper class, I have an if condition according to which I
> decide
> > If I want to put something in the context or not.
> > If my condition is not match, I want that mapper does not give any output
> > to the hdfs.
> > But apparently, this does not worj as I expected. Once I run my job, a
> file
> > per mapper in the hdfs with 87 kb of size.
> >
> > the if block that I'm using in the map method is as following:
> > if (ip == null || ip.equals(cip)) {
> > Text value = new Text(mwrapper.toJson());
> > word.set(ip);
> > context.write( word, value);
> > } else {
> > log.info("ip not match [" + ip + "]");
> > }
> > }
> > }//end of mapper method
> >
> > How can I manage that? Does mapper always need to have an output?
> >
> > --
> > "Find a job you enjoy, and you'll never work a day in your life."
> > Confucius
> >
>
--
"Find a job you enjoy, and you'll never work a day in your life."
Confucius
