Hey Shashank How are you doing? I an not sure about existing API but I had written a yarn custom application myself to get info from nodes directly by running Linux command. Would be nice to know any API way of doing it . Thx
On Mon, Aug 13, 2018, 15:47 Shashank Prabhakara <[email protected]> wrote: > Hi All, > > Is there an existing API which will retrieve the number of physical cores > available in the worker nodes of the cluster(not sure about the term > physical cores, but essentially actual number of cpu cores as opposed to > vcores) ? > > I was not able to find any in RM or NM APIs for 2.8, so if it really is > not available, how would you suggest I proceed - my best guess would be a > distributed shell-like application with an AM that would run one container > per node on all nodes and essentially execute "nproc --all". > > I'm exploring my options right now so anything related to physical cores > will be helpful, whether it's total cores at cluster level or at a worker > level and whether it is the number of physical cores available to yarn or > the total irrespective of yarn config. > > Thanks in advance. > > Regards, > Shashank >
