That's super cool Chai - thanks for sharing! I also noticed that, and was seeing how we can reach out to the Fujitsu guys so they can contribute back into MXNet...
On Mon, Apr 8, 2019 at 10:14 AM Lin Yuan <[email protected]> wrote: > Chai, > > Thanks for sharing. This is awesome news! > > Lin > > On Mon, Apr 8, 2019 at 8:48 AM Chaitanya Bapat <[email protected]> > wrote: > > > Greetings! > > > > Great start to a Monday morning, as I came across this news on Import AI, > > an AI newsletter. > > > > The newsletter talked about Apache MXNet, hence thought of sharing it > with > > our community. This seems to be a great achievement worth paying > attention > > to. > > > > *75 seconds: How long it takes to train a network against ImageNet:* > > *...Fujitsu Research claims state-of-the-art ImageNet training scheme...* > > Researchers with Fujitsu Laboratories in Japan have further reduced the > > time it takes to train large-scale, supervised learning AI models; their > > approach lets them train a residual network to around 75% accuracy on the > > ImageNet dataset after 74.7 seconds of training time. This is a big leap > > from where we were in 2017 (an hour), and is impressive relative to > > late-2018 performance (around 4 minutes: see issue #121 > > < > > > https://twitter.us13.list-manage.com/track/click?u=67bd06787e84d73db24fb0aa5&id=28edafc07a&e=0b77acb987 > > > > > ). > > > > *How they did it: *The researchers trained their system across *2,048 > Tesla > > V100 GPUs* via the Amazon-developed MXNet deep learning framework. They > > used a large mini-batch size of 81,920, and also implemented layer-wise > > adaptive scaling (LARS) and a 'warming up' period to increase learning > > efficiency. > > > > *Why it matters:* Training large models on distributed infrastructure is > a > > key component of modern AI research, and the reduction in time we've seen > > on ImageNet training is striking - I think this is emblematic of the > > industrialization of AI, as people seek to create systematic approaches > to > > efficiently training models across large amounts of computers. This trend > > ultimately leads to a speedup in the rate of research reliant on > > large-scale experimentation, and can unlock new paths of research. > > * Read more:* Yet Another Accelerated SGD: ResNet-50 Training on > ImageNet > > in 74.7 seconds (Arxiv) > > < > > > https://twitter.us13.list-manage.com/track/click?u=67bd06787e84d73db24fb0aa5&id=d2b13c879f&e=0b77acb987 > > > > > . > > > > NVIDIA article - > > > > > https://news.developer.nvidia.com/fujitsu-breaks-imagenet-record-with-v100-tensor-core-gpus/ > > > > Hope that gives further impetus to strive harder! > > Have a good week! > > Chai > > > > -- > > *Chaitanya Prakash Bapat* > > *+1 (973) 953-6299* > > > > [image: https://www.linkedin.com//in/chaibapat25] > > <https://github.com/ChaiBapchya>[image: > https://www.facebook.com/chaibapat > > ] > > <https://www.facebook.com/chaibapchya>[image: > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > >[image: > > https://www.linkedin.com//in/chaibapat25] > > <https://www.linkedin.com//in/chaibapchya/> > > >
