NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:6099
Title:Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback


		
The paper introduces an elegant new two-way compression and the use of momentum for such schemes, which are two important questions and as contributions were clearly appreciated by the reviewers, which reached a good consensus. For related work, we comment that in contrast as mentioned, the parallel case of several workers with EF-SGD is considered already in Section 3.5 of https://infoscience.epfl.ch/record/262760/ Also, reviewers were asking if experiments could be expanded to clarify more on how much the momentum aspect improves upon EF alone, and the other detailed comments by the reviewers.