NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID: 3347 Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder

### Reviewer 1

The authors present a novel approach for optimizing discrete latent variable models. The approach is a straight-forward combination of the recently introduced direct loss minimization technique (originally designed for structured prediction) with Gumbel-max re-parameterization. This approach avoids the approximation to the arg-max that other methods employ, making in conceptually attractive. The authors apply the method to several datasets and a few uses cases (semi-supervised learning, learning structured latent distributions) and show competitive performance compared to existing methods. Optimizing discrete latent variable models is a basic problem with broad applicability. Methods for optimizing such models is an area of active research. Given the original approach, this work is likely to be of interest to many in the NeurIPS community. The submission appears technically sounds in all of its derivations and the results show competitive or superior results when compared to existing methods. The clarity of the work is decent but certainly could be improved. Since the approach is a relatively straightforward application of two existing methods (Gumbel-max/ direct loss minimization) a more thorough and clear summary of those methods as a background would increase the clarity of the submission and allow a more general audience to fully appreciate and understand the work. Presently, serious consultations with referenced methods are required, even for a reader fairly acquainted with the general approach. The originality of the approach, its technical quality and significance make the paper worthy of acceptance, but the clarity of the presentation detracts from it. Given that the approach appears to be the straightforward application and combination of two existing ideas, the onus falls on the authors to distill and explain these ideas to the reader.