NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2808
Title:Data-driven Estimation of Sinusoid Frequencies

Reviewer 1


		
**Originality** The proposed framework is original and interesting but it was recently introduced by authors of (23), so it is not a contribution of this paper. It's an incremental work from (23) which provides enough improvement, in particular the method to estimate the number of component. **Quality** The paper is of good quality. There is no theoretical results. The claims are well supported by numerical experiments and performances are compared to standard frequency estimation methods. The described method is completed and ready to use. Authors give an honest evaluation of their work in term of performances. Yet, when using neural network one important aspect is the ressources required for training which can differ significantly from ressources required by non-deepnet-based methods. For that reason, when authors mention running time (l217-218), I think it's a bit unfair to other methods. For a fairer comparison and considering that a network should be trained on artificial data before being applied to a specific dataset, authors should mention training time and energy consumption (or CPU number/GPU type). **Clarity** The paper is very well written and well organized. It is easy to read and it seems easy to reproduce. Minor correction: the function FR shouldn't be in italic in the text. **Significance** The results are out-performing state-of-the art methods. The main idea (learning-based techniques on artificial data) could be very useful to the signal processing/data analysis community but it was proposed in the recent paper (23). === Authors Feedback === Other reviewers and authors' response convinced me that this work was a significant improvement of existing reference (23) which was my main concern. Therefore, I increased the score.

Reviewer 2


		
The paper is well written and very clear. Novelties over state-of-the-art are well described and the authors do not oversell their method, by always referring to existing approaches. Performances of the method in this context are clearly demonstrated but I would have liked a deeper investigation on some points : - One of the novelty of the approach is the component counting module : I would have liked to see the influence of this module in terms of performances : what if this module is removed ? Are the improvement due to this module or to the change in the encoder architecture ? What if the number of components is misevaluated ? Does it degrade the results ? - Same questions about the noise level : what if this parameter is misevaluated ? Section 3.4 gives some insights about this, but I think that the fact that the method is fully automatic is an important point that would have needed a more detailed discussion. - How would the standard PSnet behave with an additional component counting module ? Would it achieve the same performances ? After rebuttal : I'm satisfied by the answers the authors gave on the precise influence of the changes they made over [23] : it was one of my main concerns and I find their arguments convincing. The new results clearly show that they have investigated the robustness and cared to make a fully automatic method, which is a nice addition to these algorithms that are often difficult to calibrate. The conception of the counting module is also not trivial, as described in the new simulation : adding a trivial counting module to PSNet does not allow to achieve the same performances.

Reviewer 3


		
This paper describes an improved approach for sinusoid frequency estimation combined with a new approach for sinusoid counting. It then puts them together and compares with several state of the art approaches to each problem. Experiments show that it outperforms them in both tasks on mixtures of 1-10 sinusoids of length 50 samples mixed with white Gaussian noise at SNRs between 0 and 50dB. Originality: While the frequency estimation encoder builds upon previous work, the first experiment shows that when given the proper number of sinusoids, it is able to select frequencies close to the true frequencies 4-8% absolute better than the PSNet method it is based on across SNR. Thus, it is original enough to provide better results. The originality is further enhanced by the proposed source counting module, which works better than the baselines it is compared to. Quality: The paper does a good job of making measured claims and supporting them with experimental verification. This is exemplified by its statement of contribution: "Our results showcase an area of great potential impact for machine-learning methodology: problems with accurate physical models where model-based methodology breaks down due to stochastic perturbations that can be simulated accurately." Clarity: The paper is very well written and easy to follow. The contributions are stated explicitly. The theory of sinusoid frequency estimation is described clearly and concisely. The highlights of a large related literature are summarized clearly. There is one issue that should be clarified, however. For the sinusoid counting task, the error metric is not described. This is an issue in the reader's ability to understand the training of the algorithm, but more importantly its evaluation in Figure 6. What is "Error" as listed on the y-axis? It seems that this is using a categorical loss where each number of sinusoids represents a different category. But this seems too strict and a regression approach might provide a cleaner training signal for the model. What is being done should at least be described clearly. Significance: sinusoid frequency estimation is an important problem in several fields. As described in the paper, these include, "radar imaging, underwater acoustics, seismic imaging, and spectroscopy." The ability to do this better than existing methods, including sinusoid counting as well, is significant. The experimental validation is not completely realistic, but it is in a challenging regime of a small number of samples and a high noise level where existing methods fail. Overall, this paper presents a significant advancement of the deep learning-ification of a classic problem in signal processing. Author feedback ---------------------- I have read the author feedback and found it to be quite convincing. The additional results provided using the proposed counting module with the existing PSNet architecture show that the improvements to the frequency estimator provide a significant boost in performance. The additional details and explanation about the counting module are helpful as well.