Paper ID: 666
Title: The Brain Uses Reliability of Stimulus Information when Making Perceptual Decisions
Current Reviews

Submitted by Assigned_Reviewer_1

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
The authors show the drift diffusion model (DDM) of perceptual decision making implicitly assumes that the brain makes online estimates of stimulus reliability. By formulating a probabilistic equivalent, the authors compare versions of the DDM with and without trial-dependent reliability estimates, and show that behavioral and neurophysiological data are consistent with DDMs that do estimate reliability.

In the introduction, it seems to me that the authors conflate two points: first, that the brain uses estimates of stimulus reliability when making decision; and second, that stimulus reliability affects tuning curves multiplicatively. Although the two are related in the PPC framework cited by the authors, in general the former relates to decoding, the latter to encoding. Also in the introduction, the authors propose that "the multiplicative effect on tuning curves results from adaptation of internal estimates of measurement reliability". However, throughout the paper there is no further explanation of what they mean by adaptation, and the above idea is not further elaborated, except perhaps in the last paragraph of discussion. Section 2, first paragraph: since the authors highlighted multiplicative effects of tuning curves, it is worth pointing out that the exact definition of coherence actually matters and could determine how reliability (or coherence) changes tuning curves. For instance, in the experiments cited, coherent dots move in a fixed direction and the other dots directions are sampled from the uniform distribution; an alternative choice could be to draw all dots from eg a circular Gaussian peaked at the chosen direction, and vary reliability by changing the width of the distribution. Importantly, this is in fact the generative model of the stimulus considered in eq. 3. This latter choice would lead to changes in tuning width, not just multiplicative scaling, when reliability changes. Fig 1, the use of rose histograms is misleading: the area of the circular sector is quadratic (not linear) in the radius (i.e the count). Eq 2 requires some more detail about \rho, the "population" firing rate. To be optimal, the "population rate" should be obtained by weighting each neuron's response by a weight that depends on its tuning curve derivative and on the covariance matrix of the population (see eg Ma et al 2006).

Eq 4 is not precise; It should be x_t conditioned on the true stimulus being left (or right). Similarly, in eq. 5, the l.h.s. should be similarly conditioned. Eq 6: the term \delta y defined in this equation is Gaussian distributed only if conditioned on a particular (left or right) true value of the stimulus. Otherwise, across different true stimuli, it is a mixture of gaussians.

Q2: Please summarize your review in 1-2 sentences
The authors show the drift diffusion model (DDM) of perceptual decision making implicitly assumes that the brain makes online estimates of stimulus reliability. By formulating a probabilistic equivalent, the authors compare versions of the DDM with and without trial-dependent reliability estimates, and show that behavioral and neurophysiological data are consistent with DDMs that do estimate reliability.

Submitted by Assigned_Reviewer_2

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
The paper explored whether the brain utilizes the instant reliability information of stimulus for perceptual decision-making. The authors analyzed the standard drift diffusion model and found that only when reliablity information is included, the model justifies the experimental data. The paper was clearly written and studied a general case RDM used in the experiments, and thus it adds an important contribution to the field. However, how the neural system encodes reliability has not been properly solved.
Q2: Please summarize your review in 1-2 sentences
It is an interesting work demonstrating the brain uses instant reliability information for decision-making.

Submitted by Assigned_Reviewer_3

Q1: Comments to author(s). First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. (For detailed reviewing guidelines, see http://nips.cc/PaperInformation/ReviewerInstructions)
This paper takes a basic drift model of perceptual decision making and shows that in that model, the behavioral data are most consistent with a model where it is not the drift parameter that is constant across stimulus strengths.

The models where the diffusion parameter or the bound parameter to be constant across stimulus strengths match the behavioral data better.

The authors argue that this means the models are inherently weighting the reliability of the evidence (as the two variants where it is not the drift parameter that is kept constant across stimuli strength changes weight the evidence by the reliability).

They further argue that as the models are models of perceptual decision making in the brain, that the brain is thus estimating and using the reliability of the stimulus to make decisions.

The paper is fairly clearly written, but the methods are fairly straight forward and the result does not seem that surprising/significant.
Q2: Please summarize your review in 1-2 sentences
The paper explores drift models of perceptual decision making.

It shows that the version of the model where the drift parameter is constant across stimulus strengths is not consistent with the data and thus that the reliability of the evidence is being used by the models (and they infer by the brain).

The argument is clearly laid out but does not seem that surprising.

Author Feedback
Author Feedback
Q1:Author rebuttal: Please respond to any concerns raised in the reviews. There are no constraints on how you want to argue your case, except for the fact that your text should be limited to a maximum of 5000 characters. Note however, that reviewers and area chairs are busy and may not read long vague rebuttals. It is in your own interest to be concise and to the point.
We thank all the reviewers for their commitment and constructive evaluation of our work.

We appreciate the very positive comments of reviewers 2, 5, and 6.

Reviewers 3 and 4 confirm the validity of our work, but slightly question its significance, because our results do not seem surprising. We agree that it may be rather intuitive for researchers outside of the experimental field of perceptual decision making that the brain should use some sort of reliability in its computation of the evidence. However, we would like to point out that the conclusions drawn in our paper are definitely not main stream and novel. As we argued in the introduction, most experimenters would probably rather intuitively stick to the argument that the reliability is not used by the brain in the specific process of computing the evidence. For example, the present manuscript resulted from rather lengthy interactions with an anonymous reviewer of one our other papers where it became apparent that many of the aspects we clarified in the present manuscript are unknown and, thus, novel to part of the community. To another part of the community, to which reviewers 3 and 4 may belong, our argument makes intuitive sense, but we would like to maintain that our particular argument has not been made before.

We are particularly thankful for the detailed comments of reviewer 1. We did not intend to suggest in the introduction that we will discuss in the remainder of the text how measurement reliability may have a multiplicative effect on tuning curves. If possible, we will insert a clarifying sentence in the final version of the text. Space permitting, we would also like to incorporate the excellent comment about changes in tuning width due to changes in stimulus reliability. The rose histogram may be misleading when the area of the segments is considered, but we believe it may be the most intuitive way of presenting a radial distribution. In the comment to Eq. 2 the reviewer should consider that rho does not refer to a PPC, but rather restates the simple model in Churchland et al. (2008). In Eqs. 4 and 5 we used the +/- sign to indicate dependence on the true stimulus. We will point to this fact explicitly in the final version. The last remark about Eq. 6 is correct, but as the equation is typically applied within a single trial, delta y is usually assumed to be Gaussian.