""If you find that your own reasoning contradicts the Bayesian prescription, this suggests that it is your intuition which is in need of schooling.""

Comments (4):

Sort By:

Peer 1:

( February 7th, 2017 11:41am UTC )

Edit your feedback below

The logical weakness and empirical irrelevance of the Bayesian program is reflected in the classic example of its application. This is the situation in which a diagnostic test is supposed to have, e.g. a 99% probability of returning a positive result if the patient has the disease, and, e.g. a 95% probability of being disease-free if the test returns a negative result. We can use Bayes equation to calculate the probability that a patient who has a positive test has the disease, assuming we know the prevalence of the disease in the population (the prior probability of having the disease in a particular context).

Obviously, this straightforward application of the equation is trivial if we know all the values it requires. Let's assume that we have a good estimate of the prior probability. What about the claim about the accuracy of the test? Arriving at an estimate of test accuracy implies that we can compare its output against a more definitive test ? one that is effectively 100% accurate. The uncertain test is, implicitly (since we know more about it than it can tell us) only a quick and dirty practical substitute, used to guide a decision whether or not to perform more difficult or expensive, but more definitive, tests. Because, while the test may be probabilistic, the question of whether the patient has the disease is not. They either do or they don?t. In the absence of such better tests, any assumption about the accuracy of the initial test cannot be checked against reality.

For some reason, "Bayesians" want to dwell on the equivalent of the question of the accuracy of the initial test, or, because this cannot be empirically determined, the question of "how accurate we should believe the test is." (This is obviously a different question than "does the patient have the disease, and how can we best determine this?") They want to assess this by testing the patient over and over again, and tabulating the outcomes. In order to come up with some (untestable) value, they need to make a number of additional, equally untestable assumptions. Arguments about the general validity of these assumptions are potentially endless and ongoing, and there is apparently no empirical criterion for preferring any of them. Ergo, an arbitrary and vague standard of plausibility is invoked.

The physician and the scientist typically do not, or should not, dwell on the issue of the accuracy of one unreliable test, but reach beyond it to the better one (the existence of which, again, is implied in the paradigmatic problem). Again, as Ly et al (2016) have pointed out, the ?Bayesian prescription? cannot bring us nearer to a diagnosis, or to a more reliable description of the physical world (other than serendipitously).

Elaborations of the original equation or its logic (plus untestable assumptions/beliefs) cannot cleanse it of its original empirical/logical sin, nor make it more relevant to scientific progress.

Obviously, this straightforward application of the equation is trivial if we know all the values it requires. Let's assume that we have a good estimate of the prior probability. What about the claim about the accuracy of the test? Arriving at an estimate of test accuracy implies that we can compare its output against a more definitive test ? one that is effectively 100% accurate. The uncertain test is, implicitly (since we know more about it than it can tell us) only a quick and dirty practical substitute, used to guide a decision whether or not to perform more difficult or expensive, but more definitive, tests. Because, while the test may be probabilistic, the question of whether the patient has the disease is not. They either do or they don?t. In the absence of such better tests, any assumption about the accuracy of the initial test cannot be checked against reality.

For some reason, "Bayesians" want to dwell on the equivalent of the question of the accuracy of the initial test, or, because this cannot be empirically determined, the question of "how accurate we should believe the test is." (This is obviously a different question than "does the patient have the disease, and how can we best determine this?") They want to assess this by testing the patient over and over again, and tabulating the outcomes. In order to come up with some (untestable) value, they need to make a number of additional, equally untestable assumptions. Arguments about the general validity of these assumptions are potentially endless and ongoing, and there is apparently no empirical criterion for preferring any of them. Ergo, an arbitrary and vague standard of plausibility is invoked.

The physician and the scientist typically do not, or should not, dwell on the issue of the accuracy of one unreliable test, but reach beyond it to the better one (the existence of which, again, is implied in the paradigmatic problem). Again, as Ly et al (2016) have pointed out, the ?Bayesian prescription? cannot bring us nearer to a diagnosis, or to a more reliable description of the physical world (other than serendipitously).

Elaborations of the original equation or its logic (plus untestable assumptions/beliefs) cannot cleanse it of its original empirical/logical sin, nor make it more relevant to scientific progress.

Peer 1:

( February 7th, 2017 11:55am UTC )

Edit your feedback below

I think the conceptual problems are pretty clear, but with respect to formal arguments, it would be helpful if contemporary "Bayesians" could critique the proof against probabilistic induction published by Popper in Nature, in 1983, with a more detailed version in the Philosophical Transactions of the Royal Society of London Series A (Mathematical and Physical Sciences) vol. 321, 1987). Popper was arguing against the type of inductive philosophy being embraced with zeal by modern "Bayesians." Simply re-taking up arguments that have been effectively refuted without framing an effective counterargument seems improper and counterproductive, going backward.

Peer 1:

( February 10th, 2017 9:08am UTC )

Edit your feedback below

Brief summary:

The "Bayesian" procedure uncritically endows correlations - without regard to whether they are spurious or not, nor on what factors they're conditioned on nor by how much - with a label that has the form of a probability, and is derived by a combination of counting (the instances of the correlation) and guessing (the prior) in the frequentist sense; the derivation of this label violates logical principles of frequentist statistics, and it is thus redefined as a meaasure of plausibility. This probability/plausibility is admitted to have no necessary connection with truth or falsity, rendering the plausibility label paradoxical and conceptually uninterpretable.

Since observing correlations are in any event only the most preliminary step in (explanatory) hypothesis formation, let alone testing, this overemphasis on analyzing and quantifying the uncertainty of correlations (something which has, furthermore, been proven mathematically to be a hopeless enterprise), is puzzling. It is unlikely to be the solution to the crisis in psychology, but can only make the research even less efficient and effective, and more superficial, than it already is. Rather than a pulling back on the overemphasis on superficial correlations and statistics, it shoots it into overdrive.

The "Bayesian" procedure uncritically endows correlations - without regard to whether they are spurious or not, nor on what factors they're conditioned on nor by how much - with a label that has the form of a probability, and is derived by a combination of counting (the instances of the correlation) and guessing (the prior) in the frequentist sense; the derivation of this label violates logical principles of frequentist statistics, and it is thus redefined as a meaasure of plausibility. This probability/plausibility is admitted to have no necessary connection with truth or falsity, rendering the plausibility label paradoxical and conceptually uninterpretable.

Since observing correlations are in any event only the most preliminary step in (explanatory) hypothesis formation, let alone testing, this overemphasis on analyzing and quantifying the uncertainty of correlations (something which has, furthermore, been proven mathematically to be a hopeless enterprise), is puzzling. It is unlikely to be the solution to the crisis in psychology, but can only make the research even less efficient and effective, and more superficial, than it already is. Rather than a pulling back on the overemphasis on superficial correlations and statistics, it shoots it into overdrive.

Enter new comment below (Please read the **How To**)

https://pubpeer.com/publications/DFF89869FD1B4EBC8709FCA0BE6E79#fb116644

The context arguably doesn't improve the meaning of the quote, in the sense that it seems to be an argument in favor of irrationalism. It is a bizarre statement. Why argue that my intuition, rather than my reason, is in need of schooling? And is it appropriate, in a scientific context, to appeal to intuition, rather than reason? How can my intuition be schooled, except on the basis of reasonable arguments? The fact there exist various schools of "Bayesianism" makes the problem even more difficult; whose intuition should I trust?

The pitting of "reason" vs "intuition" by this commenter on the "Bayesian prescription" is not accidental, and is linked to the fact that the terms "probable" and "plausible" are used interchangably. For example, Mulder and Wagenmakers (2016), arguing that "Bayes factor" should replace null hypothesis significance tests (which refer to "probability" in the sense of frequency), say that Bayes factor is a "relative measure which balances between the plausibility of the null hypothesis and the alternative hypothesis where the prior...formalizes the anticipated effect if the null is not true."

As Andrew Gelman has said, lots and lots of things are plausible, but they can?t all be probable, cos total probability sums to 1." (http://andrewgelman.com/2016/01/04/plausibility-vs-probability-prior-distributions-garden-forking-paths/). "Plausibility" denotes a vague and highly personal standard. The "plausibility" standard of this "statistical" program also reflects the fact that fundamental values required by the Bayesian equation are unknown, and must be estimated "intuitively." Thus the output must also be merely "plausible," as opposed to "probable" in the statistical sense. The use of precise mathematical techniques cannot correct for the vagueness and unconstrained nature of the "belief" criterion.

The typical Bayesian response to logical problems in their procedure sis to brush them off on by reminding us that they're only referring to personal belief. But this doesn't mean that their beliefs should be everyone's beliefs, nor that they should be adopted in a scientific context. They seem to want it both ways: To imply that they are offering a rational prescription, on the one hand, but to hide under the skirts of irrationalism when they run into trouble of the "reasoning" kind. You can't have it both ways; the papering over of logical gaps and contradictions puts Bayesians (believers in the idea that one can quantify the probability (plausibility?) of a statistical hypothesis) squarely in the irrationalist camp.

In addition to being simply a matter of belief, Bayesian outputs are irrelevant. Science doesn't progress by arguing about the probability of theories or accumulating more and more inconclusive data. They dig down to strip away confounds and try to discover more direct tests. A belief in the Bayes factor would not seem to have much to contribute to this process. As Ly, Verhagen and Wagenmakers (2016) note, "the Bayes factor quantifies the support for M0 versus M1, regardless of whether these models are correct." That means the output (like the input) of the technique is free from the constraints of verisimilitude, to which scientific theories are bound. It is paradoxical for a technique which is agnostic as to truth or falsity should be presented as a guide for choosing hypotheses (even if only statistical ones). It would seem that knowing this would undermine our faith in and understanding of the Bayes factor, and its right to prescribe our belief. With neither a logical nor a natural check, it is extra-logical and supernatural.

The proliferation of "Bayesian" software packages (as those discussed by Mulder and Wagenmakers (2016) in the absence of clarification of seemingly unresolvable gaps in the reasoning seems premature. Are they statistical packages we can believe in?

Permalink

How To)## Are you sure you want to delete your feedback?