exBEERiment | Impact Knowledge Of Triangle Test Performance Has On Level Of Confidence And Degree Of Difference Perceived

Author: Marshall Schott
Brewer.Data Collector: Andy Carter

In the 8+ years Brülosophy has been around, we’ve published over 430 exBEERiment articles, each one garnering some level of critique from readers who are dissatisfied with something. One of the most common complaints we receive has to do with our participants, who we overtly acknowledge have varying levels of experience, which is by design – our goal has always been to determine if certain variables in brewing produce a difference that can be perceived by the average beer drinker. The unspoken message I admittedly take from such criticism, though some people are rather direct about it, is in essence, “More astute tasters, such as myself, would have been able to tell them apart, so this data is trash!”

This may very well be true, even though it goes against evidence suggesting one’s level of beer drinking experience plays little role in their ability to perceive differences. What seems more likely to me is that people are simply… being people! It’s well established that we tend to think more highly of our own abilities than we do others’, and another person’s inability to substantiate an accepted claim is “obvious” proof of their ineptness. From this perspective, it makes sense that a person who had it drilled into them that high mash temperatures lead to perceptibly sweeter beer would scoff at results suggesting otherwise, write the xBmt off as meaningless, and pat themselves on the back for having a more developed palate.

A few years ago, we published the results of a fun xBmt where I served a group of my friends a triangle test where all the samples were the same exact beer, the results of which supported the notion that one’s expectation of a difference influences their perception. Recently, while the crew was privately having a laugh about how highly some people seem to value their own abilities, I began to wonder how one’s reported level of confidence in their triangle test selection might be modulated by awareness of their performance. While we don’t routinely discuss the nature of xBmts with participants immediately after data collection, in the times I have, I’ve noticed some surprisingly consistent trends. Even when the results fail to achieve significance, those who identified the unique sample tend to go on about how different it was from the others, indicating a high level of confidence in their selection, while the folks who got it wrong claim the beers were so similar that they resorted to guessing.

Indeed, a non-significant test result does not indicate the absence of a difference, but rather that not enough tasters in a particular xBmt were able to distinguish them such that it could be concluded the variable had a perceptible impact. With this in mind, we designed a unique xBmt to help shed some light on how one’s level of confidence is influenced by awareness of their performance on the triangle test.

| PURPOSE |

To evaluate the impact one’s knowledge of their triangle test performance has on their reported level of confidence when making the original selection as well as the degree of difference they perceived between the beers.

| METHODS |

Given the non-comparative nature of this xBmt, we opted to repurpose a Blonde Ale from a previously published xBmt completed by Andy Carter, the brewing process of which is documented below.

Dead Horse

Andy started his brew day by making a vitality starter of Imperial Yeast A01 House.

A couple hours later, he collected RO water in his BrewZilla unit then set the controller to heat it up.

After adjusting the water to his desired mineral profile, he milled the grains.

When the water was adequately heated, the grains were incorporated and the mash was checked to ensure it was at the target temperature.

While the mash was resting, the kettle hop additions were prepared.

Following the 60 minute mash rest, Andy sparged to collect the proper pre-boil volume then boiled the wort for 60 minutes before running it through a plate chiller. A refractometer reading showed the wort achieved his target OG.

The filled fermenter was placed in a chamber and left to finish chilling to Andy’s desired fermentation temperature of 66°F/19°C for a few hours before the yeast was pitched. After a week, the temperature in the chamber was raised to 70°F/21°C and the beers were left for an additional week before a hydrometer measurement was taken showing FG had been reached.

At this point, the beer was pressure transferred to a sanitized keg.

The filled keg was placed in Andy’s keezer and left on gas for 6 weeks before data was collected.

| RESULTS |

Considering the fact every taster was served 3 samples of the same exact beer, the actual triangle test data is meaningless, as it would be expected each sample was selected a similar number of times. Out of the 29 people who participated in this xBmt, 10 chose the red cup (p = 0.52), 10 chose the blue cup (p = 0.52), and 9 chose the green cup (p = 0.67).

Following completion of the triangle test, participants were directed to a unique section based on the sample they selected. Those who chose the sample in the red cup were informed they were correct on the triangle test, those who chose the sample in the blue cup were informed they were incorrect on the triangle test, and those who chose the sample in the green cup received no information about their performance on the triangle test. All participants were then asked 2 follow-up questions:

1. How confident were you that your selection on the triangle test was correct?
2. How different do you perceive the beer you selected in the triangle test as being from the 2 other beer samples?

The raw data was compiled into a graph to give a visual representation of responses to each question based on what participants were told about their performance.

To gauge whether there were meaningful differences in participant responses to each question depending on what they were told about their performance, we compared each condition to the others using the Mann-Whitney U test.

Level Of Confidence

When compared to participants who were informed nothing about their performance on the triangle test, there was no meaningful difference in level of reported confidence for those informed they were correct (p = 0.18). However, those who were informed they were incorrect on the triangle test reported significantly lower levels of confidence compared to those who were informed nothing (p = 0.006) and those who were informed they were correct (p = 0.038).

Degree Of Difference

When compared to participants who were informed nothing about their performance on the triangle test, there were no meaningful differences in reported degree of difference perceived between the samples identified as unique and the other two samples regardless of whether they were informed they were correct (p = 0.48) or incorrect (p = 0.11). Moreover, those who were informed they were correct on the triangle test reported perceiving a similar degree of difference between the beers as those who were informed they were incorrect (p = 0.08).

| DISCUSSION |

In addition to providing brewers with some level of infotainment, the purpose of the exBEERiment series is to shed some light on the perceptible impact certain variables have on beer. The fact nearly 70% of have returned non-significant results has been just as surprising to us as everyone else, and it has led to myriad accusations being thrown our way. This got me thinking about how one’s knowledge of their performance on the triangle test affects their reported level of confidence and the degree of difference perceived between the beers. As expected, when served a triangle test where the beers were identical after being told there was a difference, a roughly equal number of participants chose each sample.

Confirming my initial hunch, people who were informed they were correct following completion of the triangle test tended to report higher levels of confidence in their selection than both those who were informed they were incorrect as well as those who were informed nothing about their performance, despite the beers being completely identical. This illustrates the influence being informed of one’s performance after the fact can have on their feeling of confidence – if I felt there was a difference and then I’m told I was right, I’m likely going to think more highly of my initial opinion than if I’m told I’m wrong. Curiously, this xBmt did not reveal any differences in perceived degree of difference between the beers based on what one is informed about their triangle test performance, meaning those who were told nothing felt the beers were equally as similar as those who were told they were correct and incorrect.

Ultimately, these findings will have basically no impact on the way perform our xBmts, as the real motivation for this was admittedly to satiate curiosity. We fully acknowledge that these results are not conclusive and prove nothing definitively, but they do offer an interesting perspective about something we’ve been curious about. Truthfully, I’m fascinated by the fickleness of human perception, and the fact we were able to demonstrate this interesting phenomenon with a group of beer drinkers was justification enough for doing it.

If you have any thoughts about this xBmt, please do not hesitate to share in the comments section below!

Support Brülosophy In Style!

All designs are available in various colors and sizes on Amazon!

Follow Brülosophy on:

FACEBOOK   |   TWITTER   |   INSTAGRAM

If you enjoy this stuff and feel compelled to support Brulosophy.com, please check out the Support page for details on how you can very easily do so. Thanks!