Материал: искусственный интеллект

Fig. 1. Gamble patterns in WLU notation order. The Sure Thing Principle pattern (g|W , g|L, g|U ) is violated by the Disjunction Effect pattern (g|W , g|L, s|U ), denoted respectively as STP+ and DE−. The index + indicates that the gamble is accepted under U, while the − index indicates it is not accepted. The decision-mirrored Sure Thing Principle pattern (s|W , s|L, s|U) is violated by the mirror Disjunction Effect pattern (s|W , s|L, g|U).

From which it is clear that e.g. an increased presence of the (g|W , g|L, s|U) pattern could push the span of p (g|W ) and p (g|L) above p (g|U) to create a DE. But it is also clear that a few other patterns could cause this to happen. A Disjunction Effect expressed as a deflative violation of the LTP, Eqs. (2), can be expressed as

where we substituted the marginal expressions using Eqs. (3)–(5). These inequalities show that the Disjunction Effect has no tie to the frequency of the Sure Thing Principle pattern, (g|W , g|L, g|U ), itself. It is contrary to the original analysis based on the frequencies of patterns (g|W , g|L, g|U ) and (g|W , g|L, s|U) by Tversky and Shafir (1992) and some of the corresponding subsequent work (Lambdin & Burdsal, 2007). Instead we remark that all of the gamble patterns (g|W , g|L, s|U), (s|W, g|L, s|U) and (g|W , s|L, s|U) will contribute to deflation, while all of (s|W , s|L, g|U), (s|W, g|L, g|U) and (g|W , s|L, g|U) contribute to inflation of the gamble probability under Unknown outcome condition. That is, except for the neutral patterns (g|W , g|L, g|U ) and (s|W , s|L, s|U), each gamble pattern contributes to the average gamble probability under Unknown outcome towards either inflative (‘upward’) or deflative (‘downward’) effect on p (g|U) with respect to p (g|W ) or p (g|L), and potentially average to a violation of the Law of Total Probability in either sense (see SM 3). In a more detailed elaboration of participant behaviour we will use the inflative/deflative potential of each gamble pattern to characterise participants (SM 8).

2. Material and methods

The application of an experimental gamble paradigm to evaluate Expected Utility Theory goes back to Edwards (1954) and Samuelson (1963). The specific method to probe the Disjunction Effect in our study stays close to the Tversky and Shafir (1992) twostage gamble paradigm. The major difference consists of examining whether the Disjunction Effect might be dependent on the size of payoff of the proposed gamble, on the order in which the Win, Lose and Unknown outcome conditioned two-stage gambles are presented, on the context of other two-stage gambles in which a gamble is taken and on whether participants are more or less risk averse. Also the fact that the participants were crowdsourced with Mturk and the short time frame in which all the gambles were performed will be considered in the interpretation of the results.

In order to understand the potential factors that lead up to the Disjunction Effect we also control for between-participants and within-participants variation of the study design. From the between-participants design for conditions {W, L, U} of the first-stage outcome we could infer whether the Disjunction Effect would emerge even if each participant is exposed to only one specific level of outcome condition. We have detailed in Section 1.1 however that an analysis of the Sure Thing Principle itself is not possible in a between-participants design since it needs to be interpreted as a consequential and rational decision at the level of the individual participant and so requires exposure to the three outcome conditions. In a within-participants design a participant will be exposed to all three conditions, which would in principle allow us to analyse the Sure Thing Principle over and above the Disjunction Effect. But at the same time a within-participants design may suppress the Disjunction Effect – and also the STP– or, on the other hand this design may altogether induce these effects by cross-correlating the participant’s decisions over the different conditions. To assess these issues we have explored both withinand between-participants designs in our study, as well as considered other key factors which may inform our understanding of the DE, across Experiments 1 and 2.

Experiment 1 essentially tested the gamble paradigm (i) in a within-participants design for all outcome conditions randomly mixed and, (ii) in a between-participants design for Win and Lose first-stage gamble outcome conditions in comparison with

5

Unknown first-stage gamble outcome conditions. Contrary to Experiment 1, in Experiment 2 we explored the impact of ordering of first-stage gamble outcome conditions. This was done using a design in which either the first-stage Win and Lose outcome gambles preceded the first-stage Unknown outcome gambles, or the other way round with the first-stage Unknown outcome gamble block preceding the first-stage Win and Lose outcome gamble block.

For both experiments the script of the task was developed in Qualtrics and transferred to MTurk for online data gathering. The participants taking the survey were MTurk Workers located in the US and received $.90 for their participation. Participants needed to be at least 18 years of age and have a good command of the English language. Precautions against bot responses included an upfront Captcha test –a ‘Completely Automated Public Turing test to tell Computers and Humans Apart’, and post hoc checking of known GPS-location anomalies (multiple location repetitions or locations documented for bot fraud). Participant engagement was monitored by the inclusion of ‘hidden’ attention tests. These tests were presented as a normal second-stage gamble but had one sentence inserted that indicated that the present gamble was in fact an attention test and that the participant needed to respond in a specified manner.

All participants were informed that all amounts won or lost in each gamble needed to be imagined, there would be no monetary implication in reality. There were four types of gamble; the second-stage gambles conditioned on Win, Lose, Unknown and, the single-stage (unconditional) gamble.

Participants saw the following text for the various kinds of gambles: 3

Second-stage gamble, Win [Lose] condition, with payoff parameter X in {.5, 1, 2, 3, 4}

You just played a new game that gave you a chance to win $100X on heads and to lose $100X /2 on tails. You tossed the coin and won $100X [ lost $100X /2]. You are now offered an identical gamble:

–On heads, you win $100X .

–On tails, you lose $100X /2. Will you toss the coin or not?

(8)

Second-stage gamble, Unknown condition, with payoff parameter X in {.5, 1, 2, 3, 4}

You just played a new game that gave you a chance to win $100X on heads and to lose $100X /2 on tails. You tossed the coin but you will not know whether

you have won $100X or lost $100X /2 until you make your next decision. You are now offered an identical gamble:

–On heads, you win $100X .

–On tails, you lose $100X /2. Will you toss the coin or not?

(9)

The single-stage gamble consisted of a gamble without any information that would result from a previous gamble. In practice it was presented as the last four lines of the Known-outcome second-stage gamble but with the word ‘identical’ replaced by ‘new’. Notice there are five levels of payoff, the gamble with lowest value of the payoff parameter X = .5 corresponds to Win $50 or Lose $25, while at its highest value X = 4 the gamble corresponds to Win $400 or Lose $200.

First Experiment 1 was carried out to compare gamble decisions in a between and within-participants design of the first-stage gamble outcome conditions, as closely replicating Tversky and Shafir (1992) as possible, but with the addition of multiple, variable payoff amounts. Experiment 1 had three participant groups assigned to three different tasks. One group was assigned to the Win and Lose conditions for all values of the payoff parameter X and also took the single-stage gambles, all in random order of outcome conditions and payoff amounts (N = 118). This group received 10 second-stage gambles, of which 5 were Win-conditioned and 5 were Lose-conditioned. They also received 5 single-stage gambles. Out of 168 participants 118 passed the attention test. For the entire task, participants required a median time of 461s. The mean age of the participants was 35.2y while the random assignment of

participants produced a gender skewed participant cohort, mgender = 0.60 (male = 1, female = 0). A second group was assigned to the Unknown outcome condition for all values of the payoff parameter X and also took the single-stage gambles, all in random order of

outcome conditions and payoff amounts. This group of participants also received 5 single-stage gambles, while, regarding secondstage gambles, participants received only the 5 Unknown gambles, Out of 134 participants 114 passed the attention test. For the entire task, participants required a median time of 460s. The mean age of the participants was 36.5y, again the cohort was slightly gender skewed, mgender = 0.44.

3 We note that the wording “stop playing”, to mean “do not gamble on this particular gamble” would not appear ambiguous to participants. Participants were first exposed to three explained gambles and practiced three gambles. All of these used the same wording and clearly showed that “stop playing” did not exit them from the survey, and did not apply to some set of gambles, but instead completed the consideration of the current gamble and allowed participants to proceed to the next gamble. Additionally, each new second-stage gamble would always begin with the sentence “You just played a new game that […]”, which also emphasizes that “stop playing” only applies to the present gamble.

6

Fig. 2. Blocked conditions and flow order of the two-stage gamble Experiment 2: participants were randomly assigned to the ‘U-to-K’ or ‘K-to-U’ flow. In the U-to-K flow participants first took all five second-stage gambles with Unknown outcome information, and then proceeded to take all ten second-stage gambles with Known outcome information. In the K-to-U order, the order of the two blocks was reversed. In each block each participant also took five single stage gambles and was exposed to an attention test. Each flow order leads to a within-participants design for the evaluation of the Disjunction Effect. The evaluation of the order effect occurs in a between-participants design.

The third group took the two-stage task in all three outcome conditions and for all values of the payoff parameter X, all in random order. This group of participants received 5 Win, 5 Lose and 5 Unknown conditioned second-stage gambles, but did not take any single-stage gambles. Out of 126 participants 94 passed the attention test. For the entire task, participants required a median time of 512s to finish the task. This group had a mean age of 35.1y, and a mean gender of 0.44.

Experiment 1 had a single a attention check and those participants who failed it were eliminated from the analysis. This check appeared as a normal second-stage gamble but had a supplementary sentence towards the end of the text on the screen that informed that this particular game (with high payoff) was an attention test and that the participant had to respond mandatorily by clicking the gamble button.

Note, surveying the results of Experiment 1, an indication for a violation of the LTP by a probability discrepancy about the size 0.1 was estimated from the data (see below and Fig. S5 in Supplementary Materials, SM 9). For Experiment 2 the number of required participants was estimated accordingly. With a hypothetical average gamble probability of p .5 and targeted standard error of SE 0.025 the size of the sample, using N = p·(1 p)/SE2, was N 400. With about 1 in 3 participants missing the attention test, for each between-participants condition – corresponding to the ordering of the Known outcome gambles relative to the Unknown outcome ones – we would require about 600 participants, and for the full experiment twice as many participants would be needed.

In Experiment 2 the task progression was structured by grouping trials in blocks of specifically conditioned gambles. One block contained randomly ordered Win and Lose conditioned second-stage gambles, and also single-stage gambles. The other block contained randomly ordered Unknown outcome conditioned second-stage gambles and again single-stage gambles (see diagram Fig. 2). All these gambles appeared in five variations, based on the five values of the payoff parameter X. The participants had to reply by clicking the radio button under “toss the coin” or “stop playing”, after which a new screen appeared with a new gamble.

In Experiment 2, the examination of the DE is within-participants for the {W , L, U} conditions, but it is between-participants for the key manipulation of order, that is, whether participants first saw the second-stage gambles with a known outcome for the first-stage gamble or they saw first the second-stage gambles with an unknown outcome for the first-stage gamble. All participants received 15 second-stage gambles, composed of 5 Win, 5 Lose and 5 Unknown gambles. They further received 10 single-stage gambles; 5 of these single-stage gambles were presented together with Known second-stage ones and 5 of these with Unknown ones. In total the block with Known outcome conditions - Win or Lose - thus had fifteen gambles, while the block with Unknown outcome conditions had ten gambles.

Each participant was randomly assigned to an ordering of first-stage outcome conditions: either first the Known block and then the Unknown block, or first the Unknown block and then the Known block. These two ordering conditions will be referred to as the ‘K-to-U’ flow and ‘U-to-K’ flow respectively. Notice that each flow order stands as a within-participants design evaluation of the Disjunction Effect. A comparison of ‘K-to-U’ and ‘U-to-K’ order effects occurs between participants.

Out of N = 1230 participants having taken the survey for Experiment 2, the response data of 65 participants needed to be eliminated because the task ordering information was not recorded and data from 46 more participants were deleted because the response records were not complete in the Win, Lose or Unknown fields. This left N = 1119 complete participant records divided over both gamble order conditions, of which N = 822 passed the attention tests. Experiment 2 included two attention checks, both presented in the format of second-stage gambles (one task was in the Known context and the other in the Unknown one). The specific format of the attention checks was as above. Participants that failed to correctly respond to these attention tasks were eliminated from the analysis. The choice behaviour of participants that failed the attention test (N = 297) is discussed in more detail in the Supplementary Materials section, (SM 11). The gamble probabilities of these participants show indifference to payoff amounts, even for the ‘More risk averse’ group where a strong dependence on payoff is expected, see SM 4. This payoff indifference shows the implemented attention test correctly identifies participants that engage only superficially with the gamble descriptions and warrants their removal from the main analysis.

Finally we remark that, separated by order condition, the demographics were sufficiently similar over both order conditions. The ‘K-to-U’ flow received 407 participants who took a median time of 562s to finish the task. Their median age was 34y and mean gender

.543. The ‘U-to-K’ flow received 415 participants who took a median time of 539s to finish. Here the median age was 35y and the mean

7

gender was .489. A non-significant gender bias was present in the random assignment of participants in the U-to-K vs K-to-U flows.4 We point out that in our experiment no actual rewards were being given nor were true monetary risks being taken by the participants. Only a flat fee for participation to the task was given to each participant. There is no evidence that true monetary rewards would produce less random performance in this paradigm. The pioneering study of Tversky and Shafir (1992), which showed a strong DE, did not involve real monetary risks, and the same applies to most subsequent replications. An exception is the work of Kühberger et al. (2001) who used real monetary risks in one of their conditions. Specifically, their procedure involved tricking students into risking their proper money first and then surprise remitting their debts at the end. 5 At face value, such a procedure should encourage more attentive behavior. However, Kühberger et al. (2001) did not find any differences in behavior between the condition with real monetary risks and a condition with gambles on hypothetical money. Such a powerful null result casts doubt on the effectiveness of (small) monetary risks and rewards in affecting participants’ behavior in the two-stage gamble paradigm. This conclusion is consistent with the general impression that small monetary incentivization is not effective in biasing behavior. For example, Camerer and Hogarth (1999) concluded that “there is no replicated study in which a theory of rational choice was rejected

at low stakes in favor of a well-specified behavioral alternative and accepted at high stakes” (p.23).

3. Experimental results.

3.1. Experiment 1

The within-participants segment of Experiment 1 showed no indication for a Disjunction Effect at any of the payoff values of X (see Fig. S5 in SM 9). We recall that contrary to the approach of Tversky and Shafir (1992) we distinguish between the violation of the LTP and the violation of the STP, Section 1.1. The aggregate probability to take the second-stage gamble in the Unknown previous outcome condition consistently satisfied the LTP over the payoff range. Also the gamble pattern distributions showed no indication for STP violation, which should have been apparent through the statistical dominance of gamble pattern (g|W , g|L, s|U) (the distribution data corresponding to X = 2 are included in Table 2).

In the between-participants segment of Experiment 1 we found a marginally significant inflative violation of LTP for lower values of the payoff variable X, most clearly at X = .5 with p (g|W ) = 0.68, p (g|L) = 0.71 and p (g|U) = 0.87. Based on contingency counts a left-tailed Fisher test for increased association of Gamble/Stop by Win/Unknown at pay-off at X = .5 showed a significance of p = .0176, with odds ratio 0.45 and CI = [0.22, 0.91] for = .05. Applying the Holm-Bonferroni correction for the five measurements over the payoff range renders the inflative effect of p (g|U) at X=.5 non-significant (uncorrected p-values over the X-range:

{.02, .22, .85, .96, .92}).

Finally we remark that in both the within-participants and the between-participants design the participants clearly distinguished the Lose outcome and Win outcome condition such that a larger preference was given to taking the gamble under Lose outcome condition than under Win outcome –an observation which runs counter to reported data on the two-stage gamble experiment (Tversky & Shafir, 1992; Kühberger et al., 2001; Lambdin & Burdsal, 2007; Surov, Pilkevich, Alodjants, & Khmelevsky, 2019).

3.2. Experiment 2

The full participant group was subdivided by condition flow order in Experiment 2. This allows an observation of a major effect on choice probability due to the block ordering, Fig. 3. In the flow order when the Unknown conditioned block precedes the Known conditioned block the LTP is violated over the whole X-range in an inflative manner p = 6.25e 06, (N = 415), by Wilcoxon signed rank test. The Wilcoxon signed-rank test was used to assess the paired differences –Lose conditioned outcome response versus Unknown conditioned outcome response– from repeated measurements on a single sample. The test allows to compare the effect of two conditions on paired outcomes –here in particular to test whether the participants gamble more on Unknown vs Known outcome conditions. It is a non-parametric test which does not assume a normally distributed population (the data range from 0 to 1 in fractions 1/5), nor does it require equal variance, and independence of the errors. For each participant the X-averaged score under Lose and Unknown outcome conditions was compared and tested for H0 hypothesis that p(g|U, X ) X < p (g|L, X ) X .

On the contrary, in the flow order where the Known block precedes the Unknown block the LTP is satisfied over the whole X-range (N = 407), Fig. 3. The decreasing tendency to gamble under increasing payoff and the differing reaction of participants to secondstage gambles conditioned on W or L are observed in both gamble order conditions.

The design and sample size of Experiment 2 allowed us to analyse the gamble probabilities for different categories of participants. In the first instance we looked at the observed gamble probabilities for the full group by flow order, in the next sections we partition those two flow ordered groups by risk attitude –more vs less risk averse. 6

4 Based on contingency counts a Fisher test for non-random association between flow order and gender showed p = .13 (two-tailed), with odds ratio 0.81 and confidence interval CI = [0.61, 1.06] for = .05.

5Note, such a procedure may introduce a sampling bias, since the participants who decide to take part in such a study would be expected to be more risk seeking.

6A further partitioning by gamble pattern range, based on ID-score, eq. (S3), is provided in the Supplementary Materials, SM 4.

8

Fig. 3. Experimental gamble probabilities, on the left for participants in K-to-U order, on the right for U-to-K order. In the U-to-K order an inflative Disjunction Effect occurs. The payoff parametrised by XLevel [1, 5] appears on the x-axis. Error bars represent the standard error of the mean.

3.2.1. More versus less risk averse participants

Considering that it is behavior relative to gambles that is at stake, it seems a shortcoming in both the original Tversky and Shafir (1992) work and later extensions that the risk aversion of participants has not been taken into account. In order to operationally characterise the risk aversion of participants their choices in the single-stage gambles were used. We recall the single-stage gambles are the same as the condition-free first stage of the two-stage gamble. By experimental design in Experiment 2 each participant is twice presented with all the single-stage gambles, once in the Known block and once in the Unknown block. We use the sum total of the instances a participant accepts the initial gamble as the operational measure of risk attitude. In our design this ‘single-gamble score’ can vary from 0 to 10. A high single-gamble score indicates a participant who frequently chooses to gamble, despite the potential loss, hence expressing low risk aversion. A low single-gamble score indicates a participant with higher risk aversion since these participants do refrain more often from a risky choice with potential loss.

9