IMGAME fieldwork

This semester has seen an impressive amount of IMGAME fieldwork and, as usual, our Local Organisers have proved to be thoroughly excellent. Since the end of October we have run Imitation Games on religion in Cardiff, Trondheim and Helsinki, with a trip to Rotterdam scheduled for the week beginning 10 December.

This is an important period for us in two ways. First, the results will provide a crucial test of the method and the underlying theory. The kinds of comparisons we are going to make (and the hypotheses we are trying to test) can be summarised as follows:

• Cardiff fieldwork: this is a repeat of the fieldwork we did in March 2012 so the results should be very similar (i.e. it is a kind of test-retest reliability measure). By using the same questions at Step Two but collecting new Pretender answers and then having them judged by new judges we should get a measure of how stable the IR is over time. We have also tried filtering the questions (i.e. selecting the best 50% of question based on how they were interpreted by Step One judges in March) to see if this makes a difference to the IR estimate.
• Scandinavian fieldwork: the intention is to compare the ability of non-Christians to pretend to be active Christians with the success of non-Christians in Poland and Sicily. We have already recorded IRs of close to zero in the predominantly Catholic countries, so we are now hoping to measure a significant, positive IR in Norway and Finland.
• Rotterdam fieldwork: This is broadly similar to the Scandinavian fieldwork and also tests the ability of non-Christians in a broadly secular country to pretend to be active Christians. Overall, we expect the IRs in Norway (Trondheim), Finland (Helsinki), Netherlands (Rotterdam) and Wales (Cardiff) to be roughly similar to each other and for there to be a statistically significant difference between these four IRs and those measured in Poland (Wroclaw) and Sicily (Palermo).

If the data supports the hypothesis, then we will have made a big step forward in developing the method.

The second reason this semester's fieldwork is important is that we are using it to test and develop the new software being developed by redweb. This is essential to the long term viability of the method as it will make it much easier for new researchers to run Imitation Games as many of the tasks we now perform manually will be automated. This should make the whole fieldwork experience much quicker and also eliminate some of the inevitable data handling errors that creep in as files are cut and copied and pasted by hand.

We will post provisional results in early 2013, assuming the Judges set their files back to us in time. In the meantime, I must end by, once again, thanking the Local Organisers for their hard work, resourcefulness and generally excellent choice of restaurants!

RCN Network Meeting, Boston

On Friday 18 and Sat 19 May, I was in Boston for the first meeting of the Research Co-ordination Network on Sustainable Energy Systems, which funded by NSF and led by Tom Seager. The RCN includes a series of internships for sustainability students in settings outside their home discipline and will use a version of the Imitation Game method in order to explore the extent to which being immersed in a different culture, if only for a few month, allows students to develop meaningful levels of interactional expertise.

Martin Weinel and I were there in order to report on the latest developments at Cardiff and to find out more about what the RCN version of the Imitation Game will look like. This was the main focus of the Friday, where we met the other participants in the Network and described our research interests to each other. In relation to the Imitation Game, we learnt that there are a couple of significant innovations being developed by the RCN, namely that:

1. The RCN version of the Imitation Game will use a Judge plus three other participants: a contributory expert from the target domain (a ‘positive’ control), a student or similar with no immersion in the target domain (a ‘negative’ control) and the student who has had the internship. The hypothesis is that, at the end of the internship, the judge should be able to order the participants and locate the intern’s level on expertise as being between that of the other two participants.
2. The numbers will be relatively small, so statistical analysis of the kind we are doing at IMGAME will be inappropriate. Instead, the success of the internship program will be measured for the individual students in their performance in the Imitation Game. In addition, by aggregating the individual results, the proportion of interns who achieve this intermediate level of interactional expertise can be calculated, giving a measure of success for the programme as a whole.

The official RCN meeting on Friday was then followed on Sat 19 May by a workshop in which Tom Seager, Evan Selinger and others described how they are using role-play games to teach ethics in relation tosustainability. This is of particular interest to Imitation Game project as, like us, they are taking role play games that could (and have?) been played in more traditional pencil-and-paper ways, and turning them into electronic formats that can be adapted for a wide range of classroom contexts. The whole set up was very impressive and provided plenty of food for thought about how we might develop the Imitation Game and (in my other life) how we might teach research methods differently.

IMGAME: Palermo fieldwork

Last week’s fieldwork trip to Palermo was a testing time in more ways than one but overall a definite success.

Network and/or software problems meant we struggled to complete the first day of data collection but sterling work by Marika and Leonardo meant that we managed to run three rounds of ‘Step One’ instead of two and at least give ourselves a chance of making up the lost ground. Big thanks must also go to Martin ‘Albert’ Hall for getting up at 4:30 am to reconstruct the databases in time for us to carry on with ‘Step Two’ as planned. After that, things went much more smoothly: recruitment picked up and we managed to collect all the remaining data as planned. In the end, we even finished slightly ahead of schedule.

Palermo also saw a new variation of the IMGAME method, in which Pretenders at Step Two answered two sets of questions rather than just one. This should give a measure of ‘Pretender noise’ and will allow us to explore the reliability of IMGAME data in a different way. We will, of course, also compare the results from Palermo, in which non-Christians pretended to be Catholics, with the results from Budapest and Cardiff, where we researched the same topics.

As ever, I must end by thanking our Local Organiser Marika for all her hard work. In this case, she had the unenviable task of finding atheists in Italy, dealing with computer problems on the first day and looking after Harry, Martin and me for a whole week. She did all this superbly and will be delighted to know that we fully intend to come back next year!J.

IMGAME: Budapest fieldwork

We arrived back in Cardiff last night after yet another successful IMGAME fieldwork trip. Local Organiser Zoltán Sallay did an excellent job recruiting participants, who almost all turned up, and we managed to complete the data collection phase is record time!

The topic for this set of Imitation Games was religion, with Pretenders given the task of passing as Christians. The results will be directly comparable with recent fieldwork at Cardiff, where secular students were also asked to pass as active Christians. We are still completing the final phase of data collection in Cardiff and will start this in Budapest after Easter. With any luck, we should have results to report at SEESHOP in June.

Lastly, we must single out our tour guide, Martin Weinel, for exceptional work in finding excellent hotels. He surpassed himself this time with the Gellert hotel, which includes entry to the Gellert baths as part of the room price…

IMGAME: Granada Fieldwork

Last week (19-24 Feb) we were in Granada for another week of IMGAME fieldwork. As ever, thanks must go to our Local Organiser, Adolfo Calatrava, and the two student helpers, Ignacio and Carolina, at the University of Granada for making the week very successful and very enjoyable.

The fieldwork topic was sexuality and, in particular, the extent to which straight participants can pass as gay. We used the new designthat was first trialled in Uppsala in which generating questions, pretender answers and then judgements are done as three separate stages rather than within single, real-time Imitation Games. The procedures are getting more efficient and our week’s work in Granada enabled us to collect data in two different conditions:

• In the first condition, pretenders (i.e. straight males) had to pretend to be gay but were allowed to use the internet to research answers. We recruited just over 80 participants to this stage of the research who each answered one of 13 different sets of questions generated on the first day of fieldwork. This data should be directly comparable to that collected in Uppsala, where pretenders were also allowed to use the internet.

• In the second condition, the pretenders (again straight males) had to pretend to be gay but were not allowed to use the internet but had to base their answers on whatever knowledge they had when they arrived. As with the first condition, each of the 80 or so participants answered one set of questions from the same set of 13. This answers, and the judgements they give rise to will be directly comparable to the other condition and will give us an important insight into the effect of using the internet on the outcome of the research.

We have now assembled the questions, non-pretender answers and pretender answers into the 175 unique dialogs and are now in the process of sending these out to judges. We hope to report the results – both the Granada/Uppsala comparison and the with/without internet comparison – at SEESHOP6 in June.

Wroclaw Fieldwork

Last week we were in Wroclaw, Poland for another round of Imitation Game fieldwork. This time the topic was religion and we tried to do the full ‘symmetrical’ version in which equal numbers of Catholics try to pass as Secular and Secular students try to pass as Catholics. This meant recruiting over350 participants so well done (and a big thank you) to Jakub, Aga and Kasha who did a fantastic job recruiting and organising so many people.

The results are not quite in yet – the final step in the data collection process should be completed this week – but we hope to have some preliminary results to report in the next couple of months.

Uppsala fieldwork for IMGAME project

Last week the IMGAME project was in Uppsala. After test runs in Maastricht and Pisa we have now started the first large scale trials of the Imitation Game method.

For Uppsala, we chose to research sexuality and, in particular, the ability of straight men to pass as gay. We managed to run nearly 30 Imitation Games in one day, half of which had gay men in the judge role. We also collected another 80 pretender answers over the next two days

We are now combining these with the original questions and non-pretender answers before sending the completed transcripts to new judges. If successful we will have found a way of significantly boosting the sample size without increasing the time or cost of data collection.

More on whether or not this works – and what the results were – when all the data are in!

In the meantime, as none of this would have been possible without the help of Annelie Drakman, who made all the local arrangements and recruited the participants, we must say a special ‘thank you’ to her.