Two Eyes, 3D

I've moved this blog to Tumblr... Click here to go to the new page.

Wired has an article about a large-scale study of the use of 3D in the classroom in Europe. Overall, I agree with pretty much everything the author says. But I want to expand on some of it.

First, this study was funded by Texas Instruments, a major player in the 3D field. They stand to gain lots of profit from increased adoption of 3D technology. Also, the study was not published in a peer reviewed journal. Instead, the results were published in the form of a white paper and YouTube video case studies. The white paper reads as a 6-page marketing brochure and gives almost no information on how the study was conducted. It certainly does not include enough information to judge its quality. That does not by itself mean the results are bad. But it means we can't truly evaluate them because the details are hidden. In other words, this is not a scientific study. So take it for what it's worth...

Secondly, the Wired commentator mentions the effect of novelty on the experiment. That is, students may learn more simply because they are paying closer attention due to the 3D effect. I am glad he mentioned this as it is often ignored in stereoscopy studies. I think it has a major, major impact and has to be controlled for. In our study, we intend to control for it by having all students wear 3D glasses in all experiments - regardless of whether they are seeing the visuals in 3D or 2D. So, at least in the beginning (and in my experience likely the entire time) the students will think they are seeing 3D whether they are or not.

Still, this Texas Instruments study is a useful one. Why? Because it is the first large scale experimental study of this type I've yet seen in the literature. So it at least offers some insight. Even though it isn't scientific and thus cannot be trusted on its face, it can be used as a sort of thought experiment to think about things differently. For example, the white paper says "The 3D pupils were more likely to use gestures or body language when describing concepts." (p. 3) I noticed this as well in my first study and that is why we are doing our assessments for this project using iPads. We are going to use the accelerometers to follow how students hold the device as they answer questions as a measurement of embodied cognition.

There is a funny statement in one of the videos:

"There is never any behavior problem [when using the glasses]..."

Hah. In my experience in an 8th grade classroom, the kids were indeed a little more subdued when wearing the glasses, but they were by no means perfect. There was plenty of joking around, picking on each other, etc. All the typical things that go one in a classroom were still going on. This was completely a marketing maneuver.

The TI web site says this is "pilot" data. So I wonder if it will ever be published. It presents the head researcher as the "Director of the International Research Agency". I have not heard of that before, and Google-fu turned up zilch. So I question the legitimacy of that organization. I did find the CV of the researcher, and it is impressive. But it's in the field of art education, not cognitive science. Again, this does not mean the research is bad. It just means that I put the chance at ultimate publication at less than 50%. Although I sincerely hope that I'm wrong. If published, it could be a nice contribution to the field.

I'm in Baltimore for the 2011 meeting of the Association of Science and Technology Centers. They are basically the main trade group for science museums, planetariums, aquariums, zoos and just about any informal science organization. They have about 1700 attendees at this conference.

Yesterday they had a morning panel on 3D film development held in the IMAX theater at the Baltimore Science Center. It was a pretty good session for someone like me. I'm new to the production side so it was an interesting glimpse into what goes into 3D film. I know the basics, of course, but my experience is on a completely different scale. For one of the films, they had something like 250+ CPUs crunching for over a week to render just a single model of a star formation region. I thought with modern machines those days were over - apparently not. Moore's law applies to animation as well as the CPUs that render it.

One person in the audience kept raising a question about fidelity to reality. Let's say you have a 3D image of the Andromeda galaxy. In real life, the human eye cannot see stereo beyond 50 feet or so. To get stereo images of mountains and such, the photographer has to physically move the cameras far apart to create the parallax needed. For galaxies, this is simply impossible. Even if we had Star Trek/Wars type of technology, we wouldn't be able to do it. The distances are just too great.

So are we lying by showing such an image? The panel basically said "Yes!". They claimed artistic license is needed to keep it entertaining. I think there was only one astronomer on the panel, the rest were primarily visualizers and directors. But I know some of them personally and I know of the others. They are actually careful to be faithful to the science, where possible. I think they were limited in their responses because the session was running out of time (since we were in a theater, we are unable to "run over" because other films were scheduled). So kudos to them for being honest, but I think the reality is more complicated.

My take is that whether you are being faithful to truth or not is based on the learning goal of the film. Is the goal to show what something would look like to the human eye? (As would be the case in training video, for example.) In that case, the stereoscopic galaxy is indeed a lie. But what if the goal is to educate about the structure and properties of galaxies? In that case, then the stereoscopic galaxy is not a lie because the 3D nature of the galaxy is real. That is, the galaxy would definitely look like that from a specific viewing angle in the Universe. It's just not a viewing angle we'd ever be able to see ourselves. The spatial properties of the central bulge, the flat disc, the orbiting globular clusters, etc. They all exist. They were not created for the film. So, for me as a learning scientists, if what you see exists in reality then it's game.

After the discussion we saw a new 3D IMAX film by National Geographic called Flying Monsters, narrated and written by Sir Richard Attenborough. I thought it was pretty good and effective at teaching a light lesson on evolution. Some of the stereo was very nice, but some of it was way too strong and ghosting was bad (I was in the sweet spot of the theater too). I REALLY hate the opening shot, which is a super stereo animation of the National Geographic logo. It almost caused me to throw off my glasses and is a classic example of why you should ease people into the 3D, not throw them into it to get a cheap thrill. It sucks that this will apparently be at the start of every National Geographic 3D film made.

Our research consists primarily of two main studies.

The first is an experimental study involving a few hundred children at the Living Laboratory in the Museum of Science, Boston. The LL is a neat space. It is setup for cognitive researchers to conduct small-scale experiments with children in the presence of their parents. The researcher is required to be present and must explain their research to the parent while the child is occupied by the experiment. The goal of the LL is to educate the public as to how scientific research is done – and what better way than to show them an authentic experiment underway in front of their own eyes?

I am going to setup an Alioscopy stereoscopic display of around 22”. This is a glasses-free display running on an OS X computer. We will write a slide show program (in java) that will randomly show the child 2D and 3D slides of highly spatial scientific objects (clouds, mountains, crystals, etc.). We will give them an iPad to answer questions about spatial properties of the slides. Questions may be like “which tree is further, the black or brown one?” or “how many clouds do you see in this picture?” The last item will be a drawing task where we ask them to draw something they saw from memory using their fingers. This item will be graded not by accuracy but by the number of spatial elements they include.

Before the slide show begins, they will be given 5-10 spatial cognition questions to help establish a rough measure of prior spatial ability. We will be using items from previously published tests that involve tasks like mentally folding pieces of paper.

While the experiment is underway, I will talk to the parents and will ask them about how much experience their child may have with computers, video games and 3D videos.

We will be looking mainly at differences in accuracy between the 2D and 3D visualizations. We’ll use ANCOVAs to see if any differences are related to prior spatial ability, gender or age. This is a relatively straightforward experimental design (which is the way I like it for early studies!). We will pilot test it this summer and there may be a lot of changes, so this is all in flux. We hope to begin taking data next fall for a 3-month period. If you are in Boston – stop by!

I’ll describe the 2nd study in the follow up post. It’s the sexier of the two since it involves making High-Definition stereoscopic movies. ☺

It is done. The NSF has formally awarded us the grant to conduct our research. The "Two Eyes, 3D" project involves two studies: one with children at the Boston Museum of Science and another with adults at the Adler Planetarium and Astronomy Museum in Chicago. In the coming weeks I'll describe the preliminary research design for both studies and begin summarizing some background literature on stereoscopy and spatial cognition.

The goal of this blog is to shed light on the research process so I hope to describe all the steps along the way - good and bad. It will be something of a sounding board of ideas, some of which will surely be poor and (hopefully) tossed aside over time. But some will likely work out and end up telling us something new about how stereoscopy can be used to teach highly spatial scientific concepts (or not).

The grant runs for 3 years, so I hope we are all around on its last day September 30, 2014. This should be a fun ride!

There has been a lot of news in the last couple of weeks about a research study that found 3D movies do not enhance enjoyment, cause stress-related physical strain and do not help memory. But let's take a closer look before jumping to the conclusion that 3D in movies sucks.

First, the study was presented at an industry conference and has not been published in a peer-reviewed journal. That doesn't mean it is poor, just that it has not been checked out and, more importantly, it means we don't have the details of the study. All have is what was reported to the media at the conference. I'll quote this article:

Carrier and colleagues had 400 students watch one of three movies in either 2-D or 3-D: "Alice in Wonderland," "Clash of the Titans" or "How to Train Your Dragon."

After viewing the film, participants went home and completed an online survey. They were asked to rate how realistic the movie was to them and to report emotions and sensations they experienced, which they selected from a list of 60 words. The words ranged from mild emotions, such as "enjoyment," to more intense ones, including "anger" and "rage." Participants were also quizzed on their knowledge of the film.

The survey showed that neither group remembered the movie better than the other. It also showed that 3-D movie-watchers did not experience a greater sense of immersion in the movie's world, nor did they pay more attention to the film or report experiencing more intense emotions.

I have a number of beefs. First of all, self reported studies always should be taken with a grain of salt. Sometimes in psychology that is all you have because studying the human brain (emotions, etc.) is quite difficult. But such research design is not very sensitive, mainly because there is no zero point. That is the subjects are not on the same scale. For example, what "anger" means to one person is different than it means to someone else. Thus, the difference between "anger" and "rage" can be different between people. Secondly, the sample sizes are small. You have 400 people divided into two groups, with each group divided into 3, thus you have about 65-70 people per film per condition. Unless the results were very strong, I doubt it would be statistically significant. Again, we'll be able to check for all this if the paper is published (a high percentage of conference presentations never make it to publication - I'd put it at around 50%). Third, how can one judge whether a cartoon (such as How to Train Your Dragon) is more realistic in 2D or 3D?

I could go on and on but will stop because it is somewhat unfair to criticize research at a presentation I did not attend. But it was publicly released and the authors have made free with the media, so it is fair to ask questions.

The authors say: ""It didn’t seem to enhance your memory at all," Carrier said. "That’s an unfortunate implication."

Again, how did they determine that? Without an experimental study I just don't see how they can say that based on the study design described in the press reports. In science the importance of a claim needs to be correlated with the strength of your evidence (as Carl Sagan said: "extraordinary claims require extraordinary evidence").

I'm not a 3D champion. My first published study on this subject actually agrees with their core result! In my experience, 3D increased cognitive load and did not have any effect on accuracy of spatial cognition tasks I gave the subjects. And I buy the argument that 3D may not increase enjoyment of a film. Anecdotally I think audiences are starting to show that. And I think this study adds a new data point to the scholarly pool. But I think it's a minor data point and it does not mean what the press says it means. The evidence is not enough to say that 3D increases discomfort and does not affect memory. Not yet.

So a lot has happened in the 2.5 years since my last post. The silence was caused mainly by my dissertation topic changing to something that has nothing to do with stereoscopy. But I'm now done with that, have graduated, had a child and received a promotion at work. But stereoscopy never left my mind or heart. And I also recently got an NSF award to spend the next 3 years doing research on stereoscopy in museum settings. More on that later...

2.5 years may not seem like a long time. But in the world of consumer electronics it is an eternity. Read some of these old blog posts. They mention "an upcoming movie called Avatar". Who knew that it would go on to become the highest grossing film of all time? Now 3D is everywhere. But mostly it's not very good.

That first stereoscopy research paper was published in the meantime. And I did some pilot work on my original dissertation topic. I'll share that research here as well. Stay tuned!