The Media and Production group for the Urban Apps and Maps program spent the summer working with new technologies in media production, specifically augmented reality technologies. Augmented Reality (AR) refers to the placement of digital content overlaying the physical world, in real-time and in three dimensions. The group, after seeing this technology, thought about ways that it could be applied to help the urban experience.
After brainstorming ideas for a few days, they came up with an idea for using the technology in classrooms. As students they remember being bored with the textbooks they had in school, and they thought AR could help motivate interest in the material. This is particularly true with young students who are learning how to read, where literacy and language acquisition is an ongoing challenge for children in cities all across the U.S., including here in Philadelphia.
According to a recent study done by the U.S. Department of Education and the National Institute of Literacy, 32 million adults, roughly 14% of the population, cannot read, including 19% of high school graduates. Additionally 21% of adults in the U.S. read below a 5th grade level. The group decided that their summer project should look at a new way of using technology to help remedy this problem.
The group took a popular childrens book – Goodnight, Goodnight, Construction Site by Sherri Duskey Rinker and Tom Lichtenheld, took photos of the pages, and used AR to add content to the images (see Figure 2).
They thought about ways that augmentation could aid with comprehension, and chose graphics that were directly related to the text to help students understand and visualize the content. The story is about a construction site winding down after a long day of work, and the machines are going to sleep. The group created augmented floating zzz’s, sleeping cats, and a light switch turning off to emphasize that concept. The characters in the book were various construction site vehicles, so we showed videos of Bob the Builder, animated pictures of bulldozers, and other construction scenes.
One of the things that the interns remembered was their parents reading to them growing up, which also helped them learn. In order to replicate this, on each page, one of the interns read the text of the book aloud, accompanied by sound effects of crunching and gears.
Interns created these augmentations through Aurasma studio, uploading trigger images and then choosing audio and visual media overlays to make the experience of reading for children more interesting and captivating.
The effectiveness of these materials still needs to be tested (and early experiments are being conducted in Temple’s Media Interface and Network Design Lab), but the interns are excited by the new possibilities for the technology, how it could solve critical urban problems, and where the media industry might be headed next. With the production and technology skills they learned this summer, they are hoping their ideas and the augmented book could change the way we learn forever.