Tapping the Crowd in Free Online Courses (And Beyond)

Team Members: Griff, Jeff, Katherine, Mila and Srishti.

It is possible to complete an online course with 40,000 students and never interact with anyone else in the virtual classroom. Which is strange, because the major providers of so-called massive open online courses, or MOOCs, boast how social they are. One MOOC provider, Coursera, promises in its marketing materials that students will “learn with four million Courserans,” and edX, founded by Harvard and MIT, says it can help students “connect with smart and passionate people, just like you, from around the world.”

The reality is that most MOOCs focus on a set of self-service learning resources consumed in isolation. Each student watches a series of lecture videos at their own pace, and then takes tests and turns in other assignments that are graded by automated software. Yes, students are taking the course together, but in the same way that hundreds of thousands of readers around the world peruse The New York Times together every day. MOOCs do have discussion forums — just as online newspapers have comment sections — but the interaction among online students has so far been minimal.

1

We hope to change that.

As part of a course at the MIT’s Media Lab this semester, we’ve developed a prototype for a tool that will help arrange MOOC students into small teams to study together or perform group projects. For inspiration, we looked to other popular services that match people: online dating services, multiplayer video games, couch-surfing sites, and more. But we also turned to the extensive literature on group dynamics and learning in teams. And we took a few too many spirit animal quizzes.

We found one grouping framework particularly compelling. It posited that there are seven types of roles people play in meetings: Encourager, compromiser, leader, summarizer, ideator, evaluator, and recorder. We visited Harvard’s Berkman Center for Internet & Society and did an informal experiment to see how useful those categories were. For this session we divided participants randomly into five groups of four to perform a task known as “the marshmallow challenge.” Each group was asked to build a tower out of 20 pieces of spaghetti, a piece of string, and some masking tape that could hold up a single marshmallow. The group with the tallest free-standing tower at the end of 18 minutes won. The experience reminded us that team interactions are complicated and defy easy categorization. In one group, for instance, one participant started out as a compromiser, but took a more active role halfway through to become an ideator and leader. Still, these categories seemed worth considering as one factor in building diverse teams.

2

In the end, we crafted a short survey asking students about their personality and preferred roles in groups. The online survey has a lighthearted tone, and will soon also include a picture of a cow with the words “don’t click me” in a speech bubble. Our system will note who clicks the cow (and how many times) in assigning them to a group (along with other variables, of course).

8

5

Based on the survey results, the system will create a profile page, and assign one of 40 animal avatars based on the role the survey suggests they might play in an online group.
Someone who is deemed an ideator might be assigned a penguin avatar, for instance. Students will have the option of changing their animal, and in the end the hope is that users will embody the role suggested by their creature.

12

The experience of taking the survey and editing the resulting profile page is meant to encourage positive participation in the resulting group. We do not include a troll as an avatar choice. Many people have had bad experiences in group projects in school or other settings, after all, where one or two people do most of the heavy lifting while others coast along. Group work is hard to manage, and our tool will never be perfect.

In the future, we hope to expand the system to give users even more control over their group placement. Students will eventually have the option of entering an online social mixer where they can recruit team members or join an existing team.

4

We hope to test our grouping tool in a MOOC that MIT’s Media Lab will offer this spring called Learning Creative Learning. And we’ll be presenting the idea at the Designing Future for Peer-to-Peer Learning conference in Baltimore in February. Here is the link to the full conference proposal where you will find more detail about our project and footnotes for our research https://www.dropbox.com/s/6djn45tbt4tvhq6/Group_Formation_in_MOOCs.pdf. You can also watch a video of our class presentation below.

And each of us have included a short reflection on what we’ve learned.

Mila’s Reflection

13

At the beginning of the semester I had no idea what to expect from this class. We were told to create groups with other people who had varying trajectories and backgrounds, many of whom I didn’t know. In my “pitch” video I mentioned taking a large group of people and giving them a customizable and engaging experience. Learning Creative Learning seemed like an obvious choice because it would give me access to a potentially massive audience. My interest in the topic stemmed from my experience working at the NYC Department of Education, which is the biggest school district in the country. We had thousands of teachers to support with a relatively small central staff. Many wanted individualized attention and assistance, but our budget restrictions and lack of manpower made it impossible to address everybody’s needs. So, my individual “how might me” probed at ways to encourage people to work out problems on their own or better yet, work with others and form strong support systems.

After I found my MOOC team we started brainstorming, something we kept doing until the final presentation. Over the course of the semester we realized our group had many leaders and lots of big ideas. Fortunately, with Philipp’s help, we managed to hone in on some general concepts. How do we make MOOCs digestible, engaging, and empowering? This was an idea that was inline with my original question so I was happily on board. I think our biggest roadblock came when we started to create the survey. There was so much information that we wanted to gather, but we had to cut it down. In contrast to our team, most MOOC providers are sitting on an embarrassingly amazing wealth of knowledge about their users and doing nothing with it. All we wanted to do was extract the tiniest bit of information from our classmates to make their relationships “better than random”.

Working with my own team and hearing feedback from other students about their group experiences helped me think about group dynamics in a more salient way. We weren’t talking about random people on the Internet; we were grouping our classmates and ourselves. Currently, our project is in a good place. The deadlines forced us to hammer out details that we could probably spend another semester discussing and debating. I was a strong supporter of having more visual cues that would help people find teammates to form more diverse groups, but this got cut to make our profiles sleeker and more digestible. In the end, I came to a surprising realization. Maybe the big MOOC providers aren’t entirely ignoring user interaction; maybe there are just too many ways to do it. When Anant Agarwal spoke at Harvard he basically said that there were thousands of improvements they could be making to their platform right now, but only so many they could focus on at one time. This experience has made me slightly more sympathetic to that notion.

Jeff’s Reflection

17

I’m usually writing about online courses, not designing them. I’m on leave from a newspaper where I’ve been covering MOOCs, so this course was a chance for me to go gonzo and be on the inside of designing an online course. I originally thought I could help build a journalism MOOC in a single semester. And one with more of a focus on students than on the professors. After all, I’d be part of a team with a mix of skills including coding, design, and education research. But what our team ended up building taught me more than I had ever expected.

I was able to stick to my original goal of focusing on a more student-centered approach to MOOCs, which I still think is an important research area and which our team agreed on. But the rest of the group convinced me that before building a specific MOOC, it was worth focusing on the broader problem of forming teams in online courses. That’s a bigger and more interesting problem than I had realized. One professor I talked to at the Harvard Ed School recommended a classic text on learning in groups, Joining Together: Group Theory and Group Skills, by David W Johnson and Roger T Johnson. I had to hunt around in the stacks of Ed School library to find the beat-up old copy. It was written before the Web existed, and I’m sure Johnson and Johnson never imagined MOOCs, yet it did offer some wisdom for our digital project. In fact, one of the biggest things I learned by reading parts of the book was that the important thing is to forget about technology and think about the broader goals — in this case, getting a few strangers to work together on a project so that they teach each other and guide each other through the material of the course.

I’m not a big fan of group projects in general, but I feel that the five of us in our group really did guide and teach each other. Someone suggested thinking about spirit animals. Another found the group roles framework we used. Sristi always reminded us that eventually we had to build something concrete, with a sense of what was possible from a technical standpoint. Everyone had ideas, and we actually found a way to mesh many of them together, or least try to.

I think our tool is worth building and trying out. I know the course is over, but I think we’ll all continue to work on this project. I hope we do settle on a specific algorithm to try for the Learning Creative Learning MOOC. Even if it’s not perfect, that will give us a starting point to iterate from in the future.

Katherine’s Reflection

15

I came into this class knowing that I was interested in working within the MOOC space. I had been intrigued by MOOCs for some time, and I was excited to try to tackle some of the challenges that they were facing. I felt strongly about gaining a better understanding of the “massive” aspects of these online courses and how we might shape meaningful learning experiences on such a large scale. Within our team, we had many wonderful conversations about all of our experiences and opinions, and we came to realize that we all believed in the idea that popular MOOC providers are underutilizing their most valuable resource: their users. This motivated our work for rest of the semester.

Attempting to answer the question of how to make meaningful groups that are better than random was incredibly daunting. We had many meetings that consisted of talking in circles, getting discouraged, and feeling like we were taking steps backwards. But we kept coming back to this idea of spirit animals, an idea that we felt had the potential to capture this “intangible” element of a good group. I am of the opinion that the best solution is the one that allows our users to choose exactly how they want to be grouped, but within some constraints. Coming up with the appropriate compromise between this notion of personalizability and digestibility is tough, but I think we are definitely on to something. We have a lot more work to do if we actually want to realize this tool (which I think we all do). But I am excited for the building, iterating, testing, data collecting, and editing process.

Srishti’s Reflection

14

At the beginning of this semester my strong interest in the space of open learning motivated me to undertake the Labx course. Having started this fall as a Master’s student, this was my first course at the Media Lab. Before joining this course, my knowledge about the MOOCs was very limited and I had no idea about the existence of different types and forms of MOOC platforms. Initially, when we were grouped in teams based on our skill set and themes of interest, I was worried that being a developer, I would be expected to build a web platform in a short span of time and thus wanted to quit the group, but Philipp explained to me that while working on this project, we need not necessarily build something concrete; it could be a mockup, a solid project idea or maybe a working prototype. I got convinced of this and decided to stay with my initial group. [To be honest, that was the best decision ever! I love my fellow MOOCsters!]

It was exciting to see that all of us in the team shared a similar vision and that was to build a digestible, engaging and empowering MOOC platform. I actually was quite intrigued by the brainstorming procedure adopted in the Labx class to divide students into project teams and I pondered over this broad idea of building a similar kind of collaborative brainstorming platform online, where people with different skill set, diverse interests, learning styles and culture could meet together to find their right match for working on a project/ idea/ subject matter. We discussed thoroughly on this for a while, came up with different if and buts and then thought of a more generalized platform (not another Coursera/ Udacity/ edX) that each one of us could apply for different use cases. For instance, I shared my vision of applying it for the Learning Creative Learning Course http://learn.media.mit.edu/ at the Media Lab for spring 2014.

We collectively watched the survey results of Learning Creative Learning Course (which grouped people based on the time zones). While going through the responses, we were quite surprised to know that most people expressed that their small group interactions were not significant. Majorly from this, we took our “how might we”, and began exploring the ways to encourage the legitimate peripheral participation in online communities, and incorporate better means to declare, network and cluster within an already existing MOOC platform. Initially I was not much convinced about using the concept of spirit animals for creating user profiles, as I thought it would be an American-style of grouping people but when Katherine and Mila made me take a bunch of spirit animal and Myers Briggs personality tests, I got really fascinated by the results. In our group itself, we started discussing the results, assigning each one of us different roles based on our spirit animals. All through this course, I played the role of a clarifier. My team members must have heard these sentences quite often from me, “So here’s what we’ve decided so far”, “I think you’re right, but we could also add….” We are still in the process of solidifying our vision and the tool to group people based on their personality traits, learning styles, goals for the course etc. We have quite big challenges ahead but still we would love to carry out our first test soon through the Learning Creative Learning Course which will take place in the Spring of 2014.

Griff’s Reflection

16

In choosing to be in the MOOC group, I was primarily interested in fostering informal learning and dialogue between students. Having taken a few MOOCs in the past, I felt like the greatest resource, the users, was being severely underutilized. My supposition was that, in harnessing the energy of the students, MOOCs could become a far more powerful tool than they are. It was surprising to me the discipline that users continually showed – I’d never heard of a MOOC hijacked by students who disagreed with the content or classes that had any fluid structure to deviate from the syllabus as prescribed at the beginning of the class. I firmly believe in the power of face-to-face interaction, yet I felt that MOOCs could do far better to create meaningful collaboration between users.

Over the course of the semester, our research prioritized ways in which we can foster deeper learning experiences for students in MOOCs. Given the diversity of MOOC contexts and rigidity of certain providers, we determined that developing a third party platform that could both foster student interaction with one another and offer better-than-random grouping would be the most meaningful way for us to engage with the problem at hand. I don’t think any of us realized just how poorly the large MOOC providers (Udacity, EdX, Coursera) were doing with user collaboration, to the point where there was no way to even message other users. The only interaction was through the one discussion board. In speaking with representatives, it was clear that measurement of deep learning was not only not being done, but not a priority. Therefore, we had to present our finding in the context that they understand – retention and revenue.

Ultimately, grouping tools need to be tested and improved upon. We’ve gone through two iteration with our MAS.S70 class and feedback received during our Berkman Center meeting. We’ve received a lot of valuable feedback, a few notable bits I’ve included below:
· Move beyond self-reflection to include what type of people you’d like to work with
· The gap between how people perceive themselves and how they actually are is very real, but there are a few projects now that seek to minimize this (having your friends assess you, using metadata to tell you that even though you say you like Radiohead, you listen to the Glee soundtrack a lot more).
· Any opportunities to gamify the experience are worth following up on. For instance, is there is there a way to ‘test’ how people interact rather than just asking?
· Once the profile is created, we still have work to do in visualizing the ‘social mixer’ What if you followed it up with something like speed dating? What if there was a jungle of animals laid across a plain/plane based on parameters you set (such as sort by weekly commitment and distance from me)?
· There are others, including elementary school teachers, who use animals as a tool to group people and encourage certain roles. We should learn more about what they have done.

Readings and Reflections

How can we inspire curiosity and passion for learning in more people, and help learners make progress towards their personal goals?  How might we use the web (or other tools) to do this? Start by asking yourself, how you developed the passion for the things you are interested in today (what are they?) and then generalize from there.

image

I am quite inspired by the, “The Hole in the Wall” experiment first performed in 1999 by Indian scientist, “Sugata Mitra” in the slum areas of New Delhi, by placing a computer in the wall to see how kids would use and learn from it. Kids with no prior experience taught themselves how to make use of a computer. This experiment generated a hypothesis that “The acquisition of basic computing skills by any set of children can be achieved through incidental learning provided the learners are given access to a suitable computing facility, with entertaining and motivating content and some minimal (human) guidance.

From this experiment, it seems likely that most learning happens through processes not structured and operates across a broad range of venues from physical to virtual spaces.

Build connectivist MOOCs

Recent developments in massive open online courses (MOOCs) such as Coursera, Udacity, edX, Khan academy have set great examples of online implementation of traditional classroom coursework. Surprisingly, user participation in these courses suffers steep decline over the course duration. Users tend to lose interest in these courses which follow a top down approach for information transfer. This brings a need for connectivist MOOC’s to make learning platforms more peer-led, crowdsourced and participatory.

This movement is unstoppable, however there are many MOOC-less places around the world. In some cases students might have limited access to technology, even if they do have access, they might lack confidence to use tools which facilitate expert-user interaction online. There is a need to reach out to internet-less remote communities with fewer instructors to make these platforms accessible to diverse populations of focused learning environments.

Effective Instructional Design

While surfing some of the online tools like MentorMob, PeddlePad, FreeWare, Forvo, I could not stay for more than a minute because of inappropriate instructional design flow. I feel that learning platforms should be more engaging and intuitive to provide users with insightful educational experience.

Encourage networking tools like IRC for Q&A

Whenever I am stuck while writing code, I usually look up Q&A sites like StackOverflow and StackExchange for answers to solve a problem. But at the same time, I would not prefer to ask questions of these websites, and wait for the answers which would slow down the development process. On the other hand, an IRC channel can be an excellent resource to tap into huge pools of knowledge. The advantage of using IRC channels over these websites is to switch to the channel of your choice and ask relevant questions and get a response immediately. I’ve been on IRC channels like #qt,  #gnome,  #pyladies for quite a long time and have solved countless issues I ran into because some smart people from different time zones happened to be around all the time.

Legitimate peripheral participation

While an undergrad, I made my first contribution to the open source community Wikipedia, which encourages a legitimate peripheral participation. This helped me to learn how to be in a community rather than just learning about it. Furthermore, this experience led to my being a GNOME contributor later.

Learning through discovery and interpretation

Environments

Discovery

  • Unhangout  – An open source platform to host un-conferences online.
  • P2PU – A peer to peer university, a social computing platform that promotes peer-created, peer-led, online learning environments.
  • Barcamp – It is an international network of user-generated unconferences in physical spaces. 

Interpretation : These platforms provide rich and focused learning environments for the growth of diverse populations.

Activities

Discovery

  • Collaborative tinkering – Picture taken from a workshop organized in the lab.
  • Playful Learning – Little kid and his mother in the picture are trying to bounce the mini Lego figures on scratch, using hand gestures http://scratch.mit.edu/projects/10123832/
  • Blogging, online discussion forums, commenting

Interpretation : Learning happens best when we embrace creative collaboration.

Interactions

Discovery

  • Brainstorming session – Picture taken from session two of Media Lab X course.
  • Speed Geeking
  • 99 Fridays – Picture taken from media labs 99fridays party which is a social gathering aimed at exploring festival technologies, art installations and ludic spaces.

Interpretation : Interactions in different environments influence learning styles.

Objects

Discovery

  • Building Blocks – Picture taken from a workshop organized for kids in the media lab using LittleBits and Lego.
  • Do-It-Yourself kits – Picture of DIY Cellphone, a cell phone that you can make by yourself.
  • E-Learning Kiosks – Picture of a kiosk set up at media lab which gives information about lab’s different groups, projects and people.

Interpretation : Learning happens when a learner is engaged in creating personally meaningful things that can be shared with others.

Users

  • Designers
  • Educators
  • Kids