Group items tagged

This week the focus is on assessment and evidence of understanding. The emphasis is, of course, on how we assess our students. However, eteachers can be more easily observed, so I thought the community might find it interesting as to how I, as an eteacher, am assessed. (The students do an automated self-paced lesson and then take a practice quiz. If they have questions during the lesson or in order to go the next one, we interact. Based on the quiz and other questions/discussions we have, I pass them to the next lesson. I have up to 4 students in one session.) Once a month, my supervisor listens to one of the recorded sessions and writes up an assessment. What I find interesting that is unique to elearning, is that my use of the technology environment goes into my assessment. I need to be logged in on time, I need to effectively switch among my students, etc. Also, if an area is weak, a request by my supervisor to re-take an automated class I have taken before - such as effective questioning - might be assigned.

The online environment provides for easy capture of the session. The requirement for teaching include having a degree and a background check, but not a teaching certificate. I feel the monitoring is an extra check and balance on both how we are teaching and our online presence. We go through training and have guidelines on what we can and cannot discuss with the students - such as sharing emails, contacts outside of class, etc. Recently this has come into focus, as the regular school teachers at the school I am tutoring with (we are a scheduled once a week class during the school year) encouraged the students to ask where we live, etc., to feel more connected. On our teacher platform, the teachers and our supervisor were able to post and share with each other on how they were and should be handling this. It is a balance - just as in a regular classroom - to be connected, yet have privacy. I do not find what information I should and should share or ask to be a problem for me, but the time focus is. The students are used to texting all the time and feel comofrtable with using the time that way. I have now developed my own transitions - from the explicit: "we've chatted enough for right now, please go on with your lesson" to a softer transition by using the information in our dialog to set-up a problem: You got so much snow! So if you had eight snowballs and split them evenly....".

This is fascinating! I've honestly never seen an instrument for a supervisor to evaluate an online teaching session. (Or maybe I just don't remember.) For distance education at BC, instructors/divisions can opt in to the online student evaluations. I don't know if Norma signed up to have this course included in the evals. If so, you may have received access to that today. As far as I know, that's the primary way teachers are evaluated here. It compares to how they are evaluated in f2f classes. For me, I like to get constant feedback from students, and that's why you see me poking around a lot, sending lots of email, and including self-checks. It's always an awful feeling to get to the end of a course and discover you could have made changes early on that would have made a big difference in learning. (But I always expect to discover some of that at the end, as well!) Thanks for sharing this!

I read the assignment about assessment and suddenly thought...this is really crap! I dislike the whole concept of grades. In fact, this video shows that if you really want to assess assessment...what you need to do is teach the most difficult class and find a way to help all students learn it. Learning is a process. I think you cannot learn as much without the help or couching of an instructor or professor. If they show you what concepts they want you to learn...every step along the way, through course design and asking questions to stimulate thought, learners will learn even the most difficult concepts and apply them.. Feedback is essential. Not just feedback; but feedback that helps the learner build on what they already know and move forward. Interactive learning with others continues the process and helps give the learner even more ways to continue the learning process.

One of the biggest sea-changes I've seen in my own teaching experience at Bellevue College since the 1990's is a greater shift towards student-centered learning. One of the major indicators to me of this change was the "Affirmation of Inclusion" initiative that states as an up front policy that Bellevue College is committed to treat all students with respect and support no matter of their ethnic background, age, disability, gender, etc. This policy is visibly posted in every classroom. The older standard educational model based on what Rick Stiggins calls "producing dependable rank order", ie.. consisting of winners and losers - is giving way to a educational approach with lifetime learning as a foundation. We are all on a continuous learning path during the entire arc of our lives, and can potentially all help each other wherever we are on the curve. Rather than specialized pidgeon holes, we are more aptly described by Buckminster Fuller's book title in the 1960's - I seem to be a verb...

I dislike being graded! Yet the grade factor has given me a reason to dig deeply into any studies I take. I wanted to be in charge of my learning and assess myself...unfortunately, I set standards far higher than any instructor would.

I received the following from Norma Whitacre and thought I'd pass it along. Our focus this week in EDUC251 is on this and is appropriate introduction to this course. from Norma: The third class in the series is offered spring quarter. It is Assessment in eLearning, EDCU 252. I hired Robin Jeffers, an assessment expert, to teach it, and she is great. Class begins the second week of the quarter and meets on campus two days only: Tuesdays. April 12 and June 7, 4:30-6:20, item 2239. A full class description is attached. (bookmarked above) We are also offering EDUC 250, the Introductory class if you know anyone who wants it.

I haven't seen the content for this course, but it's one I'd actually consider taking myself. (Though I imagine I'd stir up some controversy.)

Jennifer, it would AWESOME to have to in the class and have another voice to help reason through and learn about assessment. Please do take it!!!!

I found this a interesting study of assessment.

A British video on formative assessment. An interesting concept.

thought someone might be interested in assessment

Thanks for posting this, Jennifer! I found some good questions within the text--questions I will continue to think about as I work my way through the elearning assessment course we just began. Hope all is well in your world!

Dan Meyer's interesting process for math assessment.

Joy, Your posts are quite sobering, regarding your own experiences working towards completing your university education. I strongly agree with the following quote in your post: "In sum, it is time for educators to make college and higher learning relevant to their students. It is not about us, it is about them. We are the facilitators of their learning. We must prepare them for the world they will find themselves in when they leave the protected walls of our colleges and universities" I found the recent events in Cairo to be fascinating, where large numbers of people suddenly rose up against an authoritarian and out-of-touch government. This largely peaceful uprising was fueled by the same Web2.0 technologies that are underpinning this class. It's possible that many traditional schools will soon find themselves in a similar situation to the Egyptian government, where students pass them by in favor of lower cost and more relevant educational experiences - many of them available online. Bill Gates recently came under fire for making the statement that valid online college degrees may soon be available for $2000. He may have said this to exaggerate a point, but the evidence of a growing number of online low cost educational resources can't be ignored. One of them is the Khan Academy - run by an MIT graduate who has created a large free archive of mathematics courses that are used by students all over the world - many who claim they are better than their university courses. Colleges and universities ignore this development at their own peril. It's not clear yet how we'll make this transition to relevancy for students, but the next decade will be very interesting...

Thanks you for your comments Mary Ann...the bottom line reads, we may be killing off our aged population by sentencing them to give up and give in to old out-moded concepts!

Joy: I've been following your blog postings for some time now and have been meaning to let you know I always learn and find your postings valuable. Keep the information coming our way.

Jenkins is an important player in this online learning stuff. This piece brings up some important considerations. We will be talking about assessment next week, but I wanted to toss this to you now, to get your gears spinning.

I cannot understand why he is so worried about open book testing of indivduals; rather than thinking about how to set up a test that would allow all contributors to weigh in and answer. What is the point of the process of collabrative learning then? Or am I just naive?

One response to this article was the practical - give the questions before the exam and the students can do whatever they want prior to the test, but when the test is actually given, make it closed book. This emphasizes information retnetion more than information retreival, which seems to be his concern. That being said - whether a test or not - his points are interesting and requires a great deal of thinking through. Allowing collaboration, for example as Joy profers, during a short course time brings up the issue of bandwidth and accessibility, so a teacher needs to spend time on logistics.

I can't recall if I shared this before, but it's appropriate for this week, especially for anyone doing blended learning. Tony is an instructor I used to work with at BTC. He's using laptops for small group work, where they work together on projects and he visits the groups one at a time to check for understanding. It's a good example of continuous assessment in a classroom setting. How could this type of model be translated to a fully online course?

An interesting blog post about a change in how we manage online courses and research at an institution.

I like the way he reasoned that you have to change the assessment of these classes and this in turn changes the classes and the time frame as well.

I must admit that I don't really have a good definition of what LMS means to the author. He refers to learning environments and gives the "follow the teacher" example, so I'll assume he means personalized elearning, but I would have preferred that he provided a definition. Anyway, I agree to have to change the assessment, as well as the expectations for the teacher. It would be a challenge for the teacher to keep up with many students, but probably also provide him or her some additional insights to his/her work.

This link is to my development of a learning activity that maps to a course learning objective for Module 7. The activity I've chosen to work on for this project is in response to one of the learning objectives for a class called Visual Storytelling (CMST 115) that I'm teaching for Bellevue College. The learning objective (or understanding) I'm addressing is a final project that has students demonstrate their ability to put into practice a selection of visual storytelling skills that were taught during the quarter. This activity was designed to make use of authentic learning principles, derived from the ideas of Marilyn Lombardi - which are available in complete form in her article, Authentic Learning for the 21st Century: an Overview.

Very impressive, Bruce! This assessment piece is definitely in line with Lombardi's ideas AND is a perfect showcase for student understanding of the objective you set forth. I imagine your students jumped at the challenge you presented them, producing outstanding work! You continue to inspire me with your thinking and creativity!

Thanks for your comments, Mary Ann. One of the big take aways I've gotten from our EDUC 251 class so far, is the idea of relinquishing progressive amounts of control throughout the quarter - so that students at the end are teaching part of the class and demonstrating their acquired skills in some kind of collaborative, hands-on project. I've been exploring this idea over the past few years as I've become more comfortable with my teaching - and both the Boettcher reading and Jennifer's course structure validate this approach. One of the more interesting recommendations for "authentic learning" is having ill-defined goals - meaning that students have to fill in gaps with their own solutions - derived from general or even vague instructions. This enhances creativity, innovative pathways, and problem solving. Cool idea!

This is a guide from Northern AZ U. I'm not a huge fan of rubrics for assessment, but I think they can sometimes help students get an idea of your expectations. If you can implement them without stifling creativity, they might be useful to you.

I think this course is a good example of how rubrics can fall short. We've got a small group of students with very diverse experience. You all want and need something very different from this course. Your projects should all be different. While I can give you some idea of expectations, it's difficult to plan in advance for your audience and still provide the individualized learning necessary in some disciplines and courses.

I find myself feeling somewhat torn regarding how the rubrics for online student participation are presented on this Northern Arizona University website. On one hand, it provides a rock-solid method for accurately measuring various levels of student engagement. On the other hand, it means the instructor has to parse out dozens of tiny point assignments for every student - ie. a potential micromanagement purgatory. Since I haven't tried this NAU recommended approach to online assessments, I should rein in my initial resistance, and take a plunge into this rubric pool to see how it works in practice.

In today's Seattle Times, A3 section - interesting article which followed over 2,000 US students from 2005 to 2009. 45% made no significant improvements in their critical thinking, reasoning or writing skills during first 2 years of study. After completing 4 years, 36 % showed no significant gains in "higher order" thinking skills. Article also states some contributing factors, many students work part-time, however, most students only spent 1/5 time on their studies and 51% on socializing & extra-curricular. I have a strong hunch the internet is a big contributor to the socializing factor. When I read such statistics, it makes me ponder if on-line learning can truly change this social/educational trend. I am enjoying many readings on the diigo, both from Jennifer and others, however, I'm not sure I would design an on-line class which designated a considerable amt. of time to short readings and discussions. Your thoughts on this?

I too found the article interesting - thanks! I do wonder about the types of classes students completed especially in their first two years. The might be a marked difference in skills acquisition if students were completing many required large class survey classes to meet initial requirements vs. small, more focused classes in their areas of interest.

I am still trying to get the picture of "slogging" through studies! Is that what I am doing when reading all this material in Diigo? I don't think so.

This is a research article where the authors studied classroom teachers who were required to convert to on-line learning. Some of the results I found interesting, was how the teachers who taught in traditional classrooms had challenges in re-structuring the information - primarily in the area of control information flow. More assessments, for example, had to be added. There is also a discussion of increased workload by the instructors and how this was a variable in the enthusiasm of the teacher's embracing the technologin.

The amount of time is always a big shock. And most of the time, it's difficult to get administration to understand that. It's something faculty unions are definitely taking into consideration with contract negotiations. I read this book a few years ago http://www.josseybass.com/WileyCDA/WileyTitle/productCd-0787987700.html and while it has some good ideas and information, I think they were also providing a huge stipend for redevelopment of courses.

This is my portfolio post for week 5

http://www.mercurynews.com/opinion/ci_17396167?nclick_check=1 - This is an OpEd piece by David Brooks of the New York Times, which came out this week. "It could be that the nature of technlogical changes is (causing) a shift in values". This is not an opinion about education, but goes to the point about preparing students for the future/present even that assumes technology. Maybe our values are and how we assess things has to evolve, as well.

I truly thought that blog was thought provoking! I absolutely agree about changing values. But, I think that change must come from this generation and what they want for themselves, for their community and for their country. Hopefully they can think that broadly about themselves and the world. I believe they can and do!

I have added my page on Wikispace. I have a video I want my learners to view. I have asked them to research the topic and present their blogs, videos and etc.,in Diigo. I have asked them to discuss this Module's question on the need or not for a Silver College similiar to the one in Japan for aging Japanese people. Japan is one step ahead of the U.S. in trying to deal with the many issues and needs of their aging population. Grading or assessment will be determined by the quality of the participation and discussion in Diigo. The Module discussion would be set up to continue for one week and is 25% of that Module's grade. The idea is not to grade the students but to encourage them to research the topic and bring to the discussion some ideas as to whether this is a useful idea or not if applied to the Senior population in America. I will be writing a blog regarding this lesson as well. I have not finished it yet.

Joy, I really like the way you set up the assignment related to the short film on how Japan is gaining huge benefits by engaging its senior population. When you think about it, this enlightened Japanese view to making good used of aging citizens isn't controversial at all. Farming retired folks out to finish their lives out in disconnected retirement homes seems so backward, and poorly planned. We should be gathering together all of that life experience through good design, and putting it to good use!

I hear you Bruce!

The author of this study is Penny Ralstom-Berg, a Senior Instruction Designer at University of Wisconsin, where a survey of what makes a good course from a student perspective was conducted. The summary of the study starts on slide 20. The top 10 desires is consistent with what we've been focusing on in this class, however the bottom top is somewhat at odds and seems, to be, counter-intuitive to what I would have expected. Top 10: Assessments are appropriately timed, varied and appropriate; Instructions on how to access resources are sufficient and easy to understand; Course components are web-based or easily downloaded for use offline; Requirements for interaction are clearly explained; Opportunity to speak with instructor, classmates through email, bulletin boards, any time / any place; Technologies required are readily available - provided or easily downloaded; Clear instructions tell me how to get started and to find course components; Criteria for how I will be evaluated is descriptive and specific; Navigation is logical, consistent, and efficient; The grading policy is clearly stated Bottom 10: I find course-related content and share it with the instructor and classmates; I use wikis, shared documents, or other online collaborative tools to complete group work; I am asked to introduce myself to the class; I coach other students and help answer their questions; Opportunity to speak with my instructor and classmates through… same time, same place technology; Instructor does not participate in class discussions Course contains interactive games or simulations; I author / create course-related content and share it with instructor and classmates; I work in groups with other students; Course contains audio and video content

Reading through these student recommendations was very helpful, and as you say - they complement the material presented in the Boettcher textbook very well. I was somewhat confused by Penny Ralton-Berg's differentiation between the Top 10 and Bottom 10 items. My impression is that there is a low student positive response for their contributing course materials for a class site.(??)

I am not certain I grasp the meaning of that slide presentation. Some of it I get and agree with.

A blog summarizing updates to my course framework on how the course would be assessed.