Skip to main content

Home/ CTLT and Friends/ Group items tagged study

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
Gary Brown

Does testing for statistical significance encourage or discourage thoughtful ... - 1 views

genuineevaluation.com/urage-thoughtful-data-analysis

statistics

shared by Gary Brown on 27 Oct 10 - No Cached
  • Does testing for statistical significance encourage or discourage thoughtful data analysis? Posted by Patricia Rogers on October 20th, 2010
  • Epidemiology, 9(3):333–337). which argues not only for thoughtful interpretation of findings, but for not reporting statistical significance at all.
  • We also would like to see the interpretation of a study based not on statistical significance, or lack of it, for one or more study variables, but rather on careful quantitative consideration of the data in light of competing explanations for the findings.
  • ...6 more annotations...
  • we prefer a researcher to consider whether the magnitude of an estimated effect could be readily explained by uncontrolled confounding or selection biases, rather than simply to offer the uninspired interpretation that the estimated effect is significant, as if neither chance nor bias could then account for the findings.
  • Many data analysts appear to remain oblivious to the qualitative nature of significance testing.
  • statistical significance is itself only a dichotomous indicator.
  • it cannot convey much useful information
  • Even worse, those two values often signal just the wrong interpretation. These misleading signals occur when a trivial effect is found to be ’significant’, as often happens in large studies, or when a strong relation is found ’nonsignificant’, as often happens in small studies.
  • Another useful paper on this issue is Kristin Sainani, (2010) “Misleading Comparisons: The Fallacy of Comparing Statistical Significance”Physical Medicine and Rehabilitation, Vol. 2 (June), 559-562 which discusses the need to look carefully at within-group differences as well as between-group differences, and at sub-group significance compared to interaction. She concludes: ‘Readers should have a particularly high index of suspicion for controlled studies that fail to report between-group comparisons, because these likely represent attempts to “spin” null results.”
  • Gary Brown on 27 Oct 10
     
    and at sub-group significance compared to interaction. She concludes: 'Readers should have a particularly high index of suspicion for controlled studies that fail to report between-group comparisons, because these likely represent attempts to "spin" null results."
Joshua Yeidel

Strategic Directives for Learning Management System Planning | EDUCAUSE - 1 views

www.educause.edu/...213945

coursemanagement lms

shared by Joshua Yeidel on 21 Sep 10 - No Cached
  • Joshua Yeidel on 21 Sep 10
     
    A largely sensible strategic look at LMS in general. "The LMS, because of its integration with other major institutional technology systems, has itself become an enterprise-wide system. As such, higher education leaders must closely 7 monitor the possible tendency for LMSs to contribute only to maintaining the educational status quo.40 The most radical suggestion for future LMS use would dissolve the commercially enforced "course-based" model of LMS use entirely, allowing for the creation of either larger (departmental) or smaller (study groups) units of LMS access, as the case may require. This ability to cater to context awareness is perhaps the feature most lacking in most LMS products. As noted in a study in which mobile or handheld devices were used to assemble ad hoc study groups,41 this sort of implementation is entirely possible in ways that don't necessarily require interaction through an LMS interface." Requires EDUCAUSE login (free to WSU)
  • Joshua Yeidel on 23 Sep 10
     
    The EDUCAUSE paper suggests "dissolv[ing] the commercially enforced 'course-based' model of LMS". How about dissolving the "course-based" model of higher education on which the commercial LMS is based?
Gary Brown

Want Students to Take an Optional Test? Wave 25 Bucks at Them - Students - The Chronicl... - 0 views

chronicle.com/...65370

CLA motivation

shared by Gary Brown on 04 May 10 - Cached
  • cash, appears to be the single best approach for colleges trying to recruit students to volunteer for institutional assessments and other low-stakes tests with no bearing on their grades.
  • American Educational Research Association
  • A college's choice of which incentive to offer does not appear to have a significant effect on how students end up performing, but it can have a big impact on colleges' ability to round up enough students for the assessments, the study found.
  • ...6 more annotations...
  • "I cannot provide you with the magic bullet that will help you recruit your students and make sure they are performing to the maximum of their ability," Mr. Steedle acknowledged to his audience at the Denver Convention Center. But, he said, his study results make clear that some recruitment strategies are more effective than others, and also offer some notes of caution for those examining students' scores.
  • The study focused on the council's Collegiate Learning Assessment, or CLA, an open-ended test of critical thinking and writing skills which is annually administered by several hundred colleges. Most of the colleges that use the test try to recruit 100 freshmen and 100 seniors to take it, but doing so can be daunting, especially for colleges that administer it in the spring, right when the seniors are focused on wrapping up their work and graduating.
  • The incentives that spurred students the least were the opportunity to help their college as an institution assess student learning, the opportunity to compare themselves to other students, a promise they would be recognized in some college publication, and the opportunity to put participation in the test on their resume.
  • The incentives which students preferred appeared to have no significant bearing on their performance. Those who appeared most inspired by a chance to earn 25 dollars did not perform better on the CLA than those whose responses suggested they would leap at the chance to help out a professor.
  • What accounted for differences in test scores? Students' academic ability going into the test, as measured by characteristics such as their SAT scores, accounted for 34 percent of the variation in CLA scores among individual students. But motivation, independent of ability, accounted for 5 percent of the variation in test scores—a finding that, the paper says, suggests it is "sensible" for colleges to be concerned that students with low motivation are not posting scores that can allow valid comparisons with other students or valid assessments of their individual strengths and weaknesses.
  • A major limitation of the study was that Mr. Steedle had no way of knowing how the students who took the test were recruited. "If many of them were recruited using cash and prizes, it would not be surprising if these students reported cash and prizes as the most preferable incentives," his paper concedes.
  • Gary Brown on 04 May 10
     
    Since it is not clear if the incentive to participate in this study influenced the decision to participate, it remains similarly unclear if incentives to participate correlate with performance.
Joshua Yeidel

News: A Jobs Mismatch - Inside Higher Ed - 0 views

www.insidehighered.com/...jobs

higher_education

shared by Joshua Yeidel on 08 Jul 10 - Cached
  • Joshua Yeidel on 08 Jul 10
     
    A report of a Georgetown University study that describes the shift in the job market toward requiring post-secondary education (up from 20% of jobs in 1973 to 59% of jobs in 2018) and the looming shortfall of college-eduvated workers. The director of the study opines that that means more vocational orientation in college education, but the report stops short (thankfully) of that mis-conclusion.
Gary Brown

Declining Study Time Signals Falling Standards, Report Says - The Ticker - The Chronicl... - 0 views

chronicle.com/...26028

study

shared by Gary Brown on 06 Aug 10 - Cached
  • The amount of time spent studying has fallen drastically among full-time students in all demographic groups, whether they work or not, at all types of four-year colleges, according to a report released on Thursday by the American Enterprise Institute.
  • they cater to students' preferences for leisure, the report says, a shift that may slow economic growth.
  • Gary Brown on 06 Aug 10
     
    Perhaps a partial explanation for resistance to innovations that promote authentic and group work?
Gary Brown

Struggling Students Can Improve by Studying Themselves, Research Shows - Teaching - The... - 3 views

chronicle.com/...64004

feedback peer review

shared by Gary Brown on 13 Feb 10 - Cached
  • "We're trying to document the role of processes that are different from standard student-outcome measures and standard ability measures,
  • We're interested in various types of studying, setting goals for oneself, monitoring one's progress as one goes through learning a particular topic."
  • Mr. Zimmerman has spent most of his career examining what can go wrong when people try to learn new facts and skills. His work centers on two common follies: First, students are often overconfident about their knowledge, assuming that they understand material just because they sat through a few lectures or read a few chapters. Second, students tend to attribute their failures to outside forces ("the teacher didn't like me," "the textbook wasn't clear enough") rather than taking a hard look at their own study habits.
  • ...14 more annotations...
  • That might sound like a recipe for banal lectures about study skills. But training students to monitor their learning involves much more than simple nagging, Mr. Zimmerman says. For one thing, it means providing constant feedback, so that students can see their own strengths and weaknesses.
  • or one thing, it means providing constant feedback, so that students can see their own strengths and weaknesses.
  • "The first one is, Give students fast, accurate feedback about how they're doing. And the second rule, which is less familiar to most people, is, Now make them demonstrate that they actually understand the feedback that has been given."
  • "I did a survey in December," he says. "Only one instructor said they were no longer using the technique. Twelve people said they were using the technique 'somewhat,' and eight said 'a lot.' So we were pleased that they didn't forget about us after the program ended."
  • "Only one instructor said they were no longer using the technique. Twelve people said they were using the technique 'somewhat,' and eight said 'a lot.' So we were pleased that they didn't forget about us after the program ended."
  • And over time, we've realized that these methods have a much greater effect if they're embedded within the course content.
  • "Once we focus on noticing and correcting errors in whatever writing strategy we're working on, the students just become junkies for feedback,"
  • "Errors are part of the process of learning, and not a sign of personal imperfection," Mr. Zimmerman says. "We're trying to help instructors and students see errors not as an endpoint, but as a beginning point for understanding what they know and what they don't know, and how they can approach problems in a more effective way."
  • Errors are part of the process of learning, and not a sign of personal imperfection,"
  • Self-efficacy" was coined by Albert Bandura in the 1970's
  • "Self-efficacy" was coined by Albert Bandura in the 1970's,
  • The 1990 paper from _Educational Psychologist_ 25 (1), pp. 3-17) which is linked above DOES include three citations to Bandura's work.
  • The 1990 paper from _Educational Psychologist_ 25 (1), pp. 3-17) which is linked above DOES include three citations to Bandura's work.
  • What I am particularly amazed by is that the idea of feedback, reflection and explicitly demonstrated understanding (essentially a Socratic approach of teaching), is considered an innovation.
  • Gary Brown on 13 Feb 10
     
    selected for the focus on feedback. The adoption by half or fewer, depending, is also interesting as the research is of the type we would presume to be compelling.
Nils Peterson

Innovating the 21st-Century University: It's Time! (EDUCAUSE Review) | EDUCAUSE - 4 views

www.educause.edu/...195370

educause university2.0

shared by Nils Peterson on 06 Feb 10 - Cached
  • change is required in two vast and interwoven domains that permeate the deep structures and operating model of the university: (1) the value created for the main customers of the university (the students); and (2) the model of production for how that value is created. First we need to toss out the old industrial model of pedagogy (how learning is accomplished) and replace it with a new model called collaborative learning. Second we need an entirely new modus operandi for how the subject matter, course materials, texts, written and spoken word, and other media (the content of higher education) are created.
  • Research shows that mutual exploration, group problem solving, and collective meaning-making produce better learning outcomes and understanding overall. Brown and Adler cite a study by Richard J. Light, of the Harvard Graduate School of Education: "Light discovered that one of the strongest determinants of students' success in higher education . . . was their ability to form or participate in small study groups. Students who studied in groups, even only once a week, were more engaged in their studies, were better prepared for class, and learned significantly more than students who worked on their own."
  • Second, the web enables students to collaborate with others independent of time and geography. Finally, the web represents a new mode of production for knowledge, and that changes just about everything regarding how the "content" of college and university courses are created.
  • ...7 more annotations...
  • As Seymour Papert, one of the world's foremost experts on how technology can provide new ways to learn, put it: "The scandal of education is that every time you teach something, you deprive a [student] of the pleasure and benefit of discovery."14 Students need to integrate new information with the information they already have — to "construct" new knowledge structures and meaning.
  • Universities need an entirely new modus operandi for how the content of higher education is created. The university needs to open up, embrace collaborative knowledge production, and break down the walls that exist among institutions of higher education and between those institutions and the rest of the world.To do so, universities require deep structural changes — and soon. More than three years ago, Charles M. Vest published "Open Content and the Emerging Global Meta-University" in EDUCAUSE Review. In his concluding paragraph, Vest offered a tantalizing vision: "My view is that in the open-access movement, we are seeing the early emergence of a meta-university — a transcendent, accessible, empowering, dynamic, communally constructed framework of open materials and platforms on which much of higher education worldwide can be constructed or enhanced. The Internet and the Web will provide the communication infrastructure, and the open-access movement and its derivatives will provide much of the knowledge and information infrastructure." Vest wrote that the meta-university "will speed the propagation of high-quality education and scholarship. . . . The emerging meta-university, built on the power and ubiquity of the Web and launched by the open courseware movement, will give teachers and learners everywhere the ability to access and share teaching materials, scholarly publications, scientific works in progress, teleoperation of experiments, and worldwide collaborations, thereby achieving economic efficiencies and raising the quality of education through a noble and global endeavor."17
  • Used properly, wikis are tremendously powerful tools to collaborate and co-innovate new content. Tapscott wrote the foreword for a book called We Are Smarter Than Me (2008). The book, a best-seller, was written by Barry Libert, Jon Spector, and more than 4,000 people who contributed to the book's wiki. If a global collaboration can write a book, surely one could be used to create a university course. A professor could operate a wiki with other teachers. Or a professor could use a wiki with his or her students, thereby co-innovating course content with the students themselves. Rather than simply being the recipients of the professor's knowledge, the students co-create the knowledge on their own, which has been shown to be one of the most effective methods of learning.
  • The student might enroll in the primary college in Oregon and register to take a behavioral psychology course from Stanford University and a medieval history course from Cambridge. For these students, the collective syllabi of the world form their menu for higher education. Yet the opportunity goes beyond simply mixing and matching courses. Next-generation faculty will create a context whereby students from around the world can participate in online discussions, forums, and wikis to discover, learn, and produce knowledge as networked individuals and collectively.
  • But what about credentials? As long as the universities can grant degrees, their supremacy will never be challenged." This is myopic thinking. The value of a credential and even the prestige of a university are rooted in its effectiveness as a learning institution. If these institutions are shown to be inferior to alternative learning environments, their capacity to credential will surely diminish. How much longer will, say, a Harvard undergraduate degree, taught mostly through lectures by teaching assistants in large classes, be able to compete in status with the small class size of liberal arts colleges or the superior delivery systems that harness the new models of learning?
  • As part of this, the academic journal should be disintermediated and the textbook industry eliminated. In fact, the word textbook is an oxymoron today. Content should be multimedia — not just text. Content should be networked and hyperlinked bits — not atoms. Moreover, interactive courseware — not separate "books" — should be used to present this content to students, constituting a platform for every subject, across disciplines, among institutions, and around the world. The textbook industry will never reinvent itself, however, since legacy cultures and business models die hard. It will be up to scholars and students to do this collectively.
  • Ultimately, we will need more objective measures centered on students' learning performance.
Gary Brown

Why Don't Students Study Anymore? - Percolator - The Chronicle of Higher Education - 1 views

chronicle.com/...23631

students

shared by Gary Brown on 30 Apr 10 - Cached
  • Here's what they found: In 1961, the average full-time college student spent 40 hours per week on academic work (that's time in class and studying). In 2003, it was 27 hours. The authors figure that 21st-century students spend an average of 10 hours fewer every week studying than their 1961 counterparts. Over the course of a four-year college career, that would add up to something like 1,500 fewer hours spent hitting the books.
  • the difference isn't caused by more people stretching out their college experience. Also, according to the authors, the difference can't be explained by the fact that more students have jobs or by the fact that the makeup of the student body has changed since the sixties. From the paper: "The large decline in academic time investment is an important pattern its own right, and one that motivates future research into underlying causes."In other words, we don't know why. 
  • Here's the abstract for the working paper, which was written by Philip S. Babcock and Mindy Marks
  • Gary Brown on 30 Apr 10
     
    the reasons may be in the readers comments, or maybe not.
Kimberly Green

Would You Like Credit With That Internship? - Students - The Chronicle of Higher Education - 0 views

chronicle.com/...65434

learning

shared by Kimberly Green on 10 May 10 - Cached
  • Kimberly Green on 10 May 10
     
    Students pay tuition to work for free ... unpaid internships are growing in this down economy, favoring the wealthy who can afford them. But internships are generally valuable for students, administrators say. The complementary courses involve journals, essays, oral presentations, or work portfolios. Independent studies lean toward academics. Companies often see academic credit as substitute compensation that qualifies interns as legally unpaid trainees and keeps them on their colleges' liability insurance. Advertisements specify: "Candidates must be able to receive academic credit." That makes some campus officials bristle. "What they're saying is holding the institution hostage," says Kathy L. Sims, director of career services at the University of California at Los Angeles. Employers don't know colleges' academic standards, she says. "It's really not their call whether their experience is creditworthy." Colleges have dealt with that quandary in various ways. Some, especially those with traditions of experiential learning, vet and monitor internships, enrolling students in courses designed to complement their real-world work. Others let professors sponsor independent studies based on internships. More and more have devised some form of noncredit recognition to try to satisfy employers without altering academic philosophies or making students pay tuition to work free. At Bates College, the game is up. "We're quite adamant about our refusal to play along," says James W. Hughes, a professor of economics. As chairman of the department eight years ago, he got dozens of calls from students, parents, and employers asking for credit for unpaid internships, mainly in the financial industry. "Why is it that we have to evaluate this experience," he says, "just so some multibillion-dollar bank can avoid paying $7.50 an hour?" But the law is vague, and arguably antiquated. In the for-profit sector, guidelines for legally unpaid internships come from a 1947 U
Theron DesRosier

OECD Feasibility Study for the International Assessment of Higher Education Learning Ou... - 3 views

www.oecd.org/...61291_40624662_1_1_1_1,00.html

assessment accountability accreditation

shared by Theron DesRosier on 01 Jul 10 - Cached
  • Theron DesRosier on 01 Jul 10
     
    "What is AHELO? The OECD Assessment of Higher Education Learning Outcomes (AHELO) is a ground-breaking initiative to assess learning outcomes on an international scale by creating measures that would be valid for all cultures and languages. Between ten and thirty-thousand higher education students in over ten different countries will take part in a feasibility study to determine the bounds of this ambitious project, with an eye to the possible creation of a full-scale AHELO upon its completion."
Joshua Yeidel

Start-Up Aspires to Make the World 'One Big Study Group' - Wired Campus - The Chronicle... - 2 views

chronicle.com/...26780

higher_education learning web_2.0

shared by Joshua Yeidel on 13 Sep 10 - No Cached
  • Joshua Yeidel on 13 Sep 10
     
    A more modest attempt to "open up" education, using a study-group model rather than a course model like P2PU.
Gary Brown

Measuring Student Learning: Many Tools - Measuring Stick - The Chronicle of Higher Educ... - 2 views

chronicle.com/...27541

assessment

shared by Gary Brown on 28 Nov 10 - No Cached
  • The issue that needs to be addressed and spectacularly has been avoided is whether controlled studies (one group does the articulation of and then measurement of outcomes, and a control group does what we have been doing before this mania took hold) can demonstrate or falsify the claim that outcomes assessment results in better-educated students. So far as I can tell, we instead gather data on whether we have in fact been doing outcomes assessment. Not the issue, people. jwp
  • Gary Brown on 28 Nov 10
     
    The challenge--not the control study this person calls for, but the perception that outcomes assessment produces outcomes....
Theron DesRosier

School of Environmental Studies: Taking Class Outdoors | Edutopia - 0 views

www.edutopia.org/ses-video

theme based school environment

shared by Theron DesRosier on 14 Nov 08 - Cached
  • Theron DesRosier on 14 Nov 08
     
    School of Environmental Studies: Taking Class Outdoors At this Minnesota school focused on problem solving, the academic program and the architecture were designed to promote interdisciplinary project-based learning. More to this story.
Theron DesRosier

We Are Media » About Project Background - 0 views

www.wearemedia.org/About+Project+Background

university 2.0 community service

shared by Theron DesRosier on 07 Nov 08 - No Cached
  • The We Are Media Project is a community of people from nonprofits who are interested in learning and teaching about how social media strategies and tools can enable nonprofit organizations to create, compile, and distribute their stories and change the world. Curated by NTEN, the community will work in a networked way to help identify the best existing resources, people, and case studies that will give nonprofit organizations the knowledge and resources they need to be the media. The community will help identify and point to the best how-to guides and useful resources that cover all aspects of creating, aggregating, and distributing social media. The resulting curriculum which will live on this wiki and will also cover important organizational adoption issues, strategy, ROI analysis, as well as the tools.
  • Theron DesRosier on 07 Nov 08
     
    The We Are Media Project is a community of people from nonprofits who are interested in learning and teaching about how social media strategies and tools can enable nonprofit organizations to create, compile, and distribute their stories and change the world.\nCurated by NTEN, the community will work in a networked way to help identify the best existing resources, people, and case studies that will give nonprofit organizations the knowledge and resources they need to be the media. The community will help identify and point to the best how-to guides and useful resources that cover all aspects of creating, aggregating, and distributing social media. The resulting curriculum which will live on this wiki and will also cover important organizational adoption issues, strategy, ROI analysis, as well as the tools.
  • Theron DesRosier on 14 Nov 08
     
    Thanks Stephen, great bookmark. We are thinking about Change.gov right now. Wondering how we make it less broadcast and more 2.0.
Gary Brown

OECD Project Seeks International Measures for Assessing Educational Quality - Internati... - 0 views

chronicle.com/...63764

assessment

shared by Gary Brown on 28 Jan 10 - Cached
  • The first phase of an ambitious international study that intends to assess and compare learning outcomes in higher-education systems around the world was announced here on Wednesday at the conference of the Council for Higher Education Accreditation.
  • Richard Yelland, of the OECD's Education Directorate, is leading the project, which he said expects to eventually offer faculty members, students, and governments "a more balanced assessment of higher-education quality" across the organization's 31 member countries.
  • learning outcomes are becoming a central focus worldwide
  • ...7 more annotations...
  • the feasibility study is adapting the Collegiate Learning Assessment, an instrument developed by the Council for Aid to Education in the United States, to an international context.
  • At least six nations are participating in the feasibility study.
  • 14 countries are expected to participate in the full project, with an average of 10 institutions per country and about 200 students per institution,
  • The project's target population will be students nearing the end of three-year or four-year degrees, and will eventually measure student knowledge in economics and engineering.
  • While the goal of the project is not to produce another global ranking of universities, the growing preoccupation with such lists has crystallized what Mr. Yelland described as the urgency of pinning down what exactly it is that most of the world's universities are teaching and how well they are doing
  • Judith S. Eaton, president of the Council for Higher Education Accreditation, said she was also skeptical about whether the project would eventually yield common international assessment mechanisms.
  • Ms. Eaton noted, the same sets of issues recur across borders and systems, about how best to enhance student learning and strengthen economic development and international competitiveness.
  • Gary Brown on 28 Jan 10
     
    Another day, another press, again thinking comparisons
Jayme Jacobson

Evaluating the effect of peer feedback on the quality of online discourse - 0 views

ritdml.rit.edu/10092

peer feedback embedded assessment micro-feedback

shared by Jayme Jacobson on 09 Mar 10 - No Cached
  • Results indicate that continuous, anonymous, aggregated feedback had no effect on either the students' or the instructors' perception of discussion quality.
  • Jayme Jacobson on 09 Mar 10
     
    Abstract: This study explores the effect on discussion quality of adding a feedback mechanism that presents users with an aggregate peer rating of the usefulness of the participant's contributions in online, asynchronous discussion. Participants in the study groups were able to specify the degree to which they thought any posted comment was useful to the discussion. Individuals were regularly presented with feedback (aggregated and anonymous) summarizing peers' assessment of the usefulness of their contribution, along with a summary of how the individuals rated their peers. Results indicate that continuous, anonymous, aggregated feedback had no effect on either the students' or the instructors' perception of discussion quality. This is kind of a show-stopper. It's just one study but when you look at the results there appears to be no effect whatsoever from peers giving feedback about the usefulness of discussion posts, nor any perceived improvement in the quality of the discussions as evaluated by faculty. It looks like we'll need to begin looking carefully at just what kinds of feedback will really make a difference. Following up on Corinna's earlier post http://blogs.hbr.org/cs/2010/03/twitters_potential_as_microfee.html about the effectiveness of short immediate feedback being more effective than lengthier feedback that actually hinders performance. The trick will be to figure out just what kinds of feedback will actually work in embedded situations. It's interesting that an assessment of utility wasn't useful...?
Nils Peterson

AAC&U News | April 2010 | Feature - 1 views

www.aacu.org/...feature.cfm

shared by Nils Peterson on 01 Apr 10 - Cached
  • Comparing Rubric Assessments to Standardized Tests
  • First, the university, a public institution of about 40,000 students in Ohio, needed to comply with the Voluntary System of Accountability (VSA), which requires that state institutions provide data about graduation rates, tuition, student characteristics, and student learning outcomes, among other measures, in the consistent format developed by its two sponsoring organizations, the Association of Public and Land-grant Universities (APLU), and the Association of State Colleges and Universities (AASCU).
  • And finally, UC was accepted in 2008 as a member of the fifth cohort of the Inter/National Coalition for Electronic Portfolio Research, a collaborative body with the goal of advancing knowledge about the effect of electronic portfolio use on student learning outcomes.  
  • ...13 more annotations...
  • outcomes required of all UC students—including critical thinking, knowledge integration, social responsibility, and effective communication
  • “The wonderful thing about this approach is that full-time faculty across the university  are gathering data about how their  students are doing, and since they’ll be teaching their courses in the future, they’re really invested in rubric assessment—they really care,” Escoe says. In one case, the capstone survey data revealed that students weren’t doing as well as expected in writing, and faculty from that program adjusted their pedagogy to include more writing assignments and writing assessments throughout the program, not just at the capstone level. As the university prepares to switch from a quarter system to semester system in two years, faculty members are using the capstone survey data to assist their course redesigns, Escoe says.
  • the university planned a “dual pilot” study examining the applicability of electronic portfolio assessment of writing and critical thinking alongside the Collegiate Learning Assessment,
  • The rubrics the UC team used were slightly modified versions of those developed by AAC&U’s Valid Assessment of Learning in Undergraduate Education (VALUE) project. 
  • In the critical thinking rubric assessment, for example, faculty evaluated student proposals for experiential honors projects that they could potentially complete in upcoming years.  The faculty assessors were trained and their rubric assessments “normed” to ensure that interrater reliability was suitably high.
  • “It’s not some nitpicky, onerous administrative add-on. It’s what we do as we teach our courses, and it really helps close that assessment loop.”
  • There were many factors that may have contributed to the lack of correlation, she says, including the fact that the CLA is timed, while the rubric assignments are not; and that the rubric scores were diagnostic and included specific feedback, while the CLA awarded points “in a black box”:
  • faculty members may have had exceptionally high expectations of their honors students and assessed the e-portfolios with those high expectations in mind—leading to results that would not correlate to a computer-scored test. 
  • “The CLA provides scores at the institutional level. It doesn’t give me a picture of how I can affect those specific students’ learning. So that’s where rubric assessment comes in—you can use it to look at data that’s compiled over time.”
  • Their portfolios are now more like real learning portfolios, not just a few artifacts, and we want to look at them as they go into their third and fourth years to see what they can tell us about students’ whole program of study.”  Hall and Robles are also looking into the possibility of forming relationships with other schools from NCEPR to exchange student e-portfolios and do a larger study on the value of rubric assessment of student learning.
  • “We’re really trying to stress that assessment is pedagogy,”
  • “We found no statistically significant correlation between the CLA scores and the portfolio scores,”
  • In the end, Escoe says, the two assessments are both useful, but for different things. The CLA can provide broad institutional data that satisfies VSA requirements, while rubric-based assessment provides better information to facilitate continuous program improvement.
    • Nils Peterson
      Nils Peterson on 01 Apr 10
       
      CLA did not provide information for continuous program improvement -- we've heard this argument before
  • Gary Brown on 01 Apr 10
     
    The lack of correlation might be rephrased--there appears to be no corrlation between what is useful for faculty who teach and what is useful for the VSA. A corollary question: Of what use is the VSA?
Gary Brown

Professors Who Focus on Honing Their Teaching Are a Distinct Breed - Research - The Chr... - 1 views

chronicle.com/...65262

teaching

shared by Gary Brown on 27 Apr 10 - Cached
  • Professors who are heavily focused on learning how to improve their teaching stand apart as a very distinct subset of college faculties, according to a new study examining how members of the professoriate spend their time.
  • those who are focused on tackling societal problems stand apart as their own breed. Other faculty members, it suggests, are pretty much mutts, according to its classification scheme.
  • 1,000 full-time faculty members at four-year colleges and universities gathered as part of the Faculty Professional Performance Survey administered by Mr. Braxton and two Vanderbilt doctoral students in 1999. That survey had asked the faculty members how often they engaged in each of nearly 70 distinct scholarly activities, such as experimenting with a new teaching method, publishing a critical book review in a journal, or being interviewed on a local television station. All of the faculty members examined in the new analysis were either tenured or tenure-track and fell into one of four academic disciplines: biology, chemistry, history, or sociology.
  • ...7 more annotations...
  • cluster analysis,
  • nearly two-thirds of those surveyed were involved in the full range of scholarly activity they examined
  • Just over a third, however, stood out as focused almost solely on one of two types of scholarship: on teaching practices, or on using knowledge from their discipline to identify or solve societal problems.
  • pedagogy-focused scholars were found mainly at liberal-arts colleges and, compared with the general population surveyed, tended to be younger, heavily represented in history departments, and more likely to be female and untenured
  • Those focused on problem-solving were located mainly at research and doctoral institutions, and were evenly dispersed across disciplines and more likely than others surveyed to be male and tenured.
  • how faculty members rate those priorities are fairly consistent across academic disciplines,
  • The study was conducted by B. Jan Middendorf, acting director of Kansas State University's office of educational innovation and evaluation; Russell J. Webster, a doctoral student in psychology at Kansas State; and Steve Benton, a senior research officer at the IDEA Center
  • Gary Brown on 27 Apr 10
     
    Another study that documents the challenge and suggests confirmation of the 50% figure of faculty who are not focused on either research or teaching.
Gary Brown

Validity and Reliability in Higher Education Assessment - 2 views

rapidintellect.com/...10assess.htm

validity

shared by Gary Brown on 26 May 10 - Cached
  • Validity and Reliability in Higher Education Assessment
  • However, validity and reliability are not inherent features of assessments or assessment systems and must be monitored continuously throughout the design and implementation of an assessment system. Research studies of a theoretical or empirical nature addressing methodology for ensuring and testing validity and reliability in the higher education assessment process, results of validity and reliability studies, and novel approaches to the concepts of validity and reliability in higher education assessment are all welcome. To be most helpful in this academic exchange, empirical studies should be clear and explicit about their methodology so that others can replicate or advance their research.
  • Gary Brown on 26 May 10
     
    We should take this opportunity seriously and write up our work. Let me know if you want to join me.
Theron DesRosier

P2PU - Peer 2 Peer University / FrontPage - 0 views

www.p2pu.org

p2pu edupunks

shared by Theron DesRosier on 13 Aug 09 - Cached
  • The Peer 2 Peer University (P2PU) is an online community of open study groups for short university-level courses. Think of it as online book clubs for open educational resources. The P2PU helps you navigate the wealth of open education materials that are out there, creates small groups of motivated learners, and supports the design and facilitation of courses. Students and tutors get recognition for their work, and we are building pathways to formal credit as well.
  • Theron DesRosier on 13 Aug 09
     
    "The Peer 2 Peer University (P2PU) is an online community of open study groups for short university-level courses. Think of it as online book clubs for open educational resources. The P2PU helps you navigate the wealth of open education materials that are out there, creates small groups of motivated learners, and supports the design and facilitation of courses. Students and tutors get recognition for their work, and weare building pathways to formal credit as well."
1 - 20 of 103 Next › Last »
Showing 20▼ items per page