Skip to main content

Home/ CTLT and Friends/ Group items tagged standards

Rss Feed Group items tagged

Theron DesRosier

CDC Evaluation Working Group: Framework - 2 views

  • Framework for Program Evaluation
  • Purposes The framework was developed to: Summarize and organize the essential elements of program evaluation Provide a common frame of reference for conducting evaluations Clarify the steps in program evaluation Review standards for effective program evaluation Address misconceptions about the purposes and methods of program evaluation
  • Assigning value and making judgments regarding a program on the basis of evidence requires answering the following questions: What will be evaluated? (i.e. what is "the program" and in what context does it exist) What aspects of the program will be considered when judging program performance? What standards (i.e. type or level of performance) must be reached for the program to be considered successful? What evidence will be used to indicate how the program has performed? What conclusions regarding program performance are justified by comparing the available evidence to the selected standards? How will the lessons learned from the inquiry be used to improve public health effectiveness?
  • ...3 more annotations...
  • These questions should be addressed at the beginning of a program and revisited throughout its implementation. The framework provides a systematic approach for answering these questions.
  • Steps in Evaluation Practice Engage stakeholders Those involved, those affected, primary intended users Describe the program Need, expected effects, activities, resources, stage, context, logic model Focus the evaluation design Purpose, users, uses, questions, methods, agreements Gather credible evidence Indicators, sources, quality, quantity, logistics Justify conclusions Standards, analysis/synthesis, interpretation, judgment, recommendations Ensure use and share lessons learned Design, preparation, feedback, follow-up, dissemination Standards for "Effective" Evaluation Utility Serve the information needs of intended users Feasibility Be realistic, prudent, diplomatic, and frugal Propriety Behave legally, ethically, and with due regard for the welfare of those involved and those affected Accuracy Reveal and convey technically accurate information
  • The challenge is to devise an optimal — as opposed to an ideal — strategy.
  •  
    Framework for Program Evaluation by the CDC This is a good resource for program evaluation. Click through "Steps and Standards" for information on collecting credible evidence and engaging stakeholders.
Gary Brown

The Potential Impact of Common Core Standards - 2 views

  • According to the Common Core State Standards Initiative (CCSSI), the goal “is to ensure that academic expectations for students are of high quality and consistent across all states and territories.” To educators across the nation, this means they now have to sync up all curriculum in math and language arts for the benefit of the students.
  • They are evidence based, aligned with college and work expectations, include rigorous content and skills, and are informed by other top performing countries.”
  • “Educational standards help teachers ensure their students have the skills and knowledge they need to be successful by providing clear goals for student learning.” They are really just guidelines for students, making sure they are on the right track with their learning.
  • ...2 more annotations...
  • When asked the simple question of what school standards are, most students are unable to answer the question. When the concept is explained, however, they really do not know if having common standards would make a difference or not. Codie Allen, a senior in the Vail School Distract says, “I think that things will pretty much stay stagnate, people aren’t really going to change because of standards.”
  • Council of Chief State School Officers. Common Core State Standards Initiative, 2010.
Gary Brown

Online Colleges and States Are at Odds Over Quality Standards - Wired Campus - The Chro... - 1 views

  • the group called for a more uniform accreditation standard across state lines as well as a formal framework for getting a conversation on regulation started.
  • College officials claim that what states really mean when they discuss quality in online education is the credibility of online education in general. John F. Ebersole, president of Excelsior College, said “there is a bit of a double standard” when it comes to regulating online institutions; states, he feels, apply stricter standards to the online world.
  •  
    I note the underlying issue of "credibility" as the core of accreditation. It raises the question, again:  Why would standardized tests be presumed, as Excelsior does, to be a better indicator than a model of stakeholder endorsement?
Gary Brown

Online Colleges and States Are at Odds Over Quality Standards - Wired Campus - The Chro... - 2 views

  • But state officials said they are still concerned that self-imposed standards are not good enough and that online programs are not consistent in providing students with high-quality education.
  • “We’re very interested in making sure that as many good opportunities are available to students as possible,” added David Longanecker, president of the Western Interstate Commission.
  • he group called for a more uniform accreditation standard across state lines as well as a formal framework for getting a conversation on regulation started. Even with the framework in place, however, the state representatives said it will be difficult to get state-education agencies and state legislatures to agree. “Trying to bring 50 different people together is really tough,” Mr. Longanecker said.
  • ...1 more annotation...
  • Like state regulators, colleges are also facing hard decisions on quality standards. With such a diversity in online institutions, Ms. Eaton said it will be difficult to impose a uniform set of standards. “If we were in agreement about quality,” she said, “somebody’s freedom would be compromised.”
  •  
    I am dismayed to see Longanecker's position on this.
Gary Brown

NWCCU Standards Review Project - 0 views

  • Welcome to NWCCU's Standards Review Project
  •  
    A resource to track. The latest here is July 2nd's NWCCU new standards.
Gary Brown

Disciplines Follow Their Own Paths to Quality - Faculty - The Chronicle of Higher Educa... - 2 views

  • But when it comes to the fundamentals of measuring and improving student learning, engineering professors naturally have more to talk about with their counterparts at, say, Georgia Tech than with the humanities professors at Villanova
    • Gary Brown
       
      Perhaps this is too bad....
  • But there is no nationally normed way to measure the particular kind of critical thinking that students of classics acquire
  • er colleagues have created discipline-specific critical-reasoning tests for classics and political science
  • ...5 more annotations...
  • Political science cultivates skills that are substantially different from those in classics, and in each case those skills can't be measured with a general-education test.
  • he wants to use tests of reasoning that are appropriate for each discipline
  • I believe Richard Paul has spent a lifetime articulating the characteristics of discipline based critical thinking. But anyway, I think it is interesting that an attempt is being made to develop (perhaps) a "national standard" for critical thinking in classics. In order to assess anything effectively we need a standard. Without a standard there are no criteria and therefore no basis from which to assess. But standards do not necessarily have to be established at the national level. This raises the issue of scale. What is the appropriate scale from which to measure the quality and effectiveness of an educational experience? Any valid approach to quality assurance has to be multi-scaled and requires multiple measures over time. But to be honest the issues of standards and scale are really just the tip of the outcomes iceberg.
    • Gary Brown
       
      Missing the notion that the variance is in the activity more than the criteria.  We hear little of embedding nationally normed and weighted assignments and then assessing the implementation and facilitation variables.... mirror, not lens.
  • the UW Study of Undergraduate Learning (UW SOUL). Results from the UW SOUL show that learning in college is disciplinary; therefore, real assessment of learning must occur (with central support and resources)in the academic departments. Generic approaches to assessing thinking, writing, research, quantitative reasoning, and other areas of learning may be measuring something, but they cannot measure learning in college.
  • It turns out there is a six week, or 210+ hour serious reading exposure to two or more domains outside ones own, that "turns on" cross domain mapping as a robust capability. Some people just happen to have accumulated, usually by unseen and unsensed happenstance involvements (rooming with an engineer, son of a dad changing domains/careers, etc.) this minimum level of basics that allows robust metaphor based mapping.
Theron DesRosier

Obama proposes to raise academic standards by linking them to state benchmarks - washin... - 1 views

  •  
    "President Obama announced Monday he will seek to raise academic standards across the country by requiring states to certify that their benchmarks for reading and mathematics put students on track for college or a career. "
  •  
    NCLB Revives.
Theron DesRosier

Ethics in Assessment. ERIC Digest. - 2 views

  •  
    "Those who are involved with assessment are unfortunately not immune to unethical practices. Abuses in preparing students to take tests as well as in the use and interpretation of test results have been widely publicized. Misuses of test data in high-stakes decisions, such as scholarship awards, retention/promotion decisions, and accountability decisions, have been reported all too frequently. Even claims made in advertisements about the success rates of test coaching courses have raised questions about truth in advertising. Given these and other occurrences of unethical behavior associated with assessment, the purpose of this digest is to examine the available standards of ethical practice in assessment and the issues associated with implementation of these standards. "
Joshua Yeidel

Performance Assessment | The Alternative to High Stakes Testing - 0 views

  •  
    " The New York Performance Standards Consortium represents 28 schools across New York State. Formed in 1997, the Consortium opposes high stakes tests arguing that "one size does not fit all." Despite skepticism that an alternative to high stakes tests could work, the New York Performance Standards Consortium has done just that...developed an assessment system that leads to quality teaching, that enhances rather than compromises our students' education. Consortium school graduates go on to college and are successful."
Joshua Yeidel

The Tower and The Cloud | EDUCAUSE - 0 views

  •  
    "The emergence of the networked information economy is unleashing two powerful forces. On one hand, easy access to high-speed networks is empowering individuals. People can now discover and consume information resources and services globally from their homes. Further, new social computing approaches are inviting people to share in the creation and edification of information on the Internet. Empowerment of the individual -- or consumerization -- is reducing the individual's reliance on traditional brick-and-mortar institutions in favor of new and emerging virtual ones. Second, ubiquitous access to high-speed networks along with network standards, open standards and content, and techniques for virtualizing hardware, software, and services is making it possible to leverage scale economies in unprecedented ways. What appears to be emerging is industrial-scale computing -- a standardized infrastructure for delivering computing power, network bandwidth, data storage and protection, and services. Consumerization and industrialization beg the question "Is this the end of the middle?"; that is, what will be the role of "enterprise" IT in the future? Indeed, the bigger question is what will become of all of our intermediating institutions? This volume examines the impact of IT on higher education and on the IT organization in higher education."
  •  
    Consumerization and industrialization beg the question "Is this the end of the middle?"; that is, what will be the role of "enterprise" IT in the future? Indeed, the bigger question is what will become of all of our intermediating institutions? This volume examines the impact of IT on higher education and on the IT organization in higher education.
Gary Brown

Accrediting Agencies Confront New Challenges - Letters to the Editor - The Chronicle of... - 0 views

  • The Chronicle, December 17). In an era of global expansion in higher education, accreditation agencies are increasingly confronted with myriad challenges surrounding various forms of distance education (whether virtual, so-called branch campuses, or study abroad) and cross-institutional certification.
  • the American Academy for Liberal Education is particularly well placed to view this changing pedagogical and institutional landscape, both domestically and worldwide.
  • AALE goes several steps further in evaluating whether institutions meet an extensive set of pedagogical standards specifically related to liberal education—standards of effective reasoning, for instance, and broad and deep learning. This level of assessment requires extensive classroom visitations, conversations with students and faculty members, and the time to assess the climate of learning at every institution we visit.
  • ...1 more annotation...
  • Innovation and quality in higher education can only join hands when institutions aspire—and are held to—independent, third-party standards of assessment.
  •  
    a small but clear stress made for independent review
Joshua Yeidel

Gary Flake: is Pivot a turning point for web exploration? | Video on TED.com - 2 views

  •  
    "Gary Flake demos Pivot, a new way to browse and arrange massive amounts of images and data online. Built on breakthrough Seadragon technology, it enables spectacular zooms in and out of web databases, and the discovery of patterns and links invisible in standard web browsing."
  •  
    "Gary Flake demos Pivot, a new way to browse and arrange massive amounts of images and data online. Built on breakthrough Seadragon technology, it enables spectacular zooms in and out of web databases, and the discovery of patterns and links invisible in standard web browsing."
Gary Brown

Ethics? Let's Outsource Them! - Brainstorm - The Chronicle of Higher Education - 4 views

  • Many students are already buying their papers from term-paper factories located in India and other third world countries. Now we are sending those papers back there to be graded. I wonder how many people are both writing and grading student work, and whether, serendipitously, any of those people ever get the chance to grade their own writing.”
  • The great learning loop of outcomes assessment is neatly “closed,” with education now a perfect, completed circle of meaningless words.
  • With outsourced grading, it’s clearer than ever that the world of rubrics behaves like that wicked southern plant called kudzu, smothering everything it touches. Certainly teaching and learning are being covered over by rubrics, which are evolving into a sort of quasi-religious educational theory controlled by priests whose heads are so stuck in playing with statistics that they forget to try to look openly at what makes students turn into real, viable, educated adults and what makes great, or even good, teachers.
  • ...2 more annotations...
  • Writing an essay is an art, not a science. As such, people, not instruments, must take its measure, and judge it. Students have the right to know who is doing the measuring. Instead of going for outsourced grading, Ms. Whisenant should cause a ruckus over the size of her course with the administration at Houston. After all, if she can’t take an ethical stand, how can she dare to teach ethics?
  • "People need to get past thinking that grading must be done by the people who are teaching.” Sorry, Mr. Rajam, but what you should be saying is this: Teachers, including those who teach large classes and require teaching assistants and readers, need to get past thinking that they can get around grading.
  •  
    the outsourcing loop becomes a diatribe against rubrics...
  •  
    It's hard to see how either outsourced assessment or harvested assessment can be accomplished convincingly without rubrics. How else can the standards of the teacher be enacted by the grader? From there we are driven to consider how, in the absence of a rubric, the standards of the teacher can be enacted by the student. Is it "ethical" to use the Potter Stewart standard: "I'll know it when I see it"?
  •  
    Yes, who is the "priest" in the preceding rendering--one who shares principles of quality (rubrics), or one who divines a grade a proclaims who is a "real, viable, educated adult"?
Joshua Yeidel

The Answer Sheet - A principal on standardized vs. teacher-written tests - 0 views

  •  
    High school principal George Wood eloquently contrasts standardized NCLB-style testing with his school's performance assessments.
Joshua Yeidel

Refining the Recipe for a Degree, Ingredient by Ingredient - Government - The Chronicle... - 1 views

  • Supporters of the Lumina project say it holds the promise of turning educational assessment from a process that some academics might view as a threat into one that holds a solution, while also creating more-rigorous expectations for student learning. Mr. Jones, the Utah State history-department chairman, recounted in an essay published in the American Historical Association's Perspectives on History how he once blithely told an accreditation team that "historians do not measure their effectiveness in outcomes." But he has changed his mind. The Lumina project, and others, help define what learning is achieved in the process of earning a degree, he said, moving beyond Americans' heavy reliance on the standardized student credit hour as the measure of an education. "The demand for outcomes assessment should be seized as an opportunity for us to actually talk about the habits of mind our discipline needs to instill in our students," Mr. Jones wrote. "It will do us a world of good, and it will save us from the spreadsheets of bureaucrats."
  •  
    Lumina Foundation pushes a Eurpopean-style process to define education goals state- and nation-wide, with mixed success. "Chemistry, history, math, and physics have been among the most successful", whileothers have had a hard time beginning. "Supporters of the Lumina project say it holds the promise of turning educational assessment from a process that some academics might view as a threat into one that holds a solution, while also creating more-rigorous expectations for student learning. Mr. Jones, the Utah State history-department chairman, recounted in an essay published in the American Historical Association's Perspectives on History how he once blithely told an accreditation team that "historians do not measure their effectiveness in outcomes." But he has changed his mind. The Lumina project, and others, help define what learning is achieved in the process of earning a degree, he said, moving beyond Americans' heavy reliance on the standardized student credit hour as the measure of an education. "The demand for outcomes assessment should be seized as an opportunity for us to actually talk about the habits of mind our discipline needs to instill in our students," Mr. Jones wrote. "It will do us a world of good, and it will save us from the spreadsheets of bureaucrats."
Joshua Yeidel

Brainstorm - The Occupation Will Be Televised - The Chronicle of Higher Education - 0 views

  •  
    The poster in the accompanying picture says: "Education is not for sale". "In response to the massive re-orientation of education toward job training, privatization and the standardization of curricular outcomes mandated by the Bologna Process, students across Europe have been turning out by the thousands. This past June, as many as 250,000 students, parents, schoolteachers, college faculty and staff coordinated a week-long education strike in 90 cities across Germany."
  •  
    Apropos of Ashley's comments about European views of accreditation and accountability: apparently standardization of curricular outcomes is facing some opposition.
Gary Brown

Renewed Debate Over the 3-Year B.A. - The Chronicle Review - The Chronicle of Higher Ed... - 0 views

  • Zemsky, chairman of the Learning Alliance for Higher Education, wrote in The Chronicle in August. Shifting to a three-year baccalaureate, Zemsky added, would force universities to "judge whether their shorter degree programs were achieving the same learning outcomes as their four-year programs had promised; they would find themselves in need of the performance measures they had hitherto eschewed." The idea has stirred some support, as well as considerable opposition.
  • the reality is that the question of whether or not this makes sense may have already been made for us by the Bologna Process, which has been moving toward mainstreaming and standardizing three-year degrees across the European Union and beyond (46 countries are participating) for some time now.
  • This idea treats an academic credit as a purchasable commodity, and a college experience as quantifiable, subject to rules of efficiency rather than humane values. In reality, so-called "credits" have no standard meaning or value. Furthermore, the idea on its own is superficial. Why not two years? One? Five?
  • ...3 more annotations...
  • For those who see college as a place to learn marketable skills, the less time and money it takes, the better. For others who see college as a place to learn to think and to learn about the world and others as broadly as possible, and to grow into one's own, why rush? (The Choice, NYTimes.com)
  • they might want to rethink not just what time of year and how long students are in the classroom, but how student accomplishment is measured.
  • The high schools are not going to suddenly become more rigorous because the colleges reduce their expectations.
  •  
    Today's rip-tide toward measures, this time from Alexander and Zemsky (and others), and the implications of standardized measures.
Gary Brown

The Quality Question - Special Reports - The Chronicle of Higher Education - 1 views

shared by Gary Brown on 30 Aug 10 - Cached
  • Few reliable, comparable measures of student learning across colleges exist. Standardized assessments like the Collegiate Learning Assessment are not widely used—and many experts say those tests need refinement in any case.
    • Gary Brown
       
      I am hoping the assumptions underlying this sentence do not frame the discussion. The extent to which it has in the past parallels the lack of progress. Standardized comparisons evince nothing but the wrong questions.
  • "We are the most moribund field that I know of," Mr. Zemsky said in an interview. "We're even more moribund than county government."
  • Robert Zemsky
Gary Brown

Lawmakers Focus Ire on Accreditors for Abuses at For-Profit Colleges - Government - The... - 1 views

  • Six weeks after vowing to cull the bad apples from the for-profit higher-education sector, some Senate Democrats are asking whether the whole barrel is spoiled, and largely blaming accreditors for the rot.
  • Pressed by lawmakers, Mr. Kutz faulted the Education Department, saying it has failed in its oversight of the sector.
  • "Consistent oversight is going to be necessary," he said.
  • ...6 more annotations...
  • Senate Democrats focused most of their ire on accreditors, grilling the head of one national agency about the standards it uses to judge institutions.
  • "Do you think maybe your rigorous standards aren't rigorous enough?" asked Senator Franken, of Minnesota.
  • it was "apparent to me that we need a hearing on accreditation."
  • "But your on-site evaluations didn't detect it," Senator Harkin said. "It seems like you accept the schools' word on what they're doing."
  • Mr. Harkin said he planned to "look into" the financing structure of the accrediting system, saying it "seems to be a situation that is rife with conflict."
  • That should please committee Republicans, who called for a broader investigation into higher education. Sen. Michael B. Enzi of Wyoming, the top Republican on the education panel, said he would ask the GAO to expand its investigation to nonprofit colleges as well.
  •  
    Though the focus is on recruiting and finance of the for-profits, it is hard not to see more emphasis on verification coming down the pike....
Nils Peterson

Half an Hour: Open Source Assessment - 0 views

  • When posed the question in Winnipeg regarding what I thought the ideal open online course would look like, my eventual response was that it would not look like a course at all, just the assessment.
    • Nils Peterson
       
      I remembered this Downes post on the way back from HASTAC. It is some of the roots of our Spectrum I think.
  • The reasoning was this: were students given the opportunity to attempt the assessment, without the requirement that they sit through lectures or otherwise proprietary forms of learning, then they would create their own learning resources.
  • In Holland I encountered a person from an organization that does nothing but test students. This is the sort of thing I long ago predicted (in my 1998 Future of Online Learning) so I wasn't that surprised. But when I pressed the discussion the gulf between different models of assessment became apparent.Designers of learning resources, for example, have only the vaguest of indication of what will be on the test. They have a general idea of the subject area and recommendations for reading resources. Why not list the exact questions, I asked? Because they would just memorize the answers, I was told. I was unsure how this varied from the current system, except for the amount of stuff that must be memorized.
    • Nils Peterson
       
      assumes a test as the form of assessment, rather than something more open ended.
  • ...8 more annotations...
  • As I think about it, I realize that what we have in assessment is now an exact analogy to what we have in software or learning content. We have proprietary tests or examinations, the content of which is held to be secret by the publishers. You cannot share the contents of these tests (at least, not openly). Only specially licensed institutions can offer the tests. The tests cost money.
    • Nils Peterson
       
      See our Where are you on the spectrum, Assessment is locked vs open
  • Without a public examination of the questions, how can we be sure they are reliable? We are forced to rely on 'peer reviews' or similar closed and expert-based evaluation mechanisms.
  • there is the question of who is doing the assessing. Again, the people (or machines) that grade the assessments work in secret. It is expert-based, which creates a resource bottleneck. The criteria they use are not always apparent (and there is no shortage of literature pointing to the randomness of the grading). There is an analogy here with peer-review processes (as compared to recommender system processes)
  • What constitutes achievement in a field? What constitutes, for example, 'being a physicist'?
  • This is a reductive theory of assessment. It is the theory that the assessment of a big thing can be reduced to the assessment of a set of (necessary and sufficient) little things. It is a standards-based theory of assessment. It suggests that we can measure accomplishment by testing for accomplishment of a predefined set of learning objectives.Left to its own devices, though, an open system of assessment is more likely to become non-reductive and non-standards based. Even if we consider the mastery of a subject or field of study to consist of the accomplishment of smaller components, there will be no widespread agreement on what those components are, much less how to measure them or how to test for them.Consequently, instead of very specific forms of evaluation, intended to measure particular competences, a wide variety of assessment methods will be devised. Assessment in such an environment might not even be subject-related. We won't think of, say, a person who has mastered 'physics'. Rather, we might say that they 'know how to use a scanning electron microscope' or 'developed a foundational idea'.
  • We are certainly familiar with the use of recognition, rather than measurement, as a means of evaluating achievement. Ludwig Wittgenstein is 'recognized' as a great philosopher, for example. He didn't pass a series of tests to prove this. Mahatma Gandhi is 'recognized' as a great leader.
  • The concept of the portfolio is drawn from the artistic community and will typically be applied in cases where the accomplishments are creative and content-based. In other disciplines, where the accomplishments resemble more the development of skills rather than of creations, accomplishments will resemble more the completion of tasks, like 'quests' or 'levels' in online games, say.Eventually, over time, a person will accumulate a 'profile' (much as described in 'Resource Profiles').
  • In other cases, the evaluation of achievement will resemble more a reputation system. Through some combination of inputs, from a more or less define community, a person may achieve a composite score called a 'reputation'. This will vary from community to community.
  •  
    Fine piece, transformative. "were students given the opportunity to attempt the assessment, without the requirement that they sit through lectures or otherwise proprietary forms of learning, then they would create their own learning resources."
1 - 20 of 81 Next › Last »
Showing 20 items per page