Group items tagged

Generally these papers are just pumped-up versions of the five-paragraph essay, with filler added. Thesis-driven, argumentative, like the newspaper editorials the genre is based on, this "researched writing" promises to solve big questions with little effort: "Reproductive rights resolved in five pages!"

Actual writing related to research is modest, qualified, and hesitant

our actual model involves elaborately respectful conversation, demonstrating sensitivity to the most nuanced claims of previous researchers

Academic, legal, medical, and business writing has easily understandable conventions. We responsibly survey the existing literature, formally or informally creating an annotated bibliography. We write a review of the literature, identifying a "blank" spot ignored by other scholars, or a "bright" spot where we see conflicting evidence. We describe the nature of our research in terms of a contribution to the blank or bright spot in that conversation. We conclude by pointing to further questions.

Millions of pieces of research writing that aren’t essays usefully circulate in the profession through any number of sharing technologies, including presentations and posters; grant and experiment proposals; curated, arranged, translated, or visualized data; knowledgeable dialogue in online media with working professionals; independent journalism, arts reviews, and Wikipedia entries; documentary pitches, scripts and storyboards; and informative websites.

real researchers don’t write a word unless they have something to contribute. We should teach our students to do the same

the world’s most important and most valuable AI company has been built on the backs of the collective work of humanity, often without permission, and without compensation to those who created it

LLMs usually cannot do a good job of explaining how a particular passage from a longer text illuminates the whole of that longer text. Moreover, ChatGPT’s outputs on comparison and contrast are often superficial. Typically the system breaks down a task of logical comparison into bite-size pieces, conveys shallow information about each of those pieces, and then formulaically “compares” and “contrasts” in a noticeably superficial or repetitive way. 

In-class writing, whether digital or handwritten, may have downsides for students with anxiety and disabilities

ChatGPT can produce outputs that take the form of  “brainstorms,” outlines, and drafts. It can also provide commentary in the style of peer review or self-analysis. Nonetheless, students would need to coordinate multiple submissions of automated work in order to complete this type of assignment with a text generator.  

No one should present auto-generated writing as their own on the expectation that this deception is undiscoverable. 

LLMs often mimic the harmful prejudices, misconceptions, and biases found in data scraped from the internet

Show students examples of inaccuracy, bias, logical, and stylistic problems in automated outputs. We can build students’ cognitive abilities by modeling and encouraging this kind of critique. Given that social media and the internet are full of bogus accounts using synthetic text, alerting students to the intrinsic problems of such writing could be beneficial. (See the “ChatGPT/LLM Errors Tracker,” maintained by Gary Marcus and Ernest Davis.)

Since ChatGPT is good at grammar and syntax but suffers from formulaic, derivative, or inaccurate content, it seems like a poor foundation for building students’ skills and may circumvent their independent thinking.

Good journalism on language models is surprisingly hard to find since the technology is so new and the hype is ubiquitous. Here are a few reliable short pieces.     “ChatGPT Advice Academics Can Use Now” edited by Susan Dagostino, Inside Higher Ed, January 12, 2023  “University students recruit AI to write essays for them. Now what?” by Katyanna Quach, The Register, December 27, 2022  “How to spot AI-generated text” by Melissa Heikkilä, MIT Technology Review, December 19, 2022  The Road to AI We Can Trust, Substack by Gary Marcus, a cognitive scientist and AI researcher who writes frequently and lucidly about the topic. See also Gary Marcus and Ernest Davis, “GPT-3, Bloviator: OpenAI’s Language Generator Has No Idea What It’s Talking About” (2020).

“On the Dangers of Stochastic Parrots” by Emily M. Bender, Timnit Gebru, et al, FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, March 2021. Association for Computing Machinery, doi: 10.1145/3442188. A blog post summarizing and discussing the above essay derived from a Critical AI @ Rutgers workshop on the essay: summarizes key arguments, reprises discussion, and includes links to video-recorded presentations by digital humanist Katherine Bode (ANU) and computer scientist and NLP researcher Matthew Stone (Rutgers).

existing efforts "focus on the interpretive responsibilities of the individual,"

"if bad actors intentionally dump disinformation online with an aim to distract and overwhelm, is it possible to safeguard against media manipulation?"

A 2012 meta-analysis by academic researchers found that media literacy efforts could help boost students' critical awareness of messaging, bias, and representation in the media they consumed. There have been small studies suggesting that media-literacy efforts can change students' behaviors—for example, by making them less likely to seek out violent media for their own consumption. And more recently, a pair of researchers found that media-literacy training was more important than prior political knowledge when it comes to adopting a critical stance to partisan media content.

the roles of institutions, technology companies, and governments

An hour of study tonight, an hour on the weekend, another session a week from now: such so-called spacing improves later recall, without requiring students to put in more overall study effort or pay more attention, dozens of studies have found.

“The idea is that forgetting is the friend of learning,” said Dr. Kornell. “When you forget something, it allows you to relearn, and do so effectively, the next time you see it.”

cognitive scientists see testing itself — or practice tests and quizzes — as a powerful tool of learning, rather than merely assessment. The process of retrieving an idea is not like pulling a book from a shelf; it seems to fundamentally alter the way the information is subsequently stored, making it far more accessible in the future.

“Testing not only measures knowledge but changes it,” he says — and, happily, in the direction of more certainty, not less.

“Testing has such bad connotation; people think of standardized testing or teaching to the test,” Dr. Roediger said. “Maybe we need to call it something else, but this is one of the most powerful learning tools we have.”

The harder it is to remember something, the harder it is to later forget. This effect, which researchers call “desirable difficulty,”

“What surprised me was the resurgence of many of the zombie ideas about online learning creeping into the discussions, such as the idea that online learning isn’t as personal, or that you can’t have interactivity, or that it just doesn’t work. And while it is true you need to change how you think about your course — you can’t just replicate what you used to do in the classroom — there’s an opportunity to evolve your teaching practices and create a better learning experience for your students.”

What’s happening now is going to reshape education for years, if not decades.

People want the old normal, not the new normal. We will, to some degree, get back to what we know and love, but it won’t ever look like it did before

“Like your physical buildings on campus, you also have a somewhat invisible set of resources called your educational technology. If you don’t understand it well and don’t treat it as important infrastructure, your ability to move online sustainably will be challenged. Sometimes institutions see eLearning as a project, not a strategy. Online learning isn’t a fly-by-the-seat-of-your-pants project; it has to be integrated into your academic plan and institutional strategy. I hope that COVID-19 underlined that for institutions.”

“We’ve known for over 30 years now that one-hour lectures are not a great way to teach: you can have a good one-hour session, but can you have 13 over a semester? It’s about cognitive load, and students can’t focus for more than 15 to 20 minutes at a time without being distracted. There’s room for synchronous discussion, but we can do it better. There’s a huge amount of research into online learning and what happens when students have access to online learning whenever they want it. And just like in real life, you have to know how to do both synchronous and asynchronous interactions well.”

We need to make space for the voices of communities who haven’t traditionally been heard: non-traditional learners, students who are food or housing insecure, students who are neurodivergent, students of colour, and Indigenous students. We must think of all these populations and the degree to which our educational system — our technology, our platforms — has not been built for them. We do a lot of work to make our methods accessible, but at the core, our systems, institutions, and platforms aren’t really built for — or by — those students

s challenging as it is, I’m seeing online pedagogy’s focus on equity and care resonating with many of those new to the medium

I’ve used really experimental styles over the past few years, but I won’t be doing that as much over the coming year because I shouldn’t. My classes are traditionally where students get to work with tools and platforms outside of the norm. If everyone moving online treats it that way, the cognitive load on the students will be absolutely overwhelming. My right to flex my academic freedom regarding platforms should be superseded by care and consideration for my students’ cognitive loads across a program. Navigating different platforms and tools is hard and distracting.

“One of the most vital tools and resources that I’ve seen people using is their human capacities for compassion and patience — the degree to which faculty are stepping up and approaching their students from a place of care, and a place of genuine desire for students to feel a sense of hope, safety, and flexibility.”

Silicon Valley’s sudden willingness to consider taking more reputational risk arrives as tech stocks are tumbling

A chatbot that pointed to one answer directly from Google could increase its liability if the response was found to be harmful or plagiarized.

AI has been through several hype cycles over the past decade, but the furor over DALL-E and ChatGPT has reached new heights.

Soon after OpenAI released ChatGPT, tech influencers on Twitter began to predict that generative AI would spell the demise of Google search. ChatGPT delivered simple answers in an accessible way and didn’t ask users to rifle through blue links. Besides, after a quarter of a century, Google’s search interface had grown bloated with ads and marketers trying to game the system.

Inside big tech companies, the system of checks and balances for vetting the ethical implications of cutting-edge AI isn’t as established as privacy or data security. Typically teams of AI researchers and engineers publish papers on their findings, incorporate their technology into the company’s existing infrastructure or develop new products, a process that can sometimes clash with other teams working on responsible AI over pressure to see innovation reach the public sooner.

Chatbots like OpenAI routinely make factual errors and often switch their answers depending on how a question is asked

To Timnit Gebru, executive director of the nonprofit Distributed AI Research Institute, the prospect of Google sidelining its responsible AI team doesn’t necessarily signal a shift in power or safety concerns, because those warning of the potential harms were never empowered to begin with. “If we were lucky, we’d get invited to a meeting,” said Gebru, who helped lead Google’s Ethical AI team until she was fired for a paper criticizing large language models.

Rumman Chowdhury, who led Twitter’s machine-learning ethics team until Elon Musk disbanded it in November, said she expects companies like Google to increasingly sideline internal critics and ethicists as they scramble to catch up with OpenAI.“We thought it was going to be China pushing the U.S., but looks like it’s start-ups,” she said.

DLA is explic­itly res­i­den­tial and dia­log­i­cal

Not so much a replace­ment as a sup­ple­ment to dig­i­tal human­i­ties, DLA broad­ens the scope and relo­cates the cen­ter of grav­ity of what I have referred to as the dig­i­tal human­i­ties sit­u­a­tion, the recur­ring, play­ful encounter of human­ists with tech­nol­ogy.  Instead of focus­ing on what may bet­ter be described as the com­pu­ta­tional human­i­ties (a use­ful term recently pro­posed by Lev Manovich), the dig­i­tal lib­eral arts seeks to locate dig­i­tal media squarely within the frame of the lib­eral arts, broadly con­ceived as a cur­ricu­lum, not a dis­ci­pline or even set of dis­ci­plines, and as a dis­tinc­tive mode of edu­ca­tional expe­ri­ence, not a set of received the­o­ret­i­cal con­cerns. It is a fram­ing par­tic­u­larly suited to lib­eral arts col­leges — America’s great con­tri­bu­tion to higher learn­ing — but also to uni­ver­si­ties, such as UVa, whose souls are in the lib­eral arts as well.

the idea of Coursera-style MOOCs being part of the DLA is a non-starter, although dis­trib­uted and medi­ated forms of edu­ca­tion can, and I think must, become part of the lib­eral arts experience

DLA is as con­cerned with ped­a­gogy as it is with research

focus­ing on the real use of dig­i­tal col­lec­tions (for exam­ple) as much as on their cre­ation and publication

The open model of participation calls into question where value is created in the education system. Gutenberg created a means to duplicate content. The social web creates the opportunity for many-to-many interactions and to add a global social layer on content creation and knowledge growth.

Whatever can be easily duplicated cannot serve as the foundation for economic value. Integration and connectedness are economic value points.

In education, content can easily be produced (it’s important but has limited economic value). Lectures also have limited value (easy to record and to duplicate). Teaching – as done in most universities – can be duplicated. Learning, on the other hand, can’t be duplicated. Learning is personal, it has to occur one learner at a time. The support needed for learners to learn is a critical value point.

Learning, however, requires a human, social element: both peer-based and through interaction with subject area experts

Content is readily duplicated, reducing its value economically. It is still critical for learning – all fields have core elements that learners must master before they can advance (research in expertise supports this notion). - Teaching can be duplicated (lectures can be recorded, Elluminate or similar webconferencing system can bring people from around the world into a class). Assisting learners in the learning process, correcting misconceptions (see Private Universe), and providing social support and brokering introductions to other people and ideas in the discipline is critical. - Accreditation is a value statement – it is required when people don’t know each other. Content was the first area of focus in open education. Teaching (i.e. MOOCs) are the second. Accreditation will be next, but, before progress can be made, profile, identity, and peer-rating systems will need to improve dramatically. The underlying trust mechanism on which accreditation is based cannot yet be duplicated in open spaces (at least, it can’t be duplicated to such a degree that people who do not know each other will trust the mediating agent of open accreditation)

The skills that are privileged and rewarded in a MOOC are similar to those that are needed to be effective in communicating with others and interacting with information online (specifically, social media and information sources like journals, databases, videos, lectures, etc.). Creative skills are the most critical. Facilitators and learners need something to “point to”. When a participant creates an insightful blog post, a video, a concept map, or other resource/artifact it generally gets attention.

Intentional diversity – not necessarily a digital skill, but the ability to self-evaluate ones network and ensure diversity of ideologies is critical when information is fragmented and is at risk of being sorted by single perspectives/ideologies.

The volume of information is very disorienting in a MOOC. For example, in CCK08, the initial flow of postings in Moodle, three weekly live sessions, Daily newsletter, and weekly readings and assignments proved to be overwhelming for many participants. Stephen and I somewhat intentionally structured the course for this disorienting experience. Deciding who to follow, which course concepts are important, and how to form sub-networks and sub-systems to assist in sensemaking are required to respond to information abundance. The process of coping and wayfinding (ontology) is as much a lesson in the learning process as mastering the content (epistemology). Learners often find it difficult to let go of the urge to master all content, read all the comments and blog posts.

e. Learning is a social trust-based process.

Patience, tolerance, suspension of judgment, and openness to other cultures and ideas are required to form social connections and negotiating misunderstandings.

An effective digital citizenry needs the skills to participate in important conversations. The growth of digital content and social networks raises the need citizens to have the technical and conceptual skills to express their ideas and engage with others in those spaces. MOOCs are a first generation testing grounds for knowledge growth in a distributed, global, digital world. Their role in developing a digital citizenry is still unclear, but democratic societies require a populace with the skills to participate in growing a society’s knowledge. As such, MOOCs, or similar open transparent learning experiences that foster the development of citizens confidence engage and create collaboratively, are important for the future of society.