Before I forget, a couple people at the symposium on Monday afternoon asked if I could post links to the series I was talking about in my Pecha Kucha, so here ya'll go.

Main channel: https://www.youtube.com/user/MarbleHornets

totheark: https://www.youtube.com/user/totheark

Here's a handy-dandy list that gives the viewing order, since totheark doesn't always post a response video to every MH entry:
https://www.youtube.com/playlist?list=PLCACDEE1E71FC8E5E

Be warned, you may not want to watch these alone or at night. I mean, I do, but I also possess a level of masochism that most others don't, so if this keeps you up at night, don't blame me.

Hey folks, here's the info for the conference this weekend:

Thursday, March 13th: Screening of Norman McLaren's Neighbours and other 16mm shorts at 3pm in CUAG. I've been told that there will be food provided, but don't quote me on that.

Friday, March 14th: Opening reception and keynote address with Haidee Wasson. It will start at 5:30 pm in CUAG and will run until 8 pm (ish). There WILL be food and a cash bar at this event. Here's a link to some research info about Dr Wasson:
http://cinema.concordia.ca/people/film-studies-faculty/full-time/wasson-haidee.php

Saturday, March 15th: Conference talks! The first panel will begin at 10 am in RB 1200, and will run until 5 pm at the very latest. There will be coffee, breakfast stuff and lunch provided.

Well, seeing as I started us off last week, I may as well do it again.

Given the growing influence of the Digital Humanities and its emphasis on multimodal and/or "buildable" projects - for example, Mark Sample advocates in doing away with essays entirely (404) - it makes sense that a new method of evaluation for this type of work would be required. Another driving factor in this need for a new system is the fact that the existing system is far from perfect; Jonathan Dresner outlines some of the problems with both absolute and relative systems of grading - grade decline and grade inflation, respectively ("Towards a Unified Theory of Grading"). The question then becomes how to create this new framework for evaluation. For the sake of simplicity, the framework I will discuss will focus on the evaluation of students' work, rather than the recognition of scholarly work by faculties and institutions. Based on reading a variety of articles discussing this subject, I feel that a combination of elements from the conventional system of grading within a framework of peer-review provides the most effective means of evaluation. Shannon Christine Mattern openly acknowledges that her own framework is a consolidation of a variety of different models ("Evaluating Multimodal Work, Revisited"), and it is a similar method that I am using for my own model, in combining her list of considerations along with Cathy Davidson's outlining of a peer-review or crowdsourced framework ("How to Crowdsource Grading").

Mattern provides a series of criteria and considerations for evaluating multimodal scholarship that is a particularly effective means of evaluating work from the position of an instructor. She separates her criteria into sections based on concept, design and technique, documentation, academic integrity and openness, and review and critique (EMWR). Several of her considerations for evaluation stem from pre-existing grading frameworks, such as the presence of a thesis or argument, evidence of research, whether the content of the project is supplemented with documented supporting evidence, and whether it can be said to adhere to a certain level of academic integrity (EMWR). One important question that is often asked in my field in relation to specific research is the question "so what?" If the project in question can self-reflexively approach and answer this question, then it has already proven its worth to the discipline. In fact, it is a question I have seen directly asked to presenters at Film Studies conferences. All of these elements are necessary for the consideration of the efficacy of any given project, regardless of its format or medium.

At the same time, there must be acknowledgement given to the specifics of digital or multimedia work, whether it is a website, a visualization, an interactive game, a tool, or any other form of project that could fall under the purview of the digital humanities. It is here that some of her other criteria can be used effectively, such as the effectiveness of the form or interface in use, the consideration of who the audience of the project may be, its accessibility, and whether it has been reviewed, either by experts or by presenting it at a conference (EMWR). There is one specific criterion that I think is particularly important, when she questions "need this have been a multimedia project, or could it just as easily have been executed on paper?" (EMWR) This gets to the heart of the intention behind a specific project, and to how effectively the technology behind it is being used. For example, in the interactive documentary Welcome to Pine Point, it is a question that the creators engage with directly when they admit that "it could have been a book, but it probably makes more sense that it became this" (The Goggles, "About this Project").

The second part of a successful framework for evaluation lies in the use of a peer-review system. This type of framework certainly has its advantages; when used effectively, it becomes a means of teaching students to think critically (Mattern, EMWR) and to engage with each other's work directly rather than just with the assigned readings. One method of peer-review that I have experienced as a student came from a course in which our final project underwent a "draft workshop" phase. All of us submitted our first draft copies to the other students in the course and then met in a workshop environment to discuss the strengths and weaknesses of each other's projects. Because it occurred before the completion of the assignment, we were able to revise and acknowledge the comments of our peers in the final version. Structuring the peer-review portion of the system as an ongoing and collaborative process allows the students to not only improve their work, but also may point out problems or issues that even the instructor may not notice.

While scholars like Cathy Davidson advocate for a purely peer-review based system (HCG), I feel that using a combination of adapted "conventional" criteria as outlined by Shannon Christine Mattern along with a draft or testing-phase peer-review is a much more effective means of evaluating digital humanities scholarship. One important factor that is always present in any evaluative framework is, as the Modern Language Association points out, the need for adaptability; there is no such thing as a universal framework ("Guidelines for Evaluating Work in Digital Humanities and Digital Media"). The key to a successful method is openness and transparency between the students and the instructor. Ensuring that the students understand the criteria by which they are evaluated - both by the instructor and by their peers - can prevent possible issues or complaints that may arise, which is something that Davidson mentions directly in the defense of her crowdsourced system (HCG). While I acknowledge that the framework I have outlined may not be perfect, I feel it is a more effective means of evaluating digital humanities scholarship than the existing system would allow.

Works Cited:
Davidson, Cathy "How to Crowdsource Grading" HASTAC. July 26, 2009.
http://www.hastac.org/blogs/cathy-davidson/how-crowdsource-grading

Dresner, Jonathan "Towards a Unified Theory of Grading" Dresner World History. N.d.
http://dresnerworld.edublogs.org/about/towards-a-unified-theory-of-grading/

The Goggles, "About this Project" Welcome to Pine Point, NFB Interactive, 2011.
http://pinepoint.nfb.ca/#/pinepoint

Mattern, Shannon Christine "Evaluating Multimodal Work Revisited" Words in Space, August 28, 2012.
http://www.wordsinspace.net/wordpress/2012/08/28/evaluating-multimodal-work-revisited/

Sample, Mark L. "What's Wrong with Writing Essays." Debates in the Digital Humanities. Ed. Matthew K. Gold. Minneapolis: University of Minnesota Press, 2012. 404-405.

I just want to take this opportunity to address the comments made by Chris about the evaluative method I outlined in this week's assignment (yay, peer review!).

I do agree with your assessment about the "finality" of essays. I think this gets to the heart of the comparison between "conventional" media and digital ones, as we've seen outlined before by Lev Manovich in comparing narrative and database. This references the concept that the internet (and everything involved with it) is ever-changing, flexible, and adaptable. Although not all essays provide a concrete conclusion to their argument, there is certainly an emphasis placed on that as an end goal.

I also agree with you that with the "traditional" writing assignment, the likelihood of active participation has an expiry date -- whereas a multimodal or digital-based projects have a much more extended shelf-life, as it were. This is something that Mattern references in her criteria by questioning the project's linkability and reviewability. Trevor Owens also mentions something along these lines when he discusses his use of blogs for his courses (411). It was this idea of an ongoing peer engagement with the works in question that I was trying to define, though I admittedly didn't do it as effectively as I would have liked.

I also appreciate that in responding to my assignment you're actively engaging with what I was attempting to advocate for in terms of a peer-review framework. I realize that using the word "phase" as a descriptor for the process I was describing was ill-advised, although it was an accurate description of my own specific experience that I was discussing. I had envisioned the peer-review framework as an ongoing process that could be repeated and adapted as the assignment in question required.

Lastly, in relation to my previous experience in the drafting workshop, I do agree that the motivation to fix mistakes prior to "official" assessment was a factor for all of us involved in the process. It did also encapsulate Davidson's notion of the "redo" as you mentioned, because the workshop was able to bring out some major issues and methodological problems that would have otherwise gone unnoticed until it was too late. As a result of this, our deadline for the assignment was actually extended, to allow us to fix our projects as needed based on the input of our peers in the course. Admittedly, the assignment was a conventional written essay, but it is a process that I feel can work effectively for other types of projects.

Works Cited:
Davidson, Cathy. "How to Crowdsource Grading"
http://www.hastac.org/blogs/cathy-davidson/how-crowdsource-grading

Owens, Trevor. "The Public Course Blog: The Required Reading We Write Ourselves for the Course That Never Ends" in Debates in the Digital Humanities. Ed. Matthew K. Gold. 409-411.

Manovich, Lev. "Database as Symbolic Form"
http://transcriptions.english.ucsb.edu/archive/courses/warner/english197/Schedule_files/Manovich/Database_as_symbolic_form.htm

Mattern, Shannon Christine. "Evaluating Multimodal Work, Revisited"
http://www.wordsinspace.net/wordpress/2012/08/28/evaluating-multimodal-work-revisited/

You definitely bring up some interesting points, Chris, but for the sake of brevity I'll just respond to those questions you outline at the end.

Like Jordon and Alessandro, I think the word limit is a great exercise. It forces us (and the hypothetical students) to really engage with the information we're trying to convey, in order to make is as concise as possible. I think having a start date for this after reading week works best, but maybe we should see what Brian thinks of it (since he is running the course, after all).

As a film student, I obviously love the idea of doing vlogs, but it seems to be one of those things that often work better in theory than in practice (at least in my experience). This is a concept that was brought up at our graduate conference last year, and the discussion around it became very heated. I guess it just boils down to the idea of how the technology itself is being used. As we discussed in class, if you're using a vlog format, you should be using the technology involved to its full potential, not just recording yourself saying the words you could just as easily use in an essay.

I think this ties into the final question I'll engage with, the notion of our work transcending the audience of our classmates. This is where using alternative mediums and formats would come in handy. I think, for the sake of the humanities, it is important that we're able to prove our importance (dare I say relevance) to those beyond the walls of the ivory tower. As I'm sure many of you have experienced, there's a perception currently that most degrees in the humanities are impractical (the dreaded question of "what will you do with THAT?). Having a larger online presence is beginning to change this somewhat, however we need to be willing to move beyond the physical written essay if we want the content of our research to be more widely disseminated and understood. It's much easier for a layman to engage with a vlog or even a blog than it is to engage with a 30 page article on obscure text x. I guess the question then becomes who we want our audience to be; should we be limiting or excluding certain audiences, and how should we articulate our research in order to engage with these audiences.

I can't help but wonder if the issue that Christina quotes (Jan 29) from Fitzpatrick (students sticking to old "analog" habits even when using digital formats) is a combination of force of habit and an unwillingness to "play" with current technology. In regards to the latter, even though I consistently see laptops and the occasional tablet being brought to class, it seems that most of them aren't using them beyond typing notes and (obviously) checking Facebook and emails. I consider myself to be fairly tech-savvy, and many of my friends (who are also students) turn to me when they have problems or questions about buying computers; the majority of them are looking for nothing more than something they can use to go on the internet and type their assignments. Many of them don't even seem to be aware of the fact that they can highlight and make annotations on pdf files or even that sites like Diigo or even Dropbox exist (full disclosure, I learned of both from Shawn Graham). Another potential factor in this is the fact that not all courses use digital formats for the readings, and the majority of students revert to analog as a means of keeping consistency between the notes and readings for all of their classes.

The first reason I gave, habit, is also a clear factor. Like Christina, until recently I also used primarily analog methods for course readings and note taking. Even though many of the readings I have had to use in Film Studies were available digitally, I would most often print them off and highlight and annotate on the physical copy, rather than doing so digitally. Part of my reason for doing this was that I found having the physical copy in front of me made it easier to reference in seminar discussions, rather than scrolling through documents trying to find the right section. I also found it helped me to remember the content of the reading better, as if the physical act of marking up the hard copy somehow made it stick in my brain better.

As for note taking, I'm sure many of you have noticed that I take my notes by hand. This is despite the fact that my handwriting is almost illegible. While Alessandro (Jan 30) outlines some of the benefits of using a digital method of note taking, I recently came across an article discussing a psychological study done comparing the two methods in terms of efficacy. In Wray Herbert's blog Ink on Paper, he discusses the study undertaken by Pam Mueller of Princeton and Daniel Oppenheimer of UCLA, comparing, in two separate scenarios, how students performed when tested on lecture material, and how their methods of note taking may impact their results. In both cases, the students who took notes by hand performed significantly better than those who had typed on a laptop. They found that typically the students who were typing their notes would merely copy verbatim the content of the lecture (although it sounds like Alessandro is an exception), whereas the student using notebooks were engaging with the material itself through paraphrasing and restructuring. I highly recommend reading the article (will be linked at the end) because I think the results are very interesting, and its something that I've noticed (as anecdotal evidence) among my own students. I'm curious what you guys think about this.

Wray Herbert's post: http://www.huffingtonpost.com/wray-herbert/ink-on-paper-some-notes-o_b_4681440.html

The two tools I have chosen to examine are Google Ngrams and Bookworm. I picked these two tools for several reasons, such as their potential applicability in my discipline, as well as the fact that one of the tools I was interested in using, Kaleidoscope, is only compatible for Apple computers. I will admit that in using both these tools I found myself initially uncomfortable, because the potential uses of all the possible tools is so very far from the type of research and "textual analysis" to which I am accustomed. It was not until reading Rockwell's argument that "we should rethink our tools on a principle of research as disciplined play" (Rockwell, 213) that I began to consider some of the possible ways in which I could use these tools, at least as a form of loose experimentation.

As a discipline, Film Studies tends not to engage - at least as a central focus - in historiography and analysis of written texts about film. Our engagement with written texts are typically limited to the use of theoretical structures and how the analytic work of others can be supported or countered based on our own analysis of certain films. Increasingly, however, there have been some shifts in focus towards examining the ways in which we study, and write about, films. In both Ngrams and Bookworm, it is possible to search for specific terms and track their usage in books over time based on the respective databases of each tool. This would allow for an analysis of what types of words or terminology are used in specific periods, and to track how that changes over time. It was for this reason that I chose to "play" with the two tools in question.
Because both are similar in terms of both interface and purpose, I decided to search and compare a set of specific terms relating to Film Studies in order to explore the ways in which they are used in the collections of both tools. In both Google Ngrams and Bookworm I searched for the terms "film," "movie," and "cinema" and noticed some interesting results. In Ngrams, the term "film" appeared much sooner and its incline is much steeper, but that it to be expected given the multiple definitions assigned to that word. "Cinema" and "movie" appear and begin to climb around the same time, in 1914. Interestingly, both "film" and "movie" begin to decline after the year 2000, while "cinema" remains fairly steady, even increasing marginally in 2013 and 2014. Similar results were found using Bookworm by searching the Open Library database, however because that database is smaller than that of Google, the results are much lower in terms of number and percentage

Both tools certainly have their advantages: Ngrams and Bookworm both have simple and easy-to-use interfaces. Ngrams specifically benefits from access to Google's expansive archive of digitized books. Bookworm is slightly more useful, in that it has more variability in settings than Ngrams, such as zooming in on specific periods, the option to search in different libraries, and the ability to search your own data by submitting a zip file. However, both still suffer from the main problem that Manovich points out, in that "what you can do with these tools today is quite limited" (Manovich, 470). While the ability to search a massive database like Google's is convenient, it does not allow for more finessed searches, such as making a distinction between fiction and non-fiction books, or even specific genres of literature. As such, while both these tools are interesting to use as a means of written textual analysis, their utility for Film Studies at this point remains in question.

Works cited:
Rockwell, Geoffrey. "What is Text Analysis, Really?" Literary and Linguistic Computing 18.2 (2003): 209-220.
Manovich, Lev. "Trending: The Promises and Challenges of Big Social Data" in Debates in the Digital Humanities. Ed. Matthew K. Gold. Minneapolis: University of Minnesota Press, 2012. 460-475.

So I tried sending this out via CuLearn, but I heard that some people didn't get it, so I figured I would try and send it out here. We're hosting our 4th annual conference on March 14th and 15th, and I thought the topic might include work that some have you have done in the past. The deadline for abstracts is this Friday (31st) but even if you don't want to submit anything, it would be great if some of you could make it out to the panels on Saturday. Here's the website with the CFP:

http://carleton.ca/filmstudies/bordercrossings/

So I know I'm a little late to the party for the first blog post, but to be fair I was somewhat distracted by preparing for my presentation. Also I seem to have a hard enough time keeping up with my own personal blog, so I don't know why I thought this would be any different. I do, however, want to engage with the some of the points that others have brought up relating to the concept of community.

Obviously it is all well and good to aim for a broad, inclusive, interdisciplinary sense of community within Digital Humanities, but as Svensson points out in his piece, this idea is much easier said than done. I do, for the most part, agree with Danuta's use of Anderson as justification for using the term "community" rather than some of the alternatives, but I think that some of the reasoning people use for questioning it has less to do with DH's self-described inclusivity, and more to do with divisions already existing within the field. As Danuta pointed out community "is a socially constructed notion based on shared affinities that certain individuals possess" (Jan 19), but the issue, I think, stems from the fact that there are a number of different affinities at play in DH; namely, the debates surrounding the concept of making things or, as Brian put it in class, "less yak, more hack." Even the manifesto engages in this debate, when discussing the "diminshers" of DH and arguing that "[t]hey have rarely, if ever, built software, parsed code, created a database, or designed a user interface" (The Digital Humanities Manifesto v2.0, 10). While the manifesto is obviously referring to those in the academy who would argues that those "building" practices do not constitute legitimate scholarly pursuits, the language used implies that there is a division of priorities between making and everything else. I am not attempting to argue that the Digital Humanities can not, or should not, use the term community to describe themselves; however, it must be acknowledged that - as with many things - it seems that the term is much easier to use in theory than in practice.

I find the concept of community to be especially appealing given my own experiences in my discipline. Film Studies, despite its dependence on technology, has been surprisingly slow to embrace studies of digital films or the Digital Humanities. And many of those who do research that could be considered DH in nature do not tend to think of their work as such. This is especially true of the program here at Carleton, which no longer has a faculty member whose focus includes any form of digital technology. As well, even though they were hiring a new professor last year, they were looking for specialists in "American Cinema" and neither of the final candidates were researching films made after 1980. I love the program here at Carleton, and the people in it, but this near-sighted (maybe even backward) approach to the medium is rather concerning. As well, even though our program and faculty is very social (we spend a lot of time together outside of school) there is no collaboration to be found. At the end of the day, everyone returns to their own isolated research interests. I truly wish that the program will be more willing to embrace the Digital Humanities in the future, and perhaps this new Specialization will aid in speeding up this process.