Ashley Bradford of Critique-It describes his company’s way of keeping review and feedback engaging.

To what extent can and should we allow classroom feedback to be available in the public sphere? The classroom is a type of Habermasian civic society. Owning one’s discourse in that environment is critical. It has to feel human if students are to learn.

So, you can embed text, audio, and video feedback in documents, video and images. It translates docs into HTML. To make the feedback feel human, it uses slightly stamps. You can also type in comments, marking them as neutral, positive, or critique. A “critique panel” follows you through the doc as you read it, so you don’t have to scroll around. It rolls up comments and stats for the student or the faculty.

It works the same in different doc types, including Powerpoint, images, and video.

Critiques can be shared among groups. Groups can be arbitrarily defined.

It uses HTML 5. It’s written in Javascript, PHP, and uses Mysql.

“We’re starting with an environment. We’re building out tools.” Ashley aims for Critique^It to feel very human.

2 Comments »

Andy Wasklewicz and Jeff Austin from Entwine [twitter:entwinemedia] describe a multi-institutional project to build a platform-agnostic tool for enriching video through note-taking, structured annotations, and sharing. It uses HTML 5, and allows for structured tagging, time-based annotation, and more.

Be the first to comment »

Jonah Bossewich and Mark Philipsonfrom Columbia University talk about Mediathread, an open source project that makes it easy to annotate various digital sources. It’s used in many courses at Columbi, as well as around the world.

It comes from Columbia’s Center for New Media Teaching and Learning. It began with Vital, a video library tool. It let students clip and save portions of videos, and comment on them. Mediathread connects annotations to sources by bookmarking, via a bookmarklet that interoperates with a variety of collections. The bookmarklet scrapes the metadata because “We couldn’t wait for the standards to be developed.” Once an item is in Mediathread, it embeds the metadata as well.

It has always been conceived of a “small-group sharing and collaboration space.” It’s designed for classes. You can only see the annotations by people in your class. It does item-level annotation, as well as regions.

Mediathread connects assignments and responses, as well as other workflows. [He’s talking quickly :)]

Mediathread’s bookmarklet approach requires it to have to accommodate the particularities of sites. They are aiming at making the annotations interoperable in standard forms.

Be the first to comment »

Paolo Ciccarese begins by reminding us just how vast the scientific literature is. We can’t possibly read everything we should. But “science is social” so we rely on each other, and build on each other’s work. “Everything we do now is connected.”

Today’s media do provide links, but not enough. Things are so deeply linked. “How do we keep track of it?” How do we communicate with others so that when they read the same paper they get a little bit of our mental model, and see why we found the article interesting?

Paolo’s project — Domeo [twitter:DomeoTool] — is a web app for “producing, browsing, and sharing manual and semi-automatic (structure and unstructured) annotations, using open standards. Domeo shows you an article and lets you annotate fragments. You can attach a tag or an unstructured comment. The tag can be defined by the user or by a defined ontology. Domeo doesn’t care which ontologies you use, which means you could use it for annotating recipes as well as science articles.

Domeo also enables discussions; it has a threaded msg facility. You can also run text mining and entity recognition systems (Calais, etc.) that automatically annotates the work with those words, which helps with search, understanding, and curation. This too can be a social process. Domeo lets you keep the annotation private or share it with colleagues, groups, communities, or the Web. Also, Domeo can be extended. In one example, it produces information about experiments that can be put into a database where it can be searched and linked up with other experiments and articles. Another example: “hypothesis management” lets readers add metadata to pick out the assertions and the evidence. (It uses RDF) You can visualize the network of knowledge.

It supports open APIs for integrating with other systems., including into the Neuroscience Information Framework and Drupal. “Domeo is a platform.” It aims at supporting rich source, and will add the ability to follow authors and topics, etc., and enabling mashups.

Be the first to comment »

Neel Smith of Holy Cross is talking about the Homer Multitext project, a “long term project to represent the transmission of the Iliad in digital form.”

He shows the oldest extant ms of the Iliad, which includes 10th century notes. “The medieval scribes create a wonderful hypermedia” work.

“Scholarly annotation starts with citation.” He says we have a good standard: URNs, which can point to, for example, and ISBN number. His project uses URNs to refer to texts in a FRBR-like hierarchy [works at various levels of abstraction]. These are semantically rich and machine-actionable. You can google URN and get the object. You can put a URN into a URL for direct Web access. You can embed an image into a Web page via its URN [using a service, I believe].

An annotation is an association. In a scholarly notation, it’s associated with a citable entity. [He shows some great examples of the possibilities of cross linking and associating.]

The metadata is expressed as RDF triples. Within the Homer project, they’re inductively building up a schema of the complete graph [network of connections]. For end users, this means you can see everything associated with a particular URN. Building a facsimile browser, for example, becomes straightforward, mainly requiring the application of XSL and CSS to style it.

Another example: Mise en page: automated layout analysis. This in-progress project analyzes the layout of annotation info on the Homeric pages.

1 Comment »

Rob Sanderson [twitter:@azaroth42] of Los Alamos is talking about annotating Medieval manuscripts.

He says many Medieval manuscripts are being digitized. The Mellon Foundation is funding many such projects. But these have tended to reinvent the same tech, and have not been designed for interoperability with other projects. So the Digital Medieval Initiative was founded, with a long list of prestigious partners. They thought about what they’d like: distributed, linked data, interoperable, etc. For this they need a shared description format.

The traditional approach is annotate an image of a page. But it can be very difficult to know which images to annotate; he gives as an example a page that has fold-outs. “The naive assuption is that an image equals a page.” But there may be fragments, or only portions of the page have been digitized (e.g., the illuminations), etc. There may be multiple images on a page, revealed by multi-spectral imaging. There may be multiple orientations of the page, etc.

The solution? The canvas paradigm. A canvas is an empty space corresponding to the rectangle (or whatever) of the page. You allow rich resources to be associated with it, and allow users to comment. For this, they use Open Annotation. You can specify a choice of images. You can associate text with an area of the canvas. There are lots of different ways to visualize those comments: overlays, side-by-side, etc.

You can build hybrid pages. For example, and old scan might have a new color scan of its illustrations pointing at it. Or you could have a recorded performance of a piece of music pointing at the musical notation.

In summary, the SharedCanvas model uses open standards (HTML 5, Open Annotation, TEI, etc.) and can be implement distributed across reporsitories, encouraging engagement by domain experts.

Be the first to comment »

I’m at a workshop on annotation at Harvard. Philip Desenne is giving one of the keynotes.

We’re here to talk about the Web 3.0, Phil says — making the Web more fully semantic.

Phil says that we need to re-write the definition of annotation. We should be talking about hyper-nota: digital media-rich annotations. Annotations are important, he says. Try to imagine social networks with the ratings, stars, comments, etc. Annotations also spawn new scholarship.

The new dew digital annotation paradigm is the gateway to Web 3.0: connecting knowledge through a common semantic language. There are many annotation tools out there. “All are very good in their own media…But none of them share a common model to interoperate.” That’s what we’re going to work on today. “The Open Annotation Framework” is the new digital paradigm. But it’s not a simple model because it’s a complex framework. Phil shows a pyramid: Create / Search / Seek patterns / Analyze / Publish / Share. [Each of these has multiple terms and ideas that I didn’t have time to type out.]

Of course we need to abide by open standards. He points to W3C, Open Source and Creative Commons. And annotations need to include multimedia notes. We need to be able to see annotations relating to one another, building networks across the globe. [Knowledge networks FTW!] Hierarchies of meaning allow for richer connections. We can analyze text and other media and connect that metadata. We can look across regional and cultural patterns. We can publish, share and collaborate. All if we have a standard framework.

For this to happeb we beed a standardized referencing system for segments or fragments of a work. We also need to be able to export them into standard formats such as XML TEI.

Lots of work has been done on this: RDF Models and Ontologies, the Open Annotiation Community Group, the Open Annotation Model. “The Open Annotation Model is the common language.”

If we don’t adopt standards for annotation we’ll have disassociated, stagnant info. We’ll dereased innovation research, teaching, and learning knowledge. This is especially an issue when one thinks about MOOCs — a course with 150,000 students creating annotations.

Connective Collective Knowledge has existed for millennia he says. As far back as Aristarchus, marginalia had ymbols to allow pointing to different scrolls in the Library of Alexandria. Where are the connected collective knowledge systems today? Who is networking the commentaries on digital works? “Shouldn’t this be the mission of the 21st century library?”

Harvard has a portal for info about annotations: annotations.harvard.edu

3 Comments »

Cliff Lynch is giving talk this morning to the extended Harvard Library community on information stewardship. Cliff leads the Coalition for Networked Information, a project of the Association of Research Libraries and Educause, that is “concerned with the intelligent uses of information technology and networked information to enhance scholarship and intellectual life.” Cliff is helping the Harvard Library with the formulation of a set of information stewardship principles. Originally he was working with IT and the Harvard Library on principles, services, and initial projects related to digital information management. Given that his draft set of principles are broader than digital asset management, Cliff has been asked to address the larger community (says Mary Lee Kennedy).

Cliff begins by saying that the principles he’s drafted are for discussion; how they apply to any particular institution is always a policy issue, with resource implications, that needs to be discussed. He says he’ll walk us through these principles, beginning with some concepts that underpin them.

When it comes to information stewardship, “university community” should include grad students whose research materials the university supports and maintains. Undergrads, too, to some extent. The presence of a medical school here also extends and smudges the boundaries.

Cliff then raises the policy question of the relation of the alumni to the university. There are practical reasons to keep the alumni involved, but particularly for grads of the professional schools, access to materials can be crucial.

He says he uses “scholarly record” for human-created things that convey scholarly ideas across time and space: books, journals, audio, web sites, etc. “This is getting more complicated and more diverse as time goes on.” E.g., author’s software can be part of that record. And there is a growing set of data, experimental records, etc., that are becoming part of the scholarly record.

Research libraries need to be concerned about things that support scholarship but are not usually considered part of the historical record. E.g., newspapers, popular novels, movies. These give insight into the scholarly work. There are also datasets that are part of the evidentiary record, e.g., data about the Earth gathered from sensors. “It’s so hard to figure out when enough is enough.” But as more of it goes digital, it requires new strategies for acquisition, curation and access. “What are the analogs of historical newspapers for the 21st century?” he asks. They are likely to be databases from corporations that may merge and die and that have “variable and often haphazard policies about how they maintain those databases.” We need to be thinking about how to ensure that data’s continued availability.

Provision of access: Part of that is being able to discover things. This shouldn’t require knowing which Harvard-specific access mechanism to come to. “We need to take a broad view of access” so that things can be found through the “key discovery mechanisms of the day,” beyond the institution’s. (He namechecks the Digital Public Library of America.)

And access isn’t just for “the relatively low-bandwidth human reader.” [API’s, platforms and linked data, etc., I assume.]

Maintaining a record of the scholarly work that the community does is a core mission of the university. So, he says, in his report he’s used the vocabulary of obligation; that is for discussion.

The 5 principles

1. The scholarly output of the community should be captured, preserved, organized, and made accessible. This should include the evidence that underlies that output. E.g., the experimental data that underlies a paper should be preserved. This takes us beyond digital data to things like specimens and cell lines, and requires including museums and other partners. (Congress is beginning to delve into this, Cliff notes, especially with regard to preserving the evidence that enables experiments to be replicated.)

The university is not alone in addressing these needs.

2. A university has the obligation to provide its community with the best possible access to the overall scholarly record. This is something to be done in partnership with research libraries aaround the world. But Harvard has a “leadership role to play.”

Here we need to think about providing alumni with continued access to the scholarly record. We train students and then send them out into the world and cut off their access. “In many cases, they’re just out of luck. There seems to be something really wrong there.”

Beyond the scholarly record, there are issues about providing access to the cultural record and sources. No institution alone can do this. “There’s a rich set of partnerships” to be formed. It used to be easier to get that cultural record by buying it from book jobbers, DVD suppliers, etc. Now it’s data with differing license terms and subscription limitations. A lot out of it’s out on the public Web. “We’re all hoping that the Internet Archive will do a good job,” but most of our institutions of higher learning aren’t contributing to that effort. Some research libraries are creating interesting partnerships with faculty, collecting particular parts of the Web in support of particular research interests. “Those are signposts toward a future where the engagement to collect and preserve the cultural records scholar need is going to get much more complex” and require much more positive outreach by libraries, and much more discussion with the community (and the faculty in particular) about which elements are going to be important to preserve.

“Absolutely the desirable thing is share these collections broadly,” as broadly as possible.

3. “The time has come to recognize that good stewardship means creating digital records of physical objects” in order to preserve them and make them accessible. They should be stored away from the physical objects.

4. A lot goes on here in addition to faculty research. People come through putting on performances, talks, colloquia. “You need a strategy to preserve these and get them out there.”

“The stakes are getting much higher” when it comes to archives. The materials are not just papers and graphs. They include old computers and storage materials, “a microcosm of all of the horrible consumer recording technology of the 20th century,” e.g., 8mm film, Sony Betamax, etc.

We also need to think about what to archive of the classroom. We don’t have to capture every calculus discussion section, but you want to get enough to give a sense of what went on in the courses. The documentation of teaching and learning is undergoing a tremendous change. The new classroom tech and MOOCs are creating lots of data, much of it personally identifiable. “Most institutions have little or no policies around who gets to see it, how long they keep it, what sort of informed consent they need from students.” It’s important data and very sensitive data. Policy and stewardship discussions are need. There are also record management issues.

5. We know that scholarly communication is…being transformed (not as fast as some of us would like â?? online scientific journals often look like paper versions) by the affordances of digital technology. “Create an ongoing partnership with the community and with other institutions to extend and broaden the way scholarly communication happens. The institutional role is terribly important in this. We need to find the balances between innovation and sustainability.

Q&A

Q: Providing alumni with remote access is expensive. Harvard has about 100,000 living alumni, which includes people who spent one semester here. What sort of obligation does a university have to someone who, for example, spent a single semester here?

A: It’s something to be worked out. You can define alumnus as someone who has gotten a degree. You may ask for a co-payment. At some institutions, active members of the alumni association get some level of access. Also, grads of different schools may get access to different materials. Also, the most expensive items are typically those for which there are a commercial market. For example, professional grade resources for the financial industry probably won’t allow licensing to alumni because it would cannibalize their market. On the other hand, it’s probably not expensive to make JSTOR available to alumni.

Q: [robert darnton] Very helpful. We’re working on all 5 principles at Harvard. But there is a fundamental problem: we have to advance simultaneously on the digital and analog fronts. More printed books are published each year, and the output of the digital increases even faster. The pressures on our budget are enormous. What do you recommend as a strategy? And do you think Harvard has a special responsibility since our library is so much bigger, except for the Library of Congress? Smaller lilbraries can rely on Hathi etc. to acquire works.

A: “Those are really tough questions.” [audience laughs] It’s a large task but a finite one. Calculating how much money would take an institution how far “is a really good opportunity for fund raising.” Put in place measures that talk about the percentage of the collection that’s available, rather than a raw number of images. But, we are in a bad situation: continuing growth of traditional media (e.g., books), enormous expansion of digital resources. “My sense is…that for Harvard to be able to navigate this, it’s going to have to get more interdependent with other research libraries.” It’s ironic, because Harvard has been willing to shoulder enormous responsibility, and so has become a resource for other libraries. “It’s made life easier for a lot of the other research libraries” because they know Harvard will cover around the margins. “I’m afraid you may have to do that a little more for your scholars, and we are going to see more interdependence in the system. It’s unavoidable given the scope of the challenge.” “You need to be able to demonstrate that by becoming more interdependent, you’re getting more back than you’re giving up.” It’s a hard core problem, and “the institutional traditions make the challenge here unique.”

1 Comment »

I was steeling myself a couple of days ago to say something in a talk that believe but don’t want to: We shouldn’t feel guilty about relying on sources with whom we agree to contextualize breaking news. It’s ok. It’s even rational.

For example, if the Supreme Court hands down a ruling I don’t understand, or the FCC issues a policy that sounds like goobledygook to my ears, I turn to sites whose politics I basically agree with. On the one hand, I know that that’s wrong on echo chamber grounds: I’m getting reconfirmed in beliefs that I instead should be challenging. On the other hand, if I want to understand a new finding in evolutionary biology I’m not going to go to a creationist site, and if I want to understand the implications of a change in Obamacare, I’m not going to go to a Tea Party site. [Hint: I’m a liberal.] Oh, I might go afterwards to see what Those Folks are thinking, but to understand something, I’m going to go first to people with whom I basically agree.

Unfortunately, saying that in my talk meant I’d have to acknowledge that if I can to go to, say, DailyKos for primary contextualization, then it’s fine for right-wingers go to Fox News. Then I was going to have to explain how Fox and DailyKos are not truly equivalent, since Kos acknowledges facts that are unpleasant for their beliefs, and because Kos allows lots and lots of community participation. But that’s a distraction: If it’s ok for me to go to a lefty site to contextualize my news, it’s ok for you to go to your righty site. That feels wrong to me, and not only because I think right sites are wrong.

I finally realized that I’ m using the wrong sort of sites for my example. I do feel queasy about recommending that people get news interpreted for them by going to sites that operate in the broadcast mode. Fox News is like that. So are Slate and Salon, although to a lesser extent because they allow comments and because they present themselves as opinion sites, not news sites. Kos much less so because of the prominence of blogs and community. But I have no bad feelings whatsoever about taking my questions about the news to my social networks.

Because I’m old, much of social networking occurs on mailing lists. Some of the lists are based on topic, and contain people who broadly agree, but who disagree about most of the particulars; that’s what conversations are for. For example, a couple of the lists I’m on this morning are talking about what it would mean if Tom Wheeler [someone give that man a Wikipedia page!] were appointed as Chair of the FCC as seems increasingly likely. Tom comes out of the cable TV industry, which raises suspicions on my side of the swimming pool. So there has been an active set of discussions on my mailing lists among people who know much more than I do. The opinions range from he’s likely to be relatively centrist (although veering to the wrong side, where “wrong” is generally agreed upon by the list) to he’s never once stood up for users or for increasing competition and openness. Along the way, people have pointed out the occasional good point about him, although overall the tenor is negative and depressed.

Now, do I need to hear from the cable and telecoms industry about what a wonderful choice Tom would be? Sure, at some point. I even need to have my more fundamental views challenged. At some point. But not when I’m trying to find out about who this Tom Wheeler guy is. If we take understanding as a tool used for a purpose, it becomes a wildly inefficient tool — a hammer that’s all handle — if we have to go back to first principles in order to understand anything. Understanding is an efficient tool because it’s incremental: Given that I favor a wildly open Internet and given that I favor achieving this via vigorous competition, then what should I make of a Tom Wheeler FCC chairmanship? That’s my question this morning, not whether an wildly open Internet is a good thing and not whether the best way to achieve this is by increasing competition. Those are fine questions for another morning, but if I have to ask those questions every time I hear something about the FCC, then understanding has failed at its job.

So, I don’t feel bad about consulting my social network for help understanding the news.

And now, like the fine print in an offer that’s too good to be true, here are the caveats: My social networks may not be typical. Some types of news need more fundamental challenge than others. Reliance exclusively on social networks for news may put you into an impenetrable filter bubble. I acknowledge the risks, but given the situatedness of understanding, every act of interpretation is risky.

And yet there is something right in what I’m saying. I know this because going to “opposition” sites to understand the meaning of particular FCC appointments would require me to uncertainly translate out of their own unstated assumptions, and sites that try for objectivity don’t have the nuanced conversations enabled by shared, unstated assumptions. So, there is something right in what I’m saying, as well as risk and wrongness.

10 Comments »

Just a quick note updating my post yesterday about the musky Tesla-Times affair. [(‘m in an airport with just a few minutes before boarding.)

Times Man John Broder has posted his step-by-step rebuttal-explanation-apologia of Elon Musk’s data-driven accusations that Broder purposefully drove a Tesla S into a full stop. Looked at purely as a drama of argument, it just gets more and more fascinating. But it is of course not merely a drama or an example; reputations of people are at stake, and reputations determine careers and livelihoods.

Broder’s overall defense is that he was on the phone with Tesla support at most of the turning points, and followed instructions scrupulously. As a result, just about every dimension of this story is now in play and in question: Were the data accurate or did Broder misremember turning on cruise control? Were the initial conditions accounted for (e.g., different size wheels)? Were the calculations based on that data accurate, or are the Tesla algorithms off when the weather is cold? Does being a first-time driver count as a normal instance? Does being 100% reliant on the judgment of support technicians make a test optimal or atypical? Should Broder have relied on what the instruments in the car said or what Support told him? If a charging pump is in a service area but no one sees it, does it exist?

And then there’s the next level. We humans live with this sort of uncertainty — multi-certainty? — all the time. It’s mainly what we talk about when given a chance. For most of us, it’s idle chatter — you get to rail against the NY Times, I get to write about data and knowledge, and Tesla car owners get to pronounce in high dudgeon. Fun for all. But John Broder’s boss is going to have to decide how to respond. It’s quite likely that that decision is going to reflect the murky epistemology of the situation. Evidence will be weighed and announced to be probabilistic. Policy guidelines will be consulted. Ultimately the decision is likely to be pegged to a single point of policy, phrased as something like, “In order to maintain the NYT’s reputation against even unlikely accusations, we have decided to …” or “Because our reviewer followed every instruction given him by Tesla…” Or some such; I’m not trying to predict the actual decision, but only that it will prioritize one principle from among dozens of possibilities.

Thus, as is usually the case, the decision will force a false sense of closure. It will pick one principle, and over time, the decision will push an even grosser simplification, for people will remember which way the bit flipped — fired, suspended, backed fully, whatever — but not the principle, not the doubt, not the unredeemable uncertainty. This case will become yet one more example of something simple &mdash masking the fathomless complexity revealed even by a single review of a car.

That complexity is now permanently captured in the web of blue underlined text. We can always revisit it. But, we won’t, because the matter was decided, and decisions betray complexity.

[Damn. Wish I had time to re-read this before posting! Forgive typos, thinkos, etc.?]

2 Comments »