Saturday, October 27, 2007

"I Was Kind of Confused b4" - Audio of Radford on Chat Reference at Oregon VR Summit '07


On June 7, 2007 I blogged about a keynote talk I gave on June 1, 2007 at the Oregon Virtual Reference Summit 2007 organized by Caleb Tucker-Raymond, Oregon Statewide Digital Reference Services Coordinator for the L-net: Oregon Libraries Network consortium. The talk just became available as an audio file on the open web. (Thanks Caleb!) I promised to post to the blog when this happened, so am now able to make good on my promise.

If you'd like to listen to this presentation, click here: “I Was Kind of Confused b4” Interpersonal Communication Research in Virtual Reference.”

The talk focuses on the information-seeking and communication behaviors of the youngest Millennials - the Screenagers. I discuss their predilections and characteristics (multi-tasking, impatience, practicality, convenience, etc.) as well as their perceptions of librarians ("I don't trust librarians, I trust Google") and fear of cyber-predators in chat rooms that extends to chat librarians ("I don't like to chat with strangers.")

In addition, I comment on some recommendations for improving chat reference encounters with teens . These recommendations were derived from focus groups with screenagers and from in-depth chat reference transcript analysis as part of the IMLS grant project Seeking Synchronicity.

The keynote was about 50 minutes, followed by Q and A, so be forewarned that it is long. Hey, feel free (of course!) to check your e-mail while listening, or to multi-task with other activities ;)

I begin by talking about my background and how I got interested in studying chat reference, so if you want to get to the research results, fast forward through the first 15 mins. or so.

Labels: , , , , ,

Sunday, October 07, 2007

'Predatory Reference' an Interview with Bill Pardue about 'Slam the Boards.' Second Slam Coming Up on October 10, 2007!



Bill Pardue is the Virtual Services Librarian at the Arlington Heights (IL) Memorial Library. He worked previously at the Illinois Institute of Technology and received his MSLIS from the University of Illinois in 1992. Bill is also involved with the AskAway Illinois Advisory Committee and manages the website for the statewide VR service.

Bill initiated “Slam the Boards” by inviting librarians “to be bold and invade online answer sites such as Yahoo! Answers, Amazon’s Askville, and the Wikipedia Reference Desk” and to market libraries by “making it clear that this question was answered by a librarian/library professional/etc.”

Here's my recent interview with Bill.

Marie: Bill, thanks so much for visiting Library Garden today. To get us started, tell me about “Slam the Boards” and especially how the idea occurred to you.

Bill: It was a very social process. I started playing around with Yahoo! Answers on my own and realized that it might be an opportunity for librarians to interact with users who don’t even realize that libraries have reference services. Paula Moore, our Coordinator for Public Services at Arlington Heights, commented that we ought to encourage lots of librarians to do the same. At the Collaborative Virtual Reference Symposium in Denver this past July, I mentioned it to Caleb Tucker-Raymond of the Multnomah County Library. He immediately said that instead of having some vague effort to get librarians more involved, a single day should be picked and promoted, in order to provide a real focal point. It was exactly the thought I needed to take action. Within a week I’d set up a Slam the Boards wiki and started putting the word out on listservs and anywhere else I could leave a comment. Then the viral part took over. In just over a month we had participants listed from the US, Europe, even New Zealand. It just seemed like the right idea at the right time…I just set up the wiki!

Marie: Caleb has such great ideas, I visited him on June 1st at the Oregon Virtual Reference Summit.

What were you hoping to achieve with “Slam the Boards”?

Bill: Mostly awareness on both sides of the question/answer transaction. Awareness among librarians that there’s a large potential patron base that we’re missing and need to promote to, as well as an arena in which we can showcase our excellence. On the asker/patron side, I’d just like a few answer board users to be pleasantly surprised that librarians don’t only provide people with books and videos, but also provide reference service. What I certainly didn’t hope to achieve was a cessation of people using answer boards. It just won’t happen, and people get some very good answers there. But I want librarians to realize that answer boards aren’t “the enemy.”

Marie: I know that one interest you have is in evaluation of the event, and, as a researcher, I’m especially interested in looking at reference quality issues, but would you deem it to have been a success? Why?

Bill: At this point, I’m gaging success in terms of engagement. Of course, it’s great to have a reply chosen as “best answer” now and then, too! The main point, though, is that we were out there, we saw what kinds of questions people ask and we hopefully provided useful, sourced answers. Some folks have started archiving answer board responses in a special QuestionPoint account that will allow for analysis by anyone who’d care to look at them. Currently it’s at about 75 questions (too many of them mine!), but I’m hoping that number increases. Quality’s an interesting issue. In a voting environment like Yahoo! Answers, I ended up feeling extra pressure to give a really good, sourced answer. It even stung a little when someone else’s off-the-cuff reply (which may have said the exact same thing) was voted best. I’d be interested to see how a more thorough study of quality on answer boards is conducted and what kind of results come out of it. You’ve got your work cut out for you!

Marie: So, did Slam the Boards achieve what you had imagined?

Bill: I think it did, partly because it had such a simple goal…get librarians involved, get them to think beyond their library confines and get engaged in some “predatory reference.” We’re still just a drop in the bucket in terms of the total traffic on a site like Yahoo! Answers, so I have no illusions about having a measurable impact on library reference numbers or VR service statistics.

Marie: I’ve heard you talk about “predatory reference” before, and like this radical concept! Would you mind defining it for us at Library Garden?

Bill: I'll be the first to admit that it's a somewhat over-dramatic coinage for a fairly straightforward concept. Librarians need to start actively finding reference questions, rather than just waiting for them to come in. Don't limit your presence to just the reference desk or the library's IM or VR service. Instead, find out where the where the questions are and start providing answers unsolicited. Being a “virtual services librarian” I tend to think first of online options: looking for points of fact in local discussion forums, blogs, etc. Out of such activity at Arlington Heights, we've even worked out partnerships with two local discussion board that take questions from the community. One is the “What's the Fact” column of the Daily Herald's Beep Central site. The other is the “Ask an Arlington Heights Librarian” forum.

There are less virtual ways, to do this, too. One local library (and I apologize that I can't remember which) has been having reference librarians participate as judges for a local bar's weekly trivia night. The Arlington Heights Memorial Library regularly sends our librarians out to community events (festivals, senior center events, etc.) with a wireless connection that allows us to provide many of the same services that we would at the reference desk. The point is to start being a little...dare I say...pushy about showing off our skills, so that potential users will realize that libraries equal more than just books! I'm sure we could think of other ways to get involved. Show up at village council meetings and if a tough topic comes up, volunteer the library's reference service to help find some background. When you're with a group of people, listen for points at which you can mention/promote reference services. If you overhear a local business person talking about doing mailing lists, let them know that the library has tools like ReferenceUSA that can be of use (and that someone on your staff is willing to demonstrate it to them). The opportunities are out there, we just have to be looking for them.

Marie: Do you have any idea about the number of librarians who participated and/or number of questions answered, even if it is a rough guess?

Bill: Ultimately, it’s a tough call. My intuition is in the hundreds of librarians, with maybe a thousand questions…but I have absolutely no way of knowing. Some of the more enthusiastic participants put their names on the wiki.

I counted 98 names there on 10/5/07. If you figure that 2-3 times that many actually participated, and the average “load” was 5 questions (I picked up 25 myself, and I know several others had matched that number), I’d say that 1000 questions isn’t unrealistic.

Marie: This question is from Beth Cackowski of QandANJ “Were the majority of questions answered by librarians, research questions? In other words, were they questions that library customers might expect a librarian to answer, or were they questions that the general public might be surprised to see answered by a librarian, for example: automotive, sports, pop culture, medical, legal?”

Bill: The unfortunate part is that most users don’t have any expectation of what kind of questions a librarian might answer (beyond “do you have a book on…”). To keep things mixed up for myself, I bounced around from category to category, picking up homework help questions, business, arts & humanities, cooking & recipes, geography, etc. I expect others did the same. If you check the list of participants above, you can see that many have added links to their Yahoo! Answer lists, so you can check out how they moved through the categories.

Marie: I definitely agree that many people don’t have a clue as to what types of questions a librarian could answer. Our abilities are usually underestimated.

Here’s a question from Julie Strange of Maryland AskUsNow! “Do you have a sense of how librarians went to find questions? Did they sort through the subjects and go for ones they specialized in? Or did they take new questions as they came in?”

Bill: Cherry-picking is essential on the boards because so many questions aren’t really informational. “What’s your favorite shampoo?” “I really like this girl, but I’m afraid to ask her out. What should I do?” etc. So, after a little digging around, you see that certain categories in any board have a higher ratio of informational vs. social questions and you start to “hang out” there. It’s kind of like “working the room” until you find someone you want to talk to at a party! As far as specializing in a subject, I think that’s very much up to the individual librarian. I consciously tried to be a generalist, but I also picked up a couple of questions in the Science/Astronomy category because that’s my hobby.

Marie: That’s really interesting, I like your “working the room” comparison. Have you gotten any feedback from librarians about their experiences?

Bill: Most of the feedback has been very positive. A lot of librarians were a little taken by the social nature of a lot of the questions, but ultimately were able to find at least a few to answer. Finding out if you received a “best answer” can take several days, so there were numerous messages from librarians when they got word of their “wins.” I got some negative feedback beforehand from some folks who couldn’t see the point of it, but nothing from anyone who actually participated. Of course, there could be all kinds of biases that account for this!

Marie: Did the librarians get much feedback from the users of these services?

Bill: The user feedback is pretty much determined by the mechanisms in place by the answer board. After one question was chosen “best answer,” there was a “nice answer” comment from the user. I’ve received a few like that now. If you give a particularly good answer, you can get “star” ratings, up to five stars. Of course, there’s the voting, too. It can be done by the asker or other readers. It’s nice to see your count of “thumbs up,” but you get a “thumbs down” every now and then. You’ve got to have a thick skin! An interesting anecdote is that I actually had a fairly extensive post-question correspondence with a user who had a tough corporate question. I actually ended up making several phone calls, just as I would have done for my own library’s patron.

Marie: Sounds like you could have parlayed that corporate interplay into some consulting business if you wanted to be more entrepreneurial ;)

Finally, I see that you are encouraging librarians to repeat “Slam the Boards” for October 10th, tell me about your vision to keep it going.

Bill: I’m really hoping this takes on a life of its own. The success of something like this is that it ultimately shouldn’t need a specific set of individuals to keep it going. I’d like to know that there’s a spike in answer board activity each month on the 10th, as well as a baseline through the rest of the month. I’d like to see discussion of this initiative on the existing listservs (it’s a bit too insular to just have its own listserv, I think) and informal discussion groups at conferences. I’d love to hear about a dine-around at Internet Librarian this year! Unfortunately, I can’t make it myself, but that’s all the more reason for others to do this. The best thing is that this is a way to promote library reference service that costs very little money and has the potential over the long run to enhance our image with a user base that almost never thinks about us.

Marie: Nicely put Bill. Thanks so much for your candid answers! Good luck with this month’s “Slam the Boards” on October 10th. I’ll be away at the Library Research Seminar IV in London, Ontario, from Oct. 9-11th, but will see if I can find a wifi hotspot and join in some predatory reference!

Labels: , , ,

Friday, July 13, 2007

Are We Getting Warmer? Query Clarification in Virtual Reference (ALA Redux Part 2)


At the Library Research Round Table at ALA in Washington, DC, on June 23, 2007, Lynn Silipigni Connaway of OCLC and I presented our latest findings on Query Clarification in VR (aka question negotiation or the reference interview).

We examined 600 chat transcripts randomly selected from QuestionPoint bank of almost 500,000 transcripts. Here are some of our findings (and an invitation below to the Seeking Synchronicity web site to see the PowerPoint slides and handouts from this presentation).

Do librarians clarify?
75% (in 434 of 581 usable transcripts) librarians did ask clarifying questions.

Did they ask the highly recommended follow-up question? (some version of “Does this completely answer your question?”)
50% (217) of the 434 librarians who clarified did ask the follow up question.

What types of questions were asked?
66% (554 of 838 questions asked by the librarians) were closed questions.
34% (282 of 838) were open.

What did librarians ask about?
Librarians asked users questions about: topic, background, search history, type of resource needed, extent/depth of information needed, if the user wanted a referral and more.

How about the virtual reference users?
Users offered information about: topic, background, extent/depth, and to correct the librarian’s misunderstanding.

Surprising finding! 2 different patterns of clarification!
Librarians clarified more often in the beginning of the interaction
Users clarified in the middle more often.

Most important finding! How to improve accuracy in chat reference?
For the 180 ready reference questions in our sample, we looked at accuracy (see my blog posting of July 10, 2007 for more on ready reference in chat).
Clarifying the query and asking the recommended follow-up question both boosted accuracy.

Bottom Line
Always ask clarifying questions, even if you think you understand the question (one user asked for diving instructions, but had made a typo and wanted driving instructions, early clarification would have saved the librarian much searching time!)

Always ask a version of the recommended follow-up question: “Does this completely answer your question?”

Interested in more detail on the above findings? Please click on the above links to see the PowerPoint slides and handouts.
This research is funded by IMLS, Rutgers University, and OCLC.
Special thanks to Susanna Sabolcsi-Boros of Rutgers, SCILS, and Patrick Confer of OCLC, for their help with this part of the project.

Labels: , , ,

Thursday, June 07, 2007

News from Oregon Virtual Reference Summit 2007 – QandA and Ready Reference from Texting Google Mobile SMS (Beta)




I just recently returned from giving a keynote address on June 1, at the Oregon Virtual Reference Summit 2007. Caleb Tucker-Raymond, Oregon Statewide Digital Reference Services Coordinator, organized this wonderful conference that drew participants from Oregon, Washington, and California, but was mainly designed to bring together librarians who participate in L-net: Oregon Libraries Network. My plenary was called “I Was Kind of Confused b4” Interpersonal Communication Research in Virtual Reference” and I gave a workshop on “Exploring Encounters with Chat Users: Analyzing VR Transcripts.” I am willing to share ppt and handouts to LG readers if you send me an e-mail request. The plenary was videotaped and may be on the open web at some point, I will blog about it if/when this happens.

While at the conference, I attended a fascinating panel on: “What Students Need, What Schools Need.” This program brought together the viewpoints of middle and high school librarians, public librarians, and a delightful young junior high student who spoke about VR from the student perspective. After the panel, I congratulated her on her presentation, poise, and enthusiasm for VR. She mentioned that she had heard me speak in the plenary, but at first had not know what the word “plenary” meant, so she had sent a text to Google. I said: “You did WHAT???” She said: “I sent a text to Google (466453) and I put in define plenary and it gave me the definition.” I had her show me and saw that it also returns the URL where the answer was found on the web.

Some of you may already know about (or use!) this service (which is in Beta testing), but it was news to me! (It was also news to my 16 yr. old daughter, the text maven in our house, which helped me to decide to blog about this). Later one of the helpful L-net participants printed out the Google Mobile info page and I found out that not only are word definitions possible, but also you get info on weather, flight updates, movies, translations, currency conversions, driving directions, QandA and more. Google’s example for using their QandA is: abraham lincoln birthday. If any of you have tried this service out please leave a comment telling me what you asked and how it went!

Here we see Google testing the waters, as some libraries are doing, with text reference services. The latest start up I have heard of SMS for libraries was in Australia as reported on the dig_ref listserv this week by Colin J. Bain, Library Services Manager of Queensland University of Technology, Brisbane. Their SMS service just started this past Monday (June 4th) and Colin told me that they have only had 2 queries so far about library opening hours. Since they haven’t done any publicity yet, traffic will surely pick up.

Hmmm, now I am definitely going to have to spring for unlimited text messaging on my cell phone.

Labels: , , , , ,