Is virtual reference successful? Part I (Hint: yes it is)
Pascal Lupien begins his recent article on virtual reference (Virtual Reference in the Age of Pop-Up Blockers, Firewalls, and Service Pack 2 , By: Lupien, Pascal, Online, Jul/Aug2006, Vol. 30, Issue 4) "by declaring that, "the evidence indicates that libraries are not satisfied with the service." Say what? Aside from the fact that the statement is so overly broad as to be false on the face of it (which libraries? which services?), it's not about whether the libraries are satisfied with the service, IT'S ABOUT WHETHER THE CUSTOMERS ARE SATISFIED WITH THE SERVICE.The fact that Lupien goes on for nearly 3500 words with nary a mention of customer satisfaction epitomizes to me the worst of librarian-centric thinking at the expense of customer experience. 3500 words with:
- No mention of how VR customers love and rave about the convenience of the service.
- No mention of how VR customers love and rave about having a live person available to assist them with their information needs.
- No mention of how VR has changed our customers' perceptions of what libraries can offer them.
- No mention of how VR has helped make libraries more relevant to our customers by meeting their needs and exceeding their expectations.
I am feeling weary after reading Lupien's article. Weary because there is so much wrong with it that it almost demands a line-by-line critique in the spirit of Twain on Fenimore Cooper. Well Lupien isn't Fenimore Cooper and I'm certainly not Twain, and besides I'm really, really tired.
So let me address a few errors, raise a few eyebrows (two, to be precise) and share some of my own experience - uh, make that our customers’ experience - with VR via QandANJ.
A moment to share my creds: I've been involved with QandANJ since it's inception in 2001 (before that, actually,) helping to build, manage and promote the service. I've looked at thousands of transcripts and thousands of customer feedback forms. I know that our usage is through the roof. We handle as many "calls" as we can limited only by our ability to offer deeper staffing. I know that our customers tend to be very satisfied, and I know WHY our customers tend to be very satisfied. If you want to delve deeper into our stats and findings, take a look at this presentation from the VRD Conference in 2003. (there's more here) The numbers may be a little dated, but the story they tell and the trends they point to remain just as true today.
I'm not making this stuff up... Here's one of my favorite comments:
If you think this is cherry picking, it ain't. We get our share of negative comments too (usually younger users, usually wanting "faster, faster, faster" service.) The reality is our customers are happy. Why? Here's what they tell us:
We have hundreds of pages of single-spaced pages with thousands of comments that go on and on in these veins. There are many other successful collaborative VR projects like those in Maryland, Colorado, and Cleveland that could show you similar comments from their satisfied customers. The challenge isn't attracting the customers, it's managing to grow the staffing of the service to keep pace with the demand!
In part 2, I'll get a bit more nit-picky with other elements of Lupien's article.
Tags:
Labels: Peter
posted by Peter Bromberg @ 10:25 PM
7 comments
7 Comments:
As a librarian who has logged countless hours and completed (literally) 4,000 or more transactions on VR (both at MPOW and as freelancer for QandANJ) let me say that I love doing reference work this way. Yes the software glitches can be a big pain for all, but I find the range of questions that I receive on VR to be broader and more interesting than those that I receive in person at the reference desk. I enjoy chatting with the customers, being able to get back to them with email if need be, and find it rewarding to be able to answer questions late at night when libraries are closed.
Thanks for debunking this article Pete. I read it with raised eyebrows as well and was pondering posting about it myself. Glad you beat me to it as you are much better qualified than I to take this on. Go to it, Pete!
Hear hear.
I think it's fair to say the article is about virtual reference software, and in truth, it's a good summary of how library vr products fit into the bigger picture of consumer software.
That is, consumer software is complicated. What I kept looking for from Lupien's article is a sense of how that complicated picture is changing, and what we should expect from our virtual reference software going forward.
Saddest of all is that publications should focus on software, when this is the last thing that matters about virtual reference. QandANJ has proved that over and over and we are glad to follow your lead.
I was thinking some more about this on my drive to work... just want to add an observation (or perhaps it is a curiosity)that I have made about VR.
The software is difficult and the interface cumberson even with recent improvements and upgrades. YET we still have lots and lots of VR customers who choose to log on to ask their questions when in reality it can often be answered by a relatively simplistic search on any freely avaible search engine.
At first this puzzled me, but I have come to the conclusion that they choose VR over a search engine because either they want assurance that they have found the best/right information or even if they know how to search they do not know how to select the correct result from the list to get the information they need.
Knowing how to plug a few keywords in to a search engine or database does not mean that a person is information literate or able to interpret the results of the search. That, I think, is one of the key services provided by VR (and indeed most reference transactions no matter what method they may occur).
The original poster seems to have missed the point of this article, which is primarily about software. It is intended to explore the problems which libraries have faced with VR software. At my library, we have dealt with many of the problems mentioned in this article, and I know from colleagues that these are not uncommon. In fact, many libraries have dropped VR software due to the technical problems (and we will soon do the same). If libraries are not satisfied with VR software, it is because they know that their patrons are not being well served. It is important for libraries to understand the problems which patrons may be facing when attempting to log into a VR service. While vendors have responded to some of the problems, many still exist, which is why I think this article is useful and timely. In fact, I’m surprised it hasn’t been written about before. Had we been aware of how unsatisfactory the VR experience would be for our patrons, we would have never embarked on the project in the first place. Yes, its about whether customers (not libraries) are satisfied, but how can customers be satisfied when the software we use is so poor? Our concern with what out patrons want is leading us to explore IM reference. Even the original VR supporters have realized that this service just isn’t cutting it.
Peter has indeed missed the point of the article, to the extent that it appears that he has only skimmed the piece and zeroed in on a few phrases, quoted out of context. To suggest that it is librarian-centric is astonishing considering that the entire piece is focused on user satisfaction and on helping libraries to improve the user experience. The idea was to examine the widely documented problems with VR software and how these problems may be having a negative impact on usage. By understanding the problems many of our users may be having when they attempt to use VR service, libraries will be able to better address the needs of these users. The original intention was to look at the various VR packages on the market and to determine which software would allow libraries to offer the best experience for their users. Unfortunately, by testing several software products and contacting libraries who use these products, it became clear that no VR software stood head and shoulders above the others. The testing indicated that technical problems, which in some cases prevented users from even accessing the service in the first place, were common to all software products. Similarly, interviews with libraries across North America suggested that many encountered problems such as crashing during a session, loosing patrons, etc. Perhaps these results aren’t what proponents of VR would prefer to hear, but they do represent a problem that needs to be discussed, for the sake of our users. To respond to the person who claimed that software is the last thing that matters about VR, I say tell that to the user who is unable to log in because she uses a Mac, or because her computer has pop-up blockers. Tell that to the user who is “kicked off” in the middle of a session because the VR software does not function properly with the library’s licensed databases. These things happen regularly, and this article makes an attempt to discuss them.
The article does not state that no users are satisfied with VR, nor does it suggest that VR transactions are always plagued with technical difficulties. That’s not the point of the article, and others have written about user satisfaction (or lack thereof). Yes, many regular VR users appreciate the service, that isn’t being contested. The point of this article is to focus on users who are unable to log in to begin with, who encounter technical problems during a transaction, or who choose not to use the service because they would be required to disable pop-up blockers or use a particular browser, etc. We’ll never know how these users feel about VR, because they don’t get far enough into a VR transaction to make the kinds of positive comments that Peter has posted screen shots of. Shouldn’t we be thinking about these potential users as well, rather than focusing on those who already use and appreciate the service? Shouldn’t we be trying to determine if one software product could help us to improve the experience for all users, not merely the satisfied ones? Perhaps some would fear doing this, as it would reveal that their VR service isn’t as successful and user-friendly as they like to claim?
Incidentally, this is not the only article to report on user dissatisfaction. A few articles are cited in this piece, but there are many others out there. Many who have suggested that VR isn’t all it was originally thought to be (i.e. Coffman and Arret) have been sharply criticized by VR proponents (who seem to be a very sensitive bunch). I would suggest that Peter take a look at the literature on VR to develop a better understanding of what is going on beyond QandANJ. I realize that QandANJ is one of the successful VR services, and those who manage the service should be commended. Unfortunately, by dismissing anyone who dares to criticize VR as a service, we are in fact ignoring many users. Posting a few screen shots of positive user feedback into a blog proves nothing more than the fact that some users (those who can get in) appreciate the service. That’s great. However, as stated before, the idea that some users like VR was never contested. Unfortunately, many libraries and their users are not having such a great experience with VR, and many libraries are in fact closing down their VR service or moving towards more user-friendly IM software.
Perhaps if Peter had taken the time to consider some of the issues discussed in this article before jumping on that user-centric high horse, he would have come away with a better understanding of what is happening beyond QandANJ.
Maybe I'm missing something but why can't we just use common IM'ing for virtual reference instead of this software? I believe Michael Stephens said that at his (former?) library they had invested in the software and then became so disappointed in it they dropped it and juse use "regular" IM! Much more simple, and plus, "free"!
Sorry Pascal, I really don't think that software matters. The sooner we realize this, the better.
I am all for everyone having access to libraries and all of our services. The reason that software doesn't matter is because our services are choked with traffic as it is. If you can get any population to use it's library virtual reference service one tenth as much as it uses telephone or in-person reference, you'll see the same thing. We can hardly handle the traffic from people who do meet the software requirements.
Needing a scalable model is a much bigger problem than software.
If software mattered, then why would so many people be touting commercial IM products, when only a third of people going online use them? Don't more people use web browsers than use IM? What about all the people who don't IM, how will we ever hear from them?
We need to give up the idea that "virtual reference" software and "instant messaging" software are different. VR is just a cheap knock-off of IM, with some extra features that could probably stand to be abandoned.
So yes, there are technical problems, but the right way to address that is to start with ADA compliance. I can't believe we're even talking about Service Pack 2.
I was dissapointed with your article because it took a broad swipe, which as you said, needed to be done, but left out the part where you paint a better picture of what things should look like.
Again, software doesn't matter. What matters is people, online, talking to other people. Libraries should be able to make that happen without squabbling over pop-up blockers, firewalls and service pack 2.
Post a Comment
<< Home