Is virtual reference successful? Part II (Hint: yes it is)

In Part II I'd like to respond specifically to two comments. The first is from Morgan Fielman. The second from Pascal Lupien, author the article I'm discussing.

RESPONDING TO FIELMAN

Commenter Morgan Fielman wrote, "The original poster seems to have missed the point of this article, which is primarily about software."

No, I get that the point of the article is primarily about the software and not the customer experience. But the article is so broadly written and lacking in detail that it ends up saying nothing more specific than "VR software has problems."

My questions, unanswered by the article are, What software? What problems? Two of the products Lupien writes about (Tutor.com’s and QuestionPoints') recently underwent complete overhauls, in effect becoming completely new products. It is unclear from Lupien's article which versions he's writing about but my sense is he's writing about the older versions. If that's correct then most of the article is, at best, moot.

Granted, the larger issue of whether or not the software is effective is a valid issue that warrants exploration and discussion. Fielman goes on to ask, "but how can customers be satisfied when the software we use is so poor?" I say that's the wrong question. The question is "Are customers satisfied?" The answer in our customers’ experience is yes, they are satisfied. We didn’t find this out by polling 20 libraries. We found this out by asking the customers. Another good question might be, "Do the problems with VR software affect the quality of the customer experience, and if so how and to what extent." There are many people at collaborative VR services looking at a lot of data to answer that question. Lupien’s article suggests that problems with software affect the customer experience but offers no actual data to back it up. He mentions problems with popup windows, problems with Windows service pack 2, and problems with serving customers who use Macs, but he is not specific about which software products exhibit which problems and to what extent. And again, Lupien is not clear which version of Tutor and QuestionPoint he’s talking about. The newer versions of both products are compatible with Mac users, and have no problems with service pack 2 issues that I’m aware of.

Fielman concludes his comment by saying "original VR supporters have realized that this service just isn't cutting it." The fact is our service has been cutting it for almost 5 years, and we have the hard data and glowing customer comments to prove it. If your VR service isn't cutting it, you need to ask why. Are your staff trained on the software? Are they enthusiastic? What are you customer service standards? Do your librarians give kick-ass customer service in f2f encounters? What quality control mechanisms do you have in place? Do you examine your transcripts for quality? Do you have regular and convenient service hours? Are you available 24/7? (going 24/7 made a huge difference in our usage, even though usage mostly grew during hours we were already open -- go figure…) And finally, but certainly not last, do you consistently and effectively market your service to your customers? Do they know you exist???

If your service ain't cutting it maybe you need to answer these questions first before blaming the software, which is an easy way out. Consider that here in New Jersey using standard VR software (currently QP, formerly Tutor/LSSI’s eGain-based software) we’re cutting it and then some. Other statewide collaboratives are doing quite well too. And we're all working very diligently with our respective vendors to ensure that our VR platforms are stable and highly functional. While the current glitch here and there can be a real and undeniable pain in the ass, it hasn't prevented us from delivering a high quality and slightly mind-blowing experience to our customers.

RESPONDING TO LUPIEN

First, I’d like to thank Pascal Lupien for taking the time to offer an extremely well-written and thoughtful comment in response to my first post. I’d like to assure him that contrary to his assertion, I’ve read his article through thoroughly a few times. I have no problem with bad news about VR. I just want accurate and somewhat substantiated news. I’m offering up the reality of my experience at QandANJ to counter the broad statements that Lupien makes. Now to some of his specific comments.

He writes, Perhaps these results aren’t what proponents of VR would prefer to hear, but they do represent a problem that needs to be discussed, for the sake of our users.

I do not consider myself a proponent of VR, I consider myself of proponent of libraries. It is my desire that libraries remain relevant to our customers by offering a suite of high quality services. Collaborative VR is one such service, offering our customers 24/7 access where and when they want it. I want to see libraries changing their customers’ perceptions about what libraries can offer them. I want libraries to blow customer expectations out of the water. I want libraries to be around in 50 years. It is not that I don’t want to hear bad news about VR software. I’m perfectly open to hearing about the problems with the current stable of VR software offerings. It’s just that I want to hear facts, not conjecture. And I want those facts to be couched in some meaningful context and always tied back, to whatever extent possible, to the impact on our customers. I didn’t get this from Lupien’s article.

Lupien writes, To respond to the person who claimed that software is the last thing that matters about VR, I say tell that to the user who is unable to log in because she uses a Mac, or because her computer has pop-up blockers. Tell that to the user who is "kicked off" in the middle of a session because the VR software does not function properly with the library’s licensed databases. These things happen regularly, and this article makes an attempt to discuss them.

I’m pleased to see Lupien talking directly about the impact on customers. Clearly we agree that it would be optimal if VR software worked across all platforms, had no problems with pop-up blockers, and worked 100% of the time so no user was ever "kicked off." I am not suggesting that these problems don't exist, I am asking to what extent do they exist, and to what extent do they impact the customer's experience and satisfaction with VR service. Because Lupien fails to identify what versions of the various VR products he tested, and is repeatedly non-specific regarding his data, the article fails to answer these questions.

Lupien grants that, "many regular VR users appreciate the service," and that he wasn’t contesting that fact. Our experience suggests that it is not "many" but most.

Lupien writes, "Shouldn’t we be thinking about these potential users as well, rather than focusing on those who already use and appreciate the service? Shouldn’t we be trying to determine if one software product could help us to improve the experience for all users, not merely the satisfied ones? Perhaps some would fear doing this, as it would reveal that their VR service isn’t as successful and user-friendly as they like to claim?

Yes, we should absolutely be thinking about our potential users, and we should always be shooting for a platform that will provide high quality service to everyone. Again, it's a matter of facts and context. Lupien's article disappoints me on both counts.

Lupien writes, "The point of this article is to focus on users who are unable to log in to begin with, who encounter technical problems during a transaction, or who choose not to use the service because they would be required to disable pop-up blockers or use a particular browser, etc. We’ll never know how these users feel about VR, because they don’t get far enough into a VR transaction to make…comments.

Actually, we have some way of knowing. We ask. Yes sir, right there on the front page of QandANJ we say, "Click here to give us feedback on how our new software is working for you." Here's a sample of what we find: Since May 1st (79 days), we have received 23 comments. 16 of them were specifically technical (some were positive, some were of the nature, "it wasn't fast enough".) One comment came from a Mac user, 3 came from customers accessing us through the AOL interface and browser. So Mr. Lupien, we do make an effort to compile and monitor such information, looking for problematic trends with an eye on improving the service.

Finally, Lupien suggests that I have not been keeping up with the VR literature and if I had "taken the time to consider some of the issues discussed in this article before jumping on that user-centric high horse" I would have "come away with a better understanding of what is happening beyond QandANJ."

I can assure Mr. Lupien that I keep up quite well with VR literature thank you, and I'm familiar with Coffman and Arret's article, which you can read here (right at the bottom of the page, after Brenda Bailey-Hainer's reasoned response.) And if speaking from a place of fact and experience instead of conjecture and generality puts me on a high horse then what can I say? Giddyup.

In Part III (much shorter, I promise) I'll address the VR software versus IM question.

Epilogue: Customer comment from today: "I am exceedingly impressed. First time in ages I felt like I was getting something positive for my tax dollars." (Our funders sure hate to see this... Ha Ha )

Tags: VR, Virtual Reference, Libraries, Reference