Wikipedia:Help Project/Usability Testing 1

A series of usability tests were carried out by the wub in the Summer of 2012, as part of the help pages community fellowship. These are the results and observations from those tests.

Participants

Name Age Gender Wikipedia reading Wikipedia editing Occupation Other notes
J 25 M Every day Couple of typo fixes Student
E 26 F Every day None Student Also uses Hungarian Wikipedia occasionally. Thought about editing in the past, put off by code.
H 23 F Few times a week None Teaching assistant
A 52 M Few times a week None Engineer
M 48 F Few times a week None Teacher


Introductory script

Background questions

  • Before we start the tests proper, I just want to find out a bit more about your past experience with Wikipedia.
  • How often do you visit Wikipedia?
  • Do you ever use Wikipedia in any languages other than English?
  • You said you come to Wikipedia XXX often. What are you usually looking up/doing? How has your experience been in the past?
  • Have you ever edited Wikipedia at all?
  • (Non-editors) Are there any particular reasons that you haven’t edited in the past? Can you tell me what you know now about the process of editing?
  • (Editors) You mentioned that you have contributed to Wikipedia before. Please tell me about those experiences.

Think aloud and starting

  • Next I’d like to have you attempt a few tasks on Wikipedia.
  • There’s one thing that I would like you to do differently. Please think aloud as you carry out these tasks.
  • For example, if you are reading something read it out loud and feel free to say anything that comes to your mind as you read - for example "that’s interesting" or "what does that mean?" Also, talk out loud while you’re doing a task, so I can understand what you are thinking and doing. For example: "Now I’m going to try to use the search engine")
  • If you get to a point where you would naturally stop working, let me know. I’ll let you decide a task is complete, whether you find what you are looking for or not. Just tell me when you’re finished.
  • We’re going to be focusing on navigating through the website and editing entries. For the tasks that involve editing, we will be previewing your changes but we won’t actually be permanently saving any changes to Wikipedia. I'll walk you through that when we come to it.
  • Do you have any questions before we continue? Okay, let’s go to the Wikipedia website.
  • Take them to English Wikipedia main page
  • Log in to test account

Task 1: Finding help

Observations

J
  • "Why is Help under 'Interaction'? I think it should be somewhere more obvious." Clicks on Help in the sidebar. -> Help:Contents
  • "Getting started sounds right" -> Help:Contents/Getting started
  • "Wow, this is a lot of links" Reads for a while. "I'm not sure which of these is better, Introduction or Tutorial? I'll go for Introduction." -> Help:Getting started
  • "Oh, now this page is called Getting started as well? I guess I should click "Introduction — What is Wikipedia?", but I thought that's where I was." -> Help:Introduction
  • Reads page. "Hmmm, the 'Sandbox' sounds interesting. Maybe I should try that." -> editing Wikipedia:Sandbox
  • Types "hello world", and after some searching finds the button to preview the page.
  • I suggest we move on if he's happy to.
  • "Well I feel like I should read more, but it just seems to be going round in circles. And editing in the Sandbox was not too hard. So yes, I'm okay to move on."
E
  • Clicks on "Help" in the sidebar. -> Help:Contents
  • "This (Getting started) looks like what I want." -> Help:Contents/Getting started
  • After reading links for a few seconds. "This isn't what I expected. I'm going to try going back to the other page." Hits back button -> Help:Contents
  • "Ok, this time I'll try 'Editing Wikipedia'" -> Help:Contents/Editing Wikipedia
  • Reads. "I don't think I need the 'Introduction to the project' because I know quite a bit about Wikipedia, I just want to learn about editing. 'Contributing to Wikipedia' sounds more useful." -> Wikipedia:Contributing to Wikipedia
  • "Oh, these are videos. Cool." Watches File:Edit Button.ogv and File:Great Feeling.ogv. "Well, they were nice, but they didn't really tell me much about how to edit. Just said 'click edit and save'. Maybe I should just try that, rather than all this reading."
  • At this point I asked if she wanted to move onwards to tasks that involve editing, and she agreed.
A
  • Typed "editing" into the search box and pressed Enter, he was taken to Editing. "I guess this is about editing generally, not Wikipedia."
  • Eventually noticed the hatnote "Wikipedia:How to edit a page", which actually redirects to -> Help:Editing
  • Read the whole page (but did not watch the video).
M
  • Clicks on "Help" in the sidebar. -> Help:Contents
  • Clicks almost immediately on 'Getting started' -> Help:Contents/Getting started
  • "I'm not sure which of these to click. I don't think it's this one ('Who writes Wikipedia'), that says it's for readers and I'm supposed to be learning to edit. I'll try 'Introduction to Wikipedia'." -> Help:Getting started
  • "Oh, now it says Getting Started again. And there's another Introduction link. Well I'll try this one." -> Wikipedia:Introduction
  • Read the first page and made a successful Sandbox edit (in preview). -> Wikipedia:Introduction 2
  • "There's quite a lot on this page. 'Learn more about editing pages' sounds like it will be helpful." -> Help:Editing.
  • "This page is long. Is that a video though? Maybe that will be easier." -> watches File:Wikipedia video tutorial-1-Editing-en.ogv
  • "Well it was helpful, but her Wikipedia looked different from mine." (it was monobook) "Also it was too blurry to read the text properly and the audio was very out of sync in places."
H
  • Clicks on "Help" in the sidebar. -> Help:Contents
  • "Lots of links here, I'll try this one." -> Help:Contents/Getting started
  • "This has even more links! *looking around* What are these ones over here? *reading tutorials box* Some of these are very strange. 'Grutness's guide to stubbing' - what the heck does that mean? *reads for a bit longer* I think I'll go with "Introduction to Wikipedia, that sounds simple." -> Help:Getting started.
  • Reads. "If you would like to edit an article, the Basic tutorial will show you how. That sounds like what I want then." -> Wikipedia:Tutorial
  • "This looks a bit ugly compared to that other page."
  • Progresses through Tutorial up to Citing sources tab. "This is longer than I thought. And every page seems to get longer and more complicated. I kind of understood the other pages," (and she made successful edits in the Sandboxes) "but this is too confusing. It's a lot to learn all at once."
  • It takes her some time to open the "Visual inline citation guide", the show link is not very visible on a blue background. And she finds the text in there confusing.

Task 2: Help on references

Observations

J
  • Returned to Help:Contents
  • "Ah, I thought so, there's a page here about 'links and references'." -> Help:Contents/Links
  • Reads page. "How, why, what. I quite like that. Seems I want 'how'. " -> Wikipedia:Citing sources
  • "Ok. There's a link to something about footnotes. I thought it would be on this page. Maybe it is, but this page is kind of long. I'll see if the other one is any shorter." -> Help:Footnotes
  • "Doesn't look much shorter." Scrolls down fairly fast. "But at least this seems to have all the code. Yeah, I think this is the page I want." Reads the first four sections.
  • "Hmmm... I think I understand. Would like to try it out though."
E
  • "It was hard to find things from the Help menu before. Maybe I should just try searching for it."
  • Types 'footnotes' into search box and presses Enter -> Note (typography)
  • "Aha, 'For the usage of footnoting on Wikipedia, see Wikipedia:Cite sources and Wikipedia:Footnotes.'" The second one must be what I want. -> Help:Footnotes
  • Reads a bit. "I'm not sure I get it. Do I have to type these 'tags' in when I edit? Is that HTML? It looks a bit like it. It doesn't seem very clear."
A
M
  • Returned to Help:Contents
  • "I don't know, footnotes, I don't see anything about those here. Huh, what's this 'Search Help' button... oh, I type in there I suppose (points at box above).
  • Types "footnotes" into box, and clicks 'Search Help'.
  • First result is Help:Footnotes. "Good, that looks as if it's what I want." Clicks. -> Help:Footnotes
  • Reads, clicks on picture to expand. "This is kind of confusing. I'm not sure what all the stuff about tags means. And the page is so long."
H
  • H had already found and read the tutorial page on Referencing, so we skipped this task.

Task 3: Editing wikitext

Observations

Surprisingly nobody had any real trouble with the first part of this. Two initially clicked a section-edit link, but realised their mistake, backed out and were able to find the Edit tab. Those subjects (J, M and H) who had already made Sandbox edits were noticeably more confident in editing. A did comment negatively on the amount of "computer code" in the edit window (and this article is fairly simple compared to many others.)

Adding a link proved almost as easy. Having done much of the Tutorial earlier H was able to remember the markup for links. J and M figured it out from the other links in the wikitext and added square brackets manually. E and A found the link button in the toolbar, and used that successfully.

Task 4: Adding a footnote

Observations/Notes

  • As the only person aware of RefTools, A went straight to the Cite option in the toolbar, and had no trouble selecting {{cite book}} and filling out the required fields. She commented on how surprisingly straightforward it was.
  • The other subjects struggled more, despite 'Cite' being visible none of them thought to use it (perhaps because the inconsistent terminology reference/cite/footnote made it less clear?)
  • J succeeded in adding a simple bare reference, though had to refer back to Help:Footnotes numerous times.
  • H also referred back to where she learnt about references (Wikipedia:Tutorial), struggled more, but eventually after much copy and pasting, did manage to add a bare reference.
  • E and M did not manage to complete this task.