Wikipedia:WikiProjekt Benutzerfreundlichkeit/Test Februar 2006

Usability Test Results: Editing Information in the German Wikipedia

Version 1.0

Report by Ellen Reitmayr in the Scope of OpenUsability.org

In an explorative user test with ten Wikipedia readers, the process of becoming an author in the German Wikipedia was probed. The goal of this test was to learn about problems that arise when first using the Wikipedia Edit function, to inspect the given help, and to understand the users' expectations. Based on the results, suggestions how to facilitate the start for first-time authors are given.

In the test, it was shown that users had few problems to do simple text editing, but complex elements such as tables and images were difficult to understand. The provided help - starting from editing help along templates to the handbook - often was difficult to find or to use: Instead of providing relevant task instructions as quick references, users were confronted with background information and pages of continuous text before they actually found a solution what to do in the current situation.

This report summarises the major problems and provides suggestions how to solve them.

Specific information on contents of the help, tutorials and first steps are still missing, but will be added during the next weeks.

This report was created in the scope of the Wikipedia registration on OpenUsability, where the test materials and the report are available in PDF . Discussions of the results may take place in the forum on OpenUsability, or here on the discussion page.

In the user test, the German version of Wikipedia was utilized and evaluated with ten German users. The participants had different skills and backgrounds regarding wiki editing:

1. Five users had never tried to edit Wikipedia contents, and had never used another Wiki before.
2. Two users had tried to edit Wikipedia contents before, but something went wrong.
3. Three users had edited Wikipedia contents before, but never created a new article from scratch.

While members of Group 1 and 2 were unexperienced regarding MediaWiki syntax, those of Group 3 were more skilled.

There were three types of task sets:

1. The first task set focused on first-time authors (Group 1). After making changes in an article, the users were asked to extensively explore the different types of help that were provided on Wikipedia. Three participants performed this task set.
2. The second task set also focused on first-time authors (Group 1), as well as on members of Group 2. After making changes in an article, the users were asked to insert an image into the article. Four participants performed this task set.
3. The third task set focused on the creation of new pages and was assigned to members of Group 3. Three participants performed this task set.

Each user was tested in a single session with the moderator seated beside them. He gave them the task instructions one after another, and asked the participants to "think aloud". Furthermore, he asked the users for expectations, impressions, understanding and predefined questions of research while they were working on the tasks. By this, usage patterns as well as expectations, evaluations and reasons for problems could be identified. Between the tasks, the users were asked to evaluate different aspects of their experience on a five-point rating scale.

The test was performed with Firefox on a PC running Windows 2000, with a screen resolution of 1024x768 Pixels.

While the task sets overlapped regarding common tasks - e.g. simple editing, basic editing help, structuring articles, link marking - there were several usage paths that were followed by single persons only. This is due to the explorative character of the study: The task instructions were rather open, so users had the freedom to explore different usage paths to solve a task. For example, they were told to edit a page and were given the information that should be added on the page. However, they were neither instructed what paragraphs to add in which sequence, nor what type of formats to use, nor what types of help to use (if any). The only instruction they got was to behave "as if they were at home".

Therefore, the results of this test must not be taken as statistically granted, but should be used as indicators for possible problems and barriers that need to be addressed and further improved.

All participants were frequent internet users between 19 and 42 years. All of them were Wikipedia readers, most consulting Wikipedia at least once a week. Only one user consulted Wikipedia less often.

All in all, the participants' computer and internet skills were above average. This is, for example, reflected in the average time the participants pass with a computer or the internet: It was between 6 and 10 hours a day (except for one participants who passed only 1 hour a day on a Mac). Accordingly, the self-estimated computer skills ("How confident do you feel when interacting with a computer/ the internet?") lay between 4 and 5 on a five-point rating scale (except for a 2 by the participant who passed less time on a computer).

Accordingly, the results of the test indicated a certain computer expertise of the participants: Only few did not solve a task. When problems occurred, most "fought them out" by probing different approaches. This is not a common behaviour for novices.

Depending on their Group, the participants either worked on Task Set 1, 2 or 3.

This task set was for first time authors (Group 1) and was performed by three users.

1. You just returned from a vacation in the village Erpolzheim. You want to summarise the information and data you have gathered on Wikipedia.
2. Wikipedia offers diffent types of help to facilitate becoming an author. Please have a look at those types and help, and use them as you like.

The complete questionnaire (German) including predefined questions of the half-structured interview can be downloaded from:

www.openusability.org/download.php/82/leitfaden-wikipedia_2-autoren_einsteiger_v2.pdf (Wikipedia @ OpenUsability - Downloads section).

This task set was for first time authors (Group 1) or those who had problems before (Group 2) and was performed by four users.

1. You just returned from a vacation in the village Erpolzheim. You want to summarise the information and data you have gathered on Wikipedia.
2. You have also taken a picture of Erpolzheim and want to add it to your article as a thumbnail.

The complete questionnaire (German) including predefined questions of the half-structured interview can be downloaded from:

http://openusability.org/download.php/83/leitfaden-wikipedia_2-autoren_problemkinder.pdf (Wikipedia @ OpenUsability - Downloads section).

This task set was for Wikipedia authors who had never created an article (Group 3) and was performed by three users.

1. You just returned from a vacation in the village Ungstein. You want to summarise the information and data you have gathered on Wikipedia. A few weeks ago, when searching information about Ungstein, you could not find it in Wikipedia.
2. Now the article is created, but it is not yet categorized. Please categorize your article.

The complete questionnaire (German) including predefined questions of the half-structured interview can be downloaded from:

http://www.openusability.org/download.php/85/leitfaden-wikipedia_2-autoren_geuebt.pdf (Wikipedia @ OpenUsability - Downloads section).

In the test, the article and versions from 2006, February 06-12 were utilised. The most frequently used pages were:

• Erpolzheim: http://de.wikipedia.org/w/index.php?title=Erpolzheim&oldid=13581704
• Ungstein: not existent
• Editing Help: http://de.wikipedia.org/w/index.php?title=Hilfe:Bearbeitungshilfe&oldid=12978968
• Help: http://de.wikipedia.org/w/index.php?title=Wikipedia:Hilfe&oldid=13441877
• First Steps: http://de.wikipedia.org/w/index.php?title=Wikipedia:Erste_Schritte&oldid=11311856
• Tutorial: http://de.wikipedia.org/w/index.php?title=Wikipedia:Tutorial&oldid=13386916 (and following pages)
• Image tutorial: http://de.wikipedia.org/w/index.php?title=Wikipedia:Bildertutorial&oldid=12953453 (and following pages)

From the results, suggestions on how to improve the user experience were made. They partly refer to the German Wikipedia site, but most can be generalised for the whole Wikipedia.

In the section "Summary of Suggestions", the suggestions are listed per component, while in the rest of the report they are rather ordered by task scenarios.

The impression when first seeing the edit mode in Wikipedia is a crucial factor influencing the motivation to become an author. In the test, five of the participants had never seen Wikipedia's edit mode or other MediaWiki syntax before.

Their first impression was highly dependent on the Edit link they clicked: When clicking "Edit Page", the syntax of a table showing basic city information was displayed in the editor window. The complexity mostly overwhelmed the participants in the first run. But as they were experienced computer users, they scrolled down and found their way through the syntax.

• A short city article when clicking the Edit link on top of a page

Two users who started their first-time editing with a paragraph instead of the whole page were not confused by the syntax. However, they were faced with another problem: The location of the "Edit" links seemed to relate them with the paragraph above, not the one below. Therefore, when clicking the "Edit" link below "Geschichte", they expected to see the heading "Geschichte" and its contents.

• When clicking this link, two users expected to edit the paragraph "Geschichte".

Instead, the paragraph "Weblinks" was shown:

• Both users were highly confused and deleted the contents because they could not relate a meaning.

This expectation was not met, instead "Weblinks" appeared in the editor window. They were confused, did not know what to do. Finally, both participants deleted (!) the existing and valid text, and started to add their own text.

Suggestions:

• Facilitate the editing of single components:
  • Visually align "Edit" links with the paragraph they belong to, that means align="bottom" instead of align="top".
• Facilitate table syntax:
  • Provide an easier and more comprehensive table syntax.
  • Provide easy-to-find-and-to-apply templates for the default tables.
  • In default tables, avoid the mixture of html style elements and wiki syntax. Hide them behind style tags (e.g. <class ="german-city-table">), while the actual style attributes are defined in a separate css.

When users entered the edit mode by clicking "Edit Page", they mostly had problems to find the passages they actually wanted to edit. Some tried Firefox's search function, but it did not search through the editor field.

The most successful way to find relevant text passages was to open the article in view mode on another browser tab, and use it for orientation. By this they were quickly able to identify labels in the table, but also learned the purpose of different syntactic forms.

With simple text editing, we refer to adding unformatted text, but also simple formatting like headings, bold or italic text.

In order to use a proper syntax, participants either copied syntax elements from existing paragraphs, or they used the toolbar. To check their changes, they mostly used the preview functions (Details in Section 3.4).

Only one participant had severe problems to get used to the syntax, and finally aborted.

Even if there were only few problems, three participants explicitly wished a WYSIWYG editor.

With complex text editing, we refer to elements that are syntactically more complex than headings and similar formats which embrace text with one or two simple tags.

Little problems arose with links, even when more complicated combinations were required (e.g. [[Deutsche Weinstraße|Deutschen Weinstraße]] ). Information on how to handle that was either gathered from existing articles, or was found in the editing help. Links to non-existing pages also did not cause any problems - users were quite pragmatic with respect to that, and claimed they would add a non-existing link if the topic was worth it.

Only when the first external links was required users got a bit confused: It was unclear why internal links used the pipe symbol to separate URL from displayed text, while external links used white space.

More problematic was the editing of tables. Even though the participants were experienced computer users, and some of them had HTML skills, almost everybody deranged the table. Furthermore it was unclear, why HTML syntax elements were mixed with wiki syntax.

The derangements ranged from rather minor issues - one user, for example removed only the latter tags from text that was formatted italic. This had no visual effect as such minor errors seem to be caught by the MediaWiki parser. Other users overlooked comments in tables ( ), and wondered why their inserted text was not visible. In the worst cases, users simply deleted parts of the table they did not understand, or added rows without knowing the proper syntax (using no pipe symbol or row separators).

One may argue that users having so much trouble with editing tables would not do that back home. This may be partly true: The task instruction asked them to insert certain data into the table. But other reasons were self-induced: Some users extended the table to get rid of the "Data" paragraph of both Erpolzheim's and Ungstein's article, or they tried to add an image to the table because they thought it would well fit there.

Suggestions:

• Facilitate table syntax:
  • Provide an easier and more comprehensive table syntax.
  • Provide easy-to-find-and-to-apply templates for the default tables.
  • In default tables, avoid the mixture of html style elements and wiki syntax. Hide them behind style tags (e.g. <class ="german-city-table">), while the actual style attributes are defined in a separate css.

As mentioned above, the two users who used an edit link of a paragraph had problems to understand which one it referred to: The placement close to the upper paragraph induced a relation to the above paragraph, while indeed it opened the below paragraph in the editor.

Furthermore, two other users missed a link to edit only the introductory part of a page. They complained that there is either the "Edit page" link, or direct links for the subsections, but none to directly and explicitly edit the introduction of an article.

• There is no way to directly and explicitly edit the introductory paragraph.

Before starting to edit the rest of the page, one user wanted to edit the table. He searched for a link to edit the table directly, and was a bit confused because he did not find it. Finally, he decided to first edit the rest of the page. When clicking "Edit page", he finally realised that the table could be edited there.

Suggestions:

• Facilitate the editing of single components:
  • Add an extra "Edit" link to tables.
  • Visually align "Edit" links with the paragraph they belong to, that means align="bottom" instead of align="top".
  • Consider adding a separate "Edit" link for the introduction paragraph of a page. Users might then be less tempted to use the "Edit page" link, but start editing single components instead of the whole page.

Direct help should provide a quick reference relevant in the current context of use. It should provide no more than the information on how to achieve the current goal - further information should be given in related background articles.

During the tests it was shown that direct help in Wikipedia is often the other way round: Instead of providing clear-cut instructions on how to proceed, it provides the user with background information - just like an encyclopaedia should do. However, in the current context of asking for help, this is inappropriate. The user is seeking for a quick reference which should support him to fulfill his actual goal.

Examples from the test are wide-spread: They reach from instructions how to create a new page, over instructions how to upload an image (where the image syntax was explained thoroughly, but no hint how to upload an image), to the usage of templates (where the purpose and creation of templates was the most prominent help one could find). Another interesting issue was when a user clicked the related login link ("Anmelden") in the first step of the picture tutorial, and instead of getting a login mask, she was faced with a long article about benefits of getting an account.

Suggestion:

• Wikipedia administrators and authors should go through the common use cases (insert image, add category, new article, first time editing, etc.) and check the provided help. Instead of linking to explanation articles, they should interpose pages which provide step-wise instructions how to achieve a certain goal, only then provide a link to more extensive explanations and background information.

Some examples are given in the following.

The findability of the editing help was rather low in the test. Only one person found it without being explicitly asked if there is any help available.

From a usability-point of view, the problem with the placement of the "Editing help" link is that it is at the very bottom of the text editor, next to the buttons to save, preview or cancel changes. These actions are usually initiated when the user finished writing. Instead, while writing, that means when the probability that a user needs editing help is highest, the link is outside the field of attention. Additionally, on a 1024x768 screen, it is most likely outside the viewport.

• The editing help is an inconspicuous link below the editor panel.

Regarding the contents of the editing help, the users were mostly interested in the actual editing help given in the table on the right of the page. The contents on the left were scanned by a part of the participants, but were not perceived as a benefit.

While some of the information given there actually is of little use in the context of an editing help (e.g. information about conflicts or locked pages which should be given only in the context of their actual occurrence), others were important: The information to add summaries and to use the preview. However, as they were embedded into continuous text while the users were looking for a quick reference, none of the ten participants realised those important instructions.

Regarding the table on the right, some users had problems to understand that the right column provided a preview of the syntax elements provided on the left. This may be due to the different colours used for the two columns, which indicated a relation of elements in one column rather than a relation of elements in one row.

• Extract from the Editing help (Feb 2006).

Suggestions:

• To increase the findability of the editing help, move its link next to the toolbar (right-aligned).

• Reduce the amount of continuous text in the editing help to make it a quick reference:
  • Instead of continuous text, use bullet points.
  • Provide essential information only: Summary/comment, signing in discussions, possibly the advantages of login (short!).
  • Do not provide information about conflicts and locked pages here, but in the context of their actual occurrence.
• Provide further links to relevant topics:
  • Uploading images (in the images section)
  • FAQ and Help (start page).
  • Direct link to templates and code snippets.
  • Formulas.
  • Categorising articles.
  • Tutorial (First time author and image tutorial).
• Change the design of the table: Instead of alternating the colour of columns, alternate the colour of rows to visualise the relation between a syntax element and its appearance in an article.

Instead, the findability of the toolbar was quite high: Half of all participants used it without being pointed to it by the moderator. Those who used the toolbar valued its presence, but criticised the icons being used.

• The toolbar was placed on top of the editor panel.

Suggestions:

• Provide icons that better describe their purpose, e.g. standard editor icons (OOo icons or another well-knows free text editor).

Below the editor panel and the save button, there was a row of special characters that could be inserted into the editor panel. As with the toolbar, they marked text and expected the marked text to be replaced by the character they inserted. This was not the case, as the special characters do not replace a selection.

One user tried the special character   and found a bug: When inserting it, the cursor remains behind the & and does not jump to the last position (as the case for other special characters). Therefore, when adding multiple   , they are inserted within the last one which results in syntax errors.

Also, users were confused by special characters like  , [[]]or {{}}. They did not know what effect those characters would have. The tool tip also was of little help as it explained how to insert the character, but did not tell its meaning.

Suggestion:

• Fix the problem bug concerned with  
• For each special character, provide a tooltip that explains it's purpose and how to insert it (e.g. "Click to insert a line break").

Very problematic was the absence of a quick and direct image help. Out of the four users who worked on Task Set 2, three performed the tutorial, but were highly discouraged when they first opened it: "A five-step tutorial to add an image to an article? Is it that difficult?".

One had tried to use the toolbar to add an image before, and got confused because there was no hint that (and how) an image has to be uploaded before it can be added to an article. She then followed the "More Info" link in the editing help, and got confused once more: As she skipped the introduction, all the information she found was how to format images (see http://de.wikipedia.org/w/index.php?title=Hilfe:Bilder&oldid=13428185). The essential link "Image tutorial" did not imply help on uploading images, and it was embedded into the introduction where it did not attract her attention until she felt really lost and decided to start all over.

Considering that the participants in the test uploaded images for the first time, the tutorial (including login and license instructions) was quite appropriate. But for a user who has already uploaded an image before, but can not remember how to do it, the login and license parts can be skipped. A simple hint "Before adding an image you have to upload it on [http://commons.wikimedia.org/wiki/Special:Upload].

For details on uploading images see Section 3.3.1.

Suggestion:

• In the image section of the editing help, provide a direct hint: "Upload your image first on [1]".
• The same accounts for other occasions where image help appears (to be analysed!).

Templates are supposed to increase the structural consistency of articles, the sequence and labelling of sections in related articles, and the visual consistency of special elements such as tables, warnings and hints. Furthermore, templates are a good way to provide beginners with editing help as are contextual (help embedded into the context of use).

In the test, the usage of templates was probed in Task Set 3. None of the three searched for templates himself, but had to be pointed to them by the moderator ("Have you seen any templates that facilitate structuring a new article?"). Only then, they started to search for the templates.

Actually finding the templates then was another problem. One user entered "Help Templates Wikipedia" in Google and was directed to the page page Wikipedia:Vorlagen (http://de.wikipedia.org/w/index.php?title=Hilfe:Vorlagen&oldid=12987322). That page was not considered as helpful in his current context as it thoroughly described how to create templates and how to insert them via curly brackets ( {{template}}), but there was no obvious link to the templates themselves. Only at the very bottom of the page he found the proper link (Wikipedia:Formatvorlagen).

• The link to the templates themselves ("Formatvorlagen") is inconspicuous and placed at the bottom of the page.

The second users had the similar problem: While he quickly found the page Wikipedia:Vorlagen via the Wikipedia-Portal, he did not find the actual templates at all.

• In Wikipedia:Portal, the link "about templates" is listed before the link to the templates themselves.

The third user was mislead by a link labelled "Manual - including text snippets and templates" ("Handbuch (inkl. Textbausteinen und Formatvorlagen)") in the editing help. Actually, it lead the user to the Help start page which displayed a high number of links in thematical blocks. He scanned the link list for the keyword "Template", but only found it after about three minutes. He was dissatisfied that the promised contents were presented in such an unobvious way.

For details about templates themselves see Section 3.3.3.

Suggestions:

• While it is important to learn how templates are created, the more common use case is to apply templates. The related usage paths should therefore be strengthened:
  • In Wikipedia:Vorlagen, promote the link Wikipedia:Formatvorlagen at a prominent location (e.g. right navi panel).
  • Consider providing a link to the templates when a user creates a page.
  • Promote a direct link to text snippets and templates in the editing help.

All in all, authors got little direct support on how to structure their articles. As described above, a link to templates that provide structural and syntactical directions when creating an article was not existent.

On the other hand, structural help like out-commented parts in a pre-defined table (e.g. full city table were only few data was displayed) were of little use either as the comment tags ( ) were partly unknown. Also, having too many hidden fields in an article decreases the findability of relevant text paragraphs in the editor.

The most common way to structure the own article (in all task sets) was therefore to reference related articles, or to do it "as it seemed appropriate".

The problem about referencing related articles is that there it is neither guaranteed that they meet the criteria of an "excellent article", nor that their structure is appropriate: For example, one user referenced "Berlin" when he wanted to add paragraphs to the village "Erpolzheim" - confused he decided to do it "as it seemed appropriate". Another one first referenced "Wine" because Erpolzheim is in a wine-growing area.

While referencing related articles is rather easy for cities, it is more difficult for other areas of interest: What is an appropriate reference for "Paper Prototyping"? Is it necessary at all to use a reference in this context?

Besides from the findability of image help as described in Section 3.2.2.5 in the context of "Direct Help", uploading and inserting images implied some more difficulties. Some of the difficulties were induced by the image tutorial - those will be described in Section 3.5.4 in the context of "Help Contents". Others were induced by the workflow of uploading and inserting images itself, which will be described here.

To upload images, the user needs to be logged in. When clicking the link "Upload" in the left navigation panel ("Tools"), he is directed to the upload section of the current platform - that means either de.wikipedia.org, or en.wikipedia.org, or commons.wikimedia.org. For each platform, another login is required.

This fact is problematic by several reasons: Firstly, in Step 3 of the image tutorial, it was communicated that uploading images to de.wikipedia.org was not appreciated and a direct link to the Commons upload was provided. It was explained where to find it (in the left "Tools" bar). Unfortunately, the de.wikipedia.org upload link is located at the same location, so that the user was unknowingly pointed to this unappreciated link. It is questionable why a non-appreciated link is shown in the "Tools" bar at all.

• Link to Commons and information about general location of the upload links.

Secondly, it is likely that users do not think of their current platform environment while working with Wikipedia. For them, the German and the English Wikipedia are the same unit. Commons is a concept that users might not even know: In the test, two out of four who worked through the image tutorial did not realise that they were directed to Commons to upload their images. The site structure seemed to be equal, the icon had similar colours - nothing seemed to have changed.

In the test, two users had severe problems to understand why they were logged in to Wikipedia, and still could not upload images to Commons.

As a result of working with the image tutorial it might easily happen that users created a login at Commons, but later click the upload link on de.wikipedia.org. and will not understand why their login is not accepted (not probed in the usability test).

Generally speaking: It is most likely that users logged in to one platform will not understand why the upload links on this platform work, why on other platforms they first have to sign in - and even worse: their login is not accepted.

Suggestions:

• Consider carefully if upload on de.wikipedia.org is appreciated. If not, redirect all the "Upload" links provided on de.wikipedia.org to Wikimedia Commons (make the local upload unreachable via links).
  • Redirect "Upload" in de.wikipedia.org's left navigation bar to the upload area of Commons.
  • Do the same with all other links currently pointing to de.wikipedia.org's upload.
  • In the tutorial, remove the reference to the local upload, only talk about the Commons upload.
• On Wikimedia Commons, provide a clear message that the login is separate from Wikipedia, and that possibly a new login has to be created. After the new account confirmation, direct the user to the upload page (currently he is directed to a "Login Successful" page).

The uploading page itself made it hard to find the relevant parts. In the upper part, there was a long introduction how to label the file and how to summarise image information. Even if this information was read by most participants in the test, they had forgotten about it once they arrived at the corresponding input fields: In the summary field, only one of the four working on this task added information (author, source, etc.).

• Commons Upload Page.

Even if not addressed in the usability test, it is most likely that many users will not understand and use the concept of "Source filename" versus "Destination filename". The high number of images starting with "img_" on Commons supports this assumption. It is most likely that a different page layout that explains the concept contextually would be of help.

Suggestion:

• Change the layout of the upload page:
  • Instead of providing a long introduction, provide single hints embedded into their context of use. For example, next to the source and destination file fields, provide the hint that descriptive file names should be used. Move the summary hint to the summary field. Move the license hint and links to the license fields. Importantly, make sure that the hints are short and to the point. They should not disturb the upload process, but function as unintrusive additional information.
  • Provide the image code snippet ( [[Image:File.jpeg|thumb|Caption]]) after the upload, that is when the user needs it for reference.
• Make it easy to use summary code snippets. Currently, when following the link "Summary", the user either has to open it in a new tab/window, or he "loses" his upload screen.

Consider to add popup windows to Wikipedia's organisational pages, that provide short and to the point contextual help. Such popups might also be useful at other occasions in Wikipedia.

On the upload page, the user had to select a license from a drop down menu. The users had problems to scan through the variety of licenses. One of the participants in the test was unsure which license to choose and simply selected "I don't know what the license is".

• The selection of possible licenses.

As a consequence, a warning was displayed below the uploaded image, telling the user that no license was assigned, and that it would be deleted after seven days. The user wanted to avoid this, and searched for an option to assign a valid license. The warning gave no hint how to change the license, so he simply clicked the edit link for the paragraph where the license was displayed.

• Warning that no license is assigned and edit link for the license.
• and content of the editor for this paragraph

The editor for the license paragraph did not provide any help how to add a valid license. The user decided to simply deleted the warning and was glad to see the warning had disappeared. Still, he was unsure if his image was in danger to be deleted, edited the paragraph once more and added "GPL" (the correct notion would have been{{GPL}} ).

Suggestion:

• Facilitate the selection of a license:
  • Currently, the list of licenses is hard to scan. Many licenses start with the same words ("my own work", "GNU"). Possibly two separate drop-downs, where the choice of the first (e.g. own work, gnu licenses) influences the contents of the second (sub licenses), with a decent default for the second one.
• Facilitate the later editing of a license.
  • In the warning, provide a hint how to change the license.
  • Instead of a simple editor, think of providing the drop-down, or at least a link to the license templates.

Each of the three participants working on Task Set 3 - where the participants should add information about the village Ungstein to Wikipedia (an article, that was not present at that time) - had problems to create a new page. Each of them needed three to four attempts to find an appropriate way how to create an article.

In each case, the problem was not the knowledge how to create a new article (each of the knew that entering an empty reference into another article would work, or simply typing the name of the new article into the browser's address bar), but they were unsure how to best start it.

They searched for related articles where they might add a link to Ungstein, asked Google, looked it up in the tutorial - but none of the sources was perceived as helpful.

One user consulted the tutorial for help - and was deeply dissatisfied: In the tutorial, it was described that empty references were marked red, and that they could be used to create new articles, but when following the related link "Create a new page" (http://de.wikipedia.org/w/index.php?title=Hilfe:Neue_Seite_anlegen&oldid=9924950), he got confused: There, it was thoroughly explained how not to create a new article, but it did not tell the user how to do it right. Confused and disappointed, he finally chose a way marked as "unappreciated", because he simply did not know how to do it in a better way.

Interestingly, all three users who performed that task typed Ungstein into the Wikipedia search, but no one saw the option to create the article there.

• When there is no search result, there is an empty reference to create an article. This option was not seen in the test.

Out of the three who worked on this task, two finally created the article by typing "Ungstein" into the browser's address bar. On the page that came up, a lot of information was provided, but it was disregarded.

This may be due to the high amount of text, but also due to the type of information given here: The first item provides a link to edit the article, the second and third to search for the article. Only below that, there are hints what to write there, and what to move to other members of the Wikimedia family. It is strongly suggested that a user who wants to create a page won't even read down to those paragraphs. The test supports this hypothesis.

• The information when creating a page via the browser's address bar.

Suggestion:

• In the tutorial and on the background page about new articles, provide clear instructions how to create a new page.
• Consider if you want to point users to creating a new article when there are no search results. If so, make the link more eye-catching:
  • Do not embed it in continuous text. Rather write: "Create new Article: Ungstein".
• On the page to create a new article, give directions how to create a new article:
  • Group these directions with the link to force the users to read them.
  • These directions should be of high interest for both the user and Wikipedia:
    • Naming conventions
    • Links to templates and categories, or a template and category selection dialog.
    • Etc.
  • Add this page in either case when a new article is created (browser address bar, empty link, link in search results).

When being asked about naming conventions ("Should the article be labelled "Ungstein" or "Ungstein an der Weinstrasse"), the three participants who worked on this task mostly got unsure.

Suggestions:

• Before opening the page editor, provide information about naming conventions.

All the three participants copied the layout from another page instead of using a template (see above).

Suggestions:

• Before opening the page editor, provide a link to the templates section, or a template selection dialog.

The use of templates is a means that - when properly applied - might add a high value to Wikipedia's consistency and quality of contents: When users see how to structure an article, and which basic data should be provided (e.g. in a table), the efforts when writing an article ("How to start?") might be reduced. However, the workflow of finding and using templates in not supported sufficiently in the current Wikipedia.

As described above (Section 3.2.2.6 and 3.3.2), finding templates was problematic. These issues can be addressed as suggested in the corresponding sections. Still, the templates themselves require attention: When the three users who worked on Task Set 3 finally arrived at the page providing templates, each of them had problems to pick one. The reason is that Austrian and Swiss cities were grouped close to each other, both starting with the keyword "Ort" ("location"), while German cities were listed as "Gemeinden und Städte" ("counties and cities"). While "Ort" is a common label for smaller cities in Germany, "Gemeinden" is a rather formal notion which makes it harder to detect it in a list. Furthermore, Indian states and islands were listed in between. All in all there was no intuitive order in the version used in the test (http://de.wikipedia.org/w/index.php?title=Wikipedia:Formatvorlagen&oldid=13393548).

• The sequence of geographical templates is not intuitive

The template which had to be chosen in the test itself was considered as too complex (http://de.wikipedia.org/w/index.php?title=Wikipedia:Formatvorlage_Stadt&oldid=12941439). It provided the maximum information which is necessary for a huge city, but none appropriate for smaller cities or villages. The users felt overwhelmed by the number of sections and subsections, and the code snippet "zum rausschnibbeln" ("for cutting out") was perceived as too complex to use.

Suggestions:

• Clean up the template categorisation and ordering.
• For complex templates, provide a "light" version as to facilitate an easy access, and the full version as a complete reference.

In Task Set 3, participants were asked to categorise their newly created article "Ungstein" so that other users could easily find it. Out of the three participants who worked on this task, one did not succeed to categorise his article without a hint by the moderator. Another one had problems but finally succeeded, and one simply copied the footer from another article.

The one who succeeded only with a hint by the moderator expected the categorisation to work the other way round than implemented in Wikipedia: Instead of adding a category tag to the article, he went to the categories' section and searched for a way to add his article to the category "Orte in Rheinland-Pfalz". Arriving on that page, he was helpless how to proceed as he neither found a link to add an article to the category, nor did the source code allow for simply inserting an article. When he claimed that he would abort right now, the moderator pointed him to a reference article ("Erpolzheim"). There, he found the tag [[Kategorie:Ort in Rheinland-Pfalz]], and he also copied a special navigation bar and the tag {{stub}}. When being asked, he claimed that he did not know its meaning, but copied it as it seemed to belong there.

The one who had problems but succeeded without a hint did not use a reference article, but as he had found a template for villages and cities in the previous task, he returned to that templates and copied the footer [[Kategorie:Ort in Bundesland]]. Then, he followed the category link and checked if Ungstein was listed there (which it was).

• View and source of the category "Orte in Rheinland-Pfalz" (Feb 2006)
• View and source of the category "Orte in Rheinland-Pfalz" (Feb 2006)

Suggestions:

• Facilitate the proper classification of articles:
  • Either facilitate the usage of templates where you provide instructions which categories to select.
  • And/or consider providing a simple dialog to select categories in an easy way.

Two out of the three users wanted to check their categorisation by using the preview function. Both were unsettled because the category did not appear right below article preview (where expected). Actually it was shown below the editor field where both did not find it.

• The category preview is located at the very bottom of the page.

Suggestions:

• Move the category preview up to the preview of the article.

After having categorised their article, the participants were asked to double-check if the category they chose was the best suitable one. They either went bottom-up (starting "Orte in Rheinland-Pfalz") or upside-down (starting "Wikipedia:Hauptkategorien") through the tree of categories. Especially the upside down approach was perceived as highly complex, as the alphabetical order of subcategories was highly inconsistent (e.g. "G" for "Geographie (Europa)", "!" for "Ort in Europa", and "D" for "Ort in Deutschland").

As an overall evaluation, one user claimed that he would leave the task of classifying his article to "Wikipedia experts".

• 1...
• 2...
• Three different types to order subcategories alphabetically (Feb 2006).

Suggestions:

• Go through the whole tree of categories and check for inconsistencies regarding the naming and classification of categories.

After categorising their articles, the three participants working on this task were asked if they knew the Wikipedia Portals. None of them did, but they soon found "Themenportale" in the left navigation bar. Each of them went to the portal for the province "Rheinland-Pfalz" (http://de.wikipedia.org/w/index.php?title=Portal:Rheinland-Pfalz&oldid=13375213), and there found a link to cities in Rheinland-Pfalz (http://de.wikipedia.org/w/index.php?title=Liste_der_Orte_in_Rheinland-Pfalz&oldid=13446463).

• Excerpt from the "Rheinland-Pfalz" Portal page (Feb 2006).

Arriving at that page, each of the three users was at a loss: They were faced a page full of links, being separated in several subsections. None of them was sure where to add "Ungstein", and even more concrete they did not know how to start it. It was unclear if their city would appear automatically (as the case for categories), or if action was required. Only when they used the browser's search function they realised that Ungstein was not listed there automatically, and used the page's edit links to add a link to the article "Ungstein".

Each of the three participants rated the benefit of such link lists as rather low. Given the example of the province "Rheinland-Pfalz", the purpose of portals did not become clear for them as the contents of the page were very hard to scan.

Suggestions:

• Clean up all portal pages as proposed by Elian (http://de.wikipedia.org/wiki/Benutzer:Elian/Portale).
• Consider to introduce relations between categories and portals. For example, when a new article is added in a certain category, it might appear in the "New Articles" area of the corresponding portal.

On each article describing a city in the commune of Bad Dürkheim, there is a navigation bar listing the linked cities. One participant noted that Ungstein (his newly created article) was not listed there, and decided to add it.

• A navigation bar.

He entered "Wikipedia Navigationsleiste" into Google search, and arrived at the page "Wikipedia:Navigationsleiste". As he could not find any information there, he went to the discussion of that page, and finally found a thread explaining exactly the problem of editing a navigation bar. According to that explanation, he finally succeeded editing the navigation bar of the commune "Bad Dürkheim".

The participants claimed that he had a hard time finding the solution - which actually is in the interest of Wikipedia, because navigation bars usually should not be edited by common users.

Suggestions:

• If you further want to hinder users from editing navigation bars, delete the instructions from the given discussion page.

One prospective of the test was to learn if unexperienced users make use of the preview function in order to validate their changes before actually saving them. By this, the preview is meant to increase the quality of Wikipedia articles, and is meant to reduce the number of entries in an article's history.

• Save and preview buttons in the German Wikipedia.

All in all, the rather unexperienced participants in the test appreciated the preview function, but at first did not find it, or did not understand its purpose. In the editor window, the "Save" button is at a more prominent location and is marked in bold font - in short: it more attracts attention than the preview button.

One user did not make use the preview function at all, which resulted in numerous "save and edit" actions to fix syntactical errors. Such save and edit actions make it hard to track changes done by a single user, and should be avoided.

Suggestion:

• Point users to the preview:
  • Do not provide a "Save" button when first opening the editor. The user has to preview the page first, then move down to the save button in order to commit his changes. (This way of handling the preview function is, for example, implemented in Drupal.)
  • Mark the preview more clearly as preview, e.g. by painting a red border around the previewed article to avoid losses.

While the links to edit single paragraphs are not provided in the preview, the "Edit Page" link on top of the page is still available. In the test, one user clicked the link while he was in preview mode, assuming that it would open the editor containing his changes. As he had not scrolled down the page, he was not aware that below the preview panel, there was another editor pane.

• "Edit Page" link Wikipedia's preview mode.

Other than expected, the "Edit Page" link did not open the editor containing the user's unsaved changes, but the original contents. The user was irritated, but managed to recover his modifications by pressing the browser's back button. Still, he was annoyed by the unexpected behaviour.

Suggestions:

• Try to avoid this situation:
  • Either deactivate the "Edit Page" button in preview mode,
  • or open the editor without a preview. Note that it should contain the currently previewed changes.

While there are prominent buttons to save, preview and view changes of a page, the option to cancel changes is only an unobtrusive link located next to the buttons. Even if there were no problems in the test, it is advisable to provide a more prominent option to cancel changes - simply to provide unexperienced authors with a clear exit function.

At the same time it is arguable if the button to show changes ("Änderungen zeigen") provides a benefit as it hardly understood by the participants in the test.

Suggestion:

• • Remove the button to show changes.
  • Instead provide a clear function to cancel the editing.

In the German Wikipedia, there is a huge variety of help. For getting started, there is a Wikipedia Tour, there are first steps, and there is a tutorial. There is an "About Wikipedia" page, and there is a Wikipedia-Portal. There is a help, sometimes referred to as handbook. Inside the different types of help, they were often distracted by related or embedded links, or they did not know which chapter to read in the handbook.

All in all, the users in the test often were overwhelmed by all the offerings. They were unsure which type of help or chapter to choose, and the necessity to decide which one might suit best often annoyed them.

However - the right-aligned navigation bar, e.g. on the Wikipedia:Willkomen page, were of high value to guide the users. The keywords "Tutorial" and "Image Tutorial" which were listed there were perceived by almost half of the participants, and were evaluated as useful links.

• Navigation bar on the Wikipedia:Willkommen page.

For Usability-Efforts regarding Wikipedia-Help see Tina's Vorschlaege and Tipps fuer Hilfeseiten and Aenderungsvorschlaege fuer Hilfeseiten.

For Usability-Efforts regarding Wikipedia-Help see Tina's Vorschlaege and Tipps fuer Hilfeseiten and Aenderungsvorschlaege fuer Hilfeseiten.

For Usability-Efforts regarding Wikipedia-Help see Tina's Vorschlaege and Tipps fuer Hilfeseiten and Aenderungsvorschlaege fuer Hilfeseiten.

Major Problems:

• Help - were searching for long time, often did not find what they were looking for.
• High number of context links was confusing > Lost in hyperspace. Instead of providing quick references, the whole context of the problem is described. That may be interesting for a handbook, but at first there should always be instructions that facilitate solving a given problem.
• Users often gave up, claimed that they would ask a forum, or they used google.

For Usability-Efforts regarding Wikipedia-Help see Tina's Vorschlaege and Tipps fuer Hilfeseiten and Aenderungsvorschlaege fuer Hilfeseiten.

Suggestions:

• Facilitate table syntax:
  • Provide an easier and more comprehensive table syntax.
  • Provide easy-to-find-and-to-apply templates for the default tables.
  • In default tables, avoid the mixture of html style elements and wiki syntax. Hide them behind style tags (e.g. <class ="german-city-table">), while the actual style attributes are defined in a separate css.

Suggestions:

• Facilitate the editing of single components:
  • Visually align "Edit" links with the paragraph they belong to, that means align="bottom" instead of align="top".
  • Add an extra "Edit" link to tables.
  • Consider adding a separate "Edit" link for the introduction paragraph of a page. Users might then be less tempted to use the "Edit page" link, but start editing single components instead of the whole page.

Suggestions:

• To increase the findability of the editing help, move its link next to the toolbar (right-aligned).
• In the toolbar, provide icons that better describe their purpose, e.g. standard editor icons (OOo icons or another well-knows free text editor).

• Fix the problem bug concerned with the insertion of the special character  
• For each special character, provide a tooltip that explains it's purpose and how to insert it (e.g. "Click to insert a line break").

• Facilitate the proper classification of articles:

• • Either facilitate the usage of templates where you provide instructions which categories to select.
  • And/or consider providing a simple dialog to select categories in an easy way.

• Point users to the preview:
  • Do not prove a "Save" button when first opening the editor. The user has preview the page first, then move down to the save button in order to commit his changes. (This way of handling the preview function is, for example, implemented in Drupal.)
  • Mark the preview more clearly as preview, e.g. by painting a red border around the previewed article to avoid losses.
• In Preview mode:
  • Either deactivate the "Edit Page" button in preview mode,
  • or open the editor without a preview. Note that it should contain the currently previewed changes.
• Move the category preview up to the preview of the article.

• Provide a clear cancel function:
  • Remove the button to show changes.
  • Instead provide a button to cancel the editing.

Suggestions:

• Consider if you want to point users to creating a new article when there are no search results. If so, make the link more eye-catching:
  • Do not embed it in continuous text. Rather write: "Create new Article: Ungstein".

Suggestions:

• On the page to create a new article, give directions how to create a new article:
  • Group these directions with the link to force the users to read them.
  • These directions should be of high interest for both the user and Wikipedia:
    • Naming conventions
    • Links to templates and categories, or a template and category selection dialog.
    • Etc.
  • Add this page in either case when a new article is created (browser address bar, empty link, link in search results).
• Provide this page in what way ever the user created the new page:
  • After entering a non-existent URL into the address bar.
  • After clicking any red link.
  • After clicking the new page link on the search results page.

Suggestions:

• Wikipedia administrators and authors should go through the common use cases (insert image, add category, new article, first time editing, etc.) and check the provided help. Instead of linking to explanation articles, they should interpose pages which provide step-wise instructions how to achieve a certain goal, only then provide a link to more extensive explanations and background information.

For example:

• • In the image section of the editing help, provide a direct hint: "Upload your image first on [2]".

The same accounts for other occasions where image help appears (to be analysed!).

• While it is important to learn how templates are created, the more common use case is to apply templates. The related usage paths should therefore be strengthened:
  • In Wikipedia:Vorlagen, promote the link Wikipedia:Formatvorlagen at a prominent location (e.g. right navi panel).
  • Consider providing a link to the templates when a user creates a page.
  • Promote a direct link to text snippets and templates in the editing help.

• In the tutorial and on the background page about new articles, provide clear instructions how to create a new page.
• Before opening the page editor, provide information about naming conventions.

Suggestions:

• Consider carefully if upload on de.wikipedia.org is appreciated. If not, redirect all the "Upload" links provided on de.wikipedia.org to Wikimedia Commons (make the local upload unreachable via links).
  • Redirect "Upload" in de.wikipedia.org's left navigation bar to the upload area of Commons.
  • Do the same with all other links currently pointing to de.wikipedia.org's upload.
  • In the tutorial, remove the reference to the local upload, only talk about the Commons upload.

• On Wikimedia Commons, provide a clear message that the login is separate from Wikipedia, and that possibly a new login has to be created. After the new account confirmation, direct the user to the upload page (currently he is directed to a "Login Successful" page).
• Change the layout of the upload page:
  • Instead of providing a long introduction, provide single hints embedded into their context of use. For example, next to the source and destination file fields, provide the hint that descriptive file names should be used. Move the summary hint to the summary field. Move the license hint and links to the license fields. Importantly, make sure that the hints are short and to the point. They should not disturb the upload process, but function as unintrusive additional information.
  • Provide the image code snippet ( [[Image:File.jpeg|thumb|Caption]] ) after the upload, that is when the user needs it for reference.
• Make it easy to use summary code snippets. Currently, when following the link "Summary", the user either has to open it in a new tab/window, or he "loses" his upload screen.

Consider to add popup windows to Wikipedia's organisational pages, that provide short and to the point contextual help. Such popups might also be useful at other occasions in Wikipedia.

• Facilitate the selection of a license:
  • Currently, the list of licenses is hard to scan. Many licenses start with the same words ("my own work", "GNU"). Possibly two separate drop-downs, where the choice of the first (e.g. own work, gnu licenses) influences the contents of the second (sub licenses), with a decent default for the second one.
• Facilitate the later editing of a license.
  • In the warning, provide a hint how to change the license.
  • Instead of a simple editor, think of providing the drop-down, or at least a link to the license templates.

• Reduce the amount of continuous text in the editing help to make it a quick reference:
  • Instead of continuous text, use bullet points.
  • Provide essential information only: Summary/comment, signing in discussions, possibly the advantages of login (short!).
  • Do not provide information about conflicts and locked pages here, but in the context of their actual occurrence.
• Provide further links to relevant topics:
  • In the image section of the editing help, provide a direct hint: "Upload your image first on [3]".
  • FAQ and Help (start page).
  • Direct link to templates and code snippets.
  • Formulas.
  • Categorising articles.
  • Tutorial (First time author and image tutorial).
• Change the design of the table: Instead of alternating the colour of columns, alternate the colour of rows to visualise the relation between a syntax element and its appearance in an article.

Suggestions:

• Clean up the template categorisation and ordering.
• For complex templates, provide a "light" version as to facilitate an easy access, and the full version as a complete reference.

Suggestions:

• Go through the whole tree of categories and check for inconsistencies regarding the naming and classification of categories.

Suggestions:

• Clean up all portal pages as proposed by Elian (http://de.wikipedia.org/wiki/Benutzer:Elian/Portale).
• Consider to introduce relations between categories and portals. For example, when a new article is added in a certain category, it might appear in the "New Articles" area of the corresponding portal.

Suggestions:

• In the tutorial and on the background page about new articles, provide clear instructions how to create a new page.