The participants were five SCD User Forum attendees: two atmospheric scientists, one meteorologist, one data processing specialist and one retired researcher with climate and computational expertise. All were advanced computer users who use both Web browsers and command-line for data access, analysis and visualization. Each user performed 5 identical tasks using the production instance of CDP.
General Findings |
|
|---|---|
| Title | Finding |
| Browse preferred over Search | On the home page, all users gravitated to browse rather than search as their initial approach to exploring data holdings. |
| File access should be emphasized over metadata | Metadata provided is too verbose and not very useful to users with the exception of NCML metadata for individual files. Users only want to see metadata when they need to consult it and commented that summaries were too long and creator/publisher metadata too emphasized. |
| Interface feedback is critical for download and form actions | Data cart confused users at multiple steps due to lack of interface feedback ("nothing changed!") |
| Help text not read until users are already confused | Everyone missed the help text about selecting files for download in file-level data access interface as well as the help text for the geo selector. They only resort to reading it until they are already confused or in trouble. |
| English preferred over cryptic acronyms | "DODS: das dds html info" and "NCML" download links were unintelligible to users and forced users to take guesses when clicking or just click them all hoping to find what they wanted. These should be translated into general non-technical terms. |
orange = high priority/severity
Interface
Problems Identified |
||||||
|---|---|---|---|---|---|---|
| Issue | Page(s) | Problem | Proposed Solution | User | Owner | Movie |
| 1 | Home | Nav tabs don't have tooltips | Add alt text to href tags | 3 | Markus | |
| 2 | Home | "Search by metadata keyword" is unclear and reads weird | Change to "Search for datasets" | 3 | Markus | |
| 3 | Advanced Search | Every user first clicked and dragged on earth image before pressing Select Region button | Either create a direct geo select capability on the advanced search page or reduce size of earth image, reposition button above it and name button "Launch Graphical Region Selector" | 1-5 | Michael | movie |
| 4 | Advanced Search | Lat/long units and +/- unclear in form | Provide labelling and/or help text in interface | 4 | Michael | |
| 5 | Geo Applet | Not sure how to confirm selection | Change "close window" button to "Continue" or "Submit" | 1-5 | Michael | |
| 6 | Geo Applet | Can't transform selection | Add square points on 4 corners and midpoints of each line to allow for selection transform | 1,5 | Michael | movie |
| 7 | Advanced Search | Geographic search yields too many global datasets and makes user question whether the filter even worked. Why aren't more exact hits to region sorted higher in results? | Do some relevance ranking in search results sorting regional datasets above global ones. We might even consider a section break that separates the global datasets. | 4,5 | Michael | |
| 8 | Advanced Search | Advanced Search fields are not automatically cleared when returning to the form, users sometimes caught this and sometimes didn't and commented it was strange behavior | Auto-clear fields when Advanced Search tab is clicked | 1,4,5 | Markus | |
| 9 | Advanced Search | Feature request | Add pulldown menu for continents, oceans, tropics and poles to easily enter regions in geo filter | 5 | Michael | |
| 10 | Search Results | Summaries are way too long | Put a word limit on the summary display that keeps it to 2 or 3 lines like Google. Ask data providers to keep summary terse and not provide so much extraneous background info (eg. GOTEX summary is simply an ad for ATD); get right to it and describe the dataset, not the data provider. Change "view long descriptions" to "descriptions: none | short | long" with short as the default. | 1,3,4 | Michael | |
| 11 | Search Results | Search results ranking algorithm unclear and not stated | Explain somewhere how the results are ranked. Consider offering ranking options like sort by: relevance | title | date | provider. | 3,4 | Markus | |
| 12 | Search Results | Number of results is at top of page but not bottom | Add number of results to footer | 3 | Markus | |
| 13 | Search Results | Titles range from cryptic to extremely general | Recommend descriptive titles to data providers | 4 | ||
| 14 | Search Results Browser | Results browser is not noticed or used | Rename to "filter results by". More drastic measure would be providing it as categorical results on par with individual dataset results rather than as sidebar nav. | 1-4 | Markus | |
| 15 | Search Results Browser | User didn't click on triangle to expand keyword category even when given hints | When user clicks a results browser category, show the subcategories with a parent breadcrumb trail above (this is to avoid having to scroll down to see expanded categories) | 5 | Markus | movie |
| Search results, Dataset | No hitword highlighting | Provide hitword highlighting in search results and datasets if technically feasible | 3 | |||
| 16 | Dataset | Metadata above Nested Datasets is mostly useless and distracting, several users mentioned that they would prefer to get right to data browsing and only consult metadata if necessary by expanding it | Move nested datasets and file-level access to top of page and provide expandable creator/publisher and metadata sections | 2-5 | Markus | |
| 17 | Dataset | Multiple download options not noticed when one option is MSS | Rethink how to provide multiple download options. Perhaps we should provide just one option (give preference to CDP disk), simply call it download, and provide a button or link to show alternate download options if there are any for listed datasets. Names for download options also need to be rethought. "DISK [HTTPSERVER]" is too verbose. "CDP disk" would do. | 2 | Markus | movie |
| 18 | Dataset | DODS links extremely confusing to users, not clear that html link is the most important link to the form for download, didn't know what das and dds meant | "latest [resolver]" link seems like it should be in Metadata column. Recommend moving "info | das | dds " to the Metadata column. Choose english names for das and dds. In download column simply have "DODS" as a link to the html download form. We might want to consider framing DODS as a file-level subsetting feature instead. | 1,5 | Markus | movie |
| 19 | Dataset | Unclear what to press for downloading checked MSS files. Two users pressed Go button instead of "download" or "add to data cart." | Add an"add to cart" button at the top and bottom of the download column and aligned with it | 1-5 | Markus | movie |
| 20 | Dataset | Adding MSS files to data cart results in same screen redrawing which leaves user confused about whether action was successful | Take user to data cart page after pressing "add to cart" button | 1-5 | Markus | |
| 21 | Dataset | "Jump to page" interface confused users | Replace jump interface with a dropdown menu of choices in increments of 20 for pages to go to beyond those listed with "..." as the initially selected entry. | 1,5 | Markus | |
| 22 | Dataset | Took awhile to find "Aggregated Dataset selection and subsetting" link in subsetting task | Change name to "Subset Data by Variable or Coverage " | 1-5 | ||
| 23 | Dataset | Numerous datasets do not have nested datasets specified, requiring the user to get lost in a remote site, often dropped onto their home page and not even the specific dataset URL | We should require all data providers to specify nested datasets and provide a direct URL for each file. Exceptions can be made but they should be rare and only allowed after pushback from us, and the URL provided should link directly to the dataset, not the providers home page. A link to the external site should come under the nested datasets bar and read "Please visit the data provider's website for access to this dataset (will open in a new window)" followed by the url which is a clickable link. | 2 | Markus | |
| 24 | Dataset | "File-level access" unclear | Change name to "Download Files" | Markus | ||
| 25 | Dataset | DODS user wasn't sure what OpenDAP meant in Dataset-level access | Change name to "Subset Data by Variable (OpenDAP/DODS)" and think about potential collision with CDP subsetting feature | |||
| 26 | Dataset | Not clear what cog images mean or why they are available for some datasets and not all | Consider another visual approach to indicating additional functionality available for some datasets. Explore how we can expand the availability of additional functionality across more datasets. | 4 | Markus | |
| 27 | Dataset | Why are there several summaries for some datasets? | Make summaries more terse and avoid multiple elements with same name if possible | 1,4,5 | ||
| 28 | Dataset | Mailto links for publisher are unexpected | Remove link and instead add short name in parens after long name and follow with "website | email" links | 1 | Markus | |
| 29 | Dataset | It's not clear that the + Metadata link is a collapsed area | Refine interface element to make expandability obvious | 1,4,5 | Markus | |
| 30 | Dataset | Variable names repeated in parens | Don't repeat variable names in parens. It's redundant. | 3 | ||
| 31 | Dataset | It's hard to return to the CDP site after being linked to an external site | Pop a new window whenever we link to an external site | 3 | Michael | |
| 32 | Dataset | Not sure what to expect when clicking NCML link | Change NCML link in metadata column of file-level data access tables to "info" or "metadata" | 3 | Markus | |
| 33 | Dataset | n/a bytes is unacceptable | We know the file size if we have the file on disk so fill in the bytes number if not provided by the data provider, or require that data providers always fill in this field | 3 | Markus | |
| 34 | Dataset | bytes is too small a unit to conveniently read | Ideally, use K, MB or GB as appropriate given file size. At least change bytes to K and use commas to separate thousands. | 3 | Michael | |
| 35 | Dataset | Use of cryptic names and acronymns for nested dataset folders makes it difficult to determine what is what | Encourage data providers to use more descripting folder names for nested datasets | 3 | ||
| 36 | Dataset | "Nested Datasets" seems like an odd title | Lose the term "nested" and use folder icons to indicate it's a browseable hierarchy | 4 | Markus | |
| 37 | Dataset | Key global attributes should be obvious | Provide place, time span, contact info and any other helpful global attributes of dataset at the top of the list of metadata elements | 4 | Markus | |
| 38 | Dataset | Empty page for DSS catalog due to server being down | We need to display an error message when an external catalog is inaccessible, such as "THREDDS catalog inaccessible on external site: http://www.dss.ucar.edu/dir/file.xml" | 5 | ||
| 39 | Dataset | Users don't read help text until they get into trouble | Make help text for data download conditional based on download types present on page | 1 | Markus | |
| 40 | NCML | Table too narrow, doesn't take full advantage of monitor width | Expand width of NCML table | 4 | Markus | |
| 41 | All Metadata Pages | Variable names are inconsistent | Recommend data providers use controlled vocabularies for variable names | 4 | ||
| 42 | Data Cart | No feedback after pressing Transfer button, two users clicked Transfer again thinking their first click didn't work | Take user to a new page that lists the filenames that were queued and indicates they will be emailed at userid@ucar.edu when the files have been retrieved from tape and are ready for download. Automatically empty the cart. | 1,3 | Markus | |
| 43 | Data Cart | Users never emptied their cart and were later confused when they added files which were appended to already downloaded files | Either empty cart automatically after successful transfer, or better yet, move those files into a "downloaded files" workspace. | 1,2 | Markus | |
| 44 | Data Cart | Not sure whether GAU charge will be incurred for dataset download | Provide user with estimated GAU charge based on file size total or explain that GAUs are not being charged | 3 | Markus | |
| 45 | Data Cart | What if I want to see files I have downloaded in the past? | Expand data cart to show "Previously Downloaded Files" with the date the file was downloaded. Consider extending data cart functionality to all files, not just MSS files. | 2,3 | ||
| 46 | Data Cart | It's sometimes hard to remember the context of data downloaded later via MSS email | Provide a comment field in the form to enable the user to enter notes to help them remember what they are downloading and why. | 2 | ||
| 47 | Filenames | Filenames should be explanatory, output.nc doesn't cut it | Review filenames in system including generated ones for whether they are descriptive or not. Check that mass store and disk cache filenames match. | 2 | ||
| 48 | Subsetting | Users were confused about whether to press estimate or submit button | Change to "[Estimate] data size or [Submit]" | 3 | Markus | |
| 49 | Subsetting | Users were confused what had happened after progress bar was complete | Clearly label progress bar "Preparing subsetted data for download" which when done says "Dataset ready for download". Make download line and link more obvious, perhaps using a button instead. |
1-3 | Markus | |
| 50 | Subsetting | Clicking back button after subset download starts progress bar again | Offer more obvious links to return to the subset or dataset pages from the subset download page (look for other pages with problematic back button results as well) | 1-5 | Markus | movie |
| 51 | Subsetting | Wants data in GRIB format | Consider GRIB data format translator | Luca | ||
| 52 | Subsetting | Variable list sort order confusing | Sort variables by english name, not acronym | 2,5 | Luca | |
| 53 | Subsetting | Date format confusing | Provide date format explanation in interface | 2 | Markus | |
| 54 | Subsetting | Not clear how to select multiple entries (wasn't a Mac user) | Provide help text for making multiple selections in a scrolling list | 3 | Markus | |
| 55 | Subsetting | min value and max value seem like odd labels | Change to "start date" and "end date" | 3 | Markus | |
| 56 | Subsetting | Why is geo selector different from advanced search? | Use same geo selector everywhere (issue of how to deal with model indices) | 3 | Luca Michael |
|
| 57 | Subsetting | Can I compare a variable in one month across several years? | User said this is a common research need so we might want to think about how we can offer this functionality | 4 | Luca | |
| 58 | Subsetting | Not sure what z average is, there is only one choice in the popup menu anyway | Eliminate z average popup menu when there is only one choice | 4,5 | Luca | |
| 59 | All pages | Hard to see what has been visited when visited link color doesn't change | Change visited link color to purple for all styles (footer links are the only ones that currently change) | 4 | Markus | |