User Experience Lab: DMPTool Usability Testing

Some earlier conversations suggested that I would be assisting with a usability analysis of the DataUP Tool in the UTK-ORNL User Experience Lab.

However, more recent discussions between Mike Frame with the USGS and Carly Strasser and Patricia Cruse with the California Digital Library have shifted focus to the Data Management Planning Tool (known by the name DMPTool).

The DMPTool is another software program that I took some instruction in using as part of the 2013 University of New Mexico Walter Dean Environmental Information Management Institute.  I also had exposure to this tool as part of my coursework in Environmental Informatics with Professor Frame, in his role as an adjunct faculty member at the University of Tennessee School of Information Sciences.

Towards better familiarizing myself with the DMPTool in advance of user experience testing, I think I should dig through my course files to locate copies of presentations and training materials related to the DMPTool.

The DMPTool is part of the DataONE Investigator Toolkit. Essentially a Web application, it is described on DataONE.org as follows:

 The DMPTool is a means for researchers to develop practical data management plans consistent with agency requirements and available resources.

Additional information is available online via a video demonstration: <https://dmp.cdlib.org/help/video_demo>

And an online guide: <https://dmp.cdlib.org/help/guide>

This page essentially provides an explanation of who can use the tool, and tips for getting started including how to sign up and start a new plan.  These fairly detailed instructions on using the DMPTool may be useful in designing some “task oriented” user experience and usability testing.

There is also a concise, bulleted overview of the DMPTool here: <http://blog.dmptool.org/about-the-dmptool/>

The DMPTool blog also features  a Webinar series <http://blog.dmptool.org/webinar-series/>, with recordings available for download.

Mike Frame relayed to UT-ORNL UEL staff that the DMP is undergoing an iterative development cycle.

It is possible that this is related to the DMPTool2 Project, which has its own project page here: <https://bitbucket.org/dmptool/main/wiki/DMPTool2Project>

From that project page, it is of interest that there is a timeline for “Recruiting UI design manager,” listed as “complete” on 3-15-13.  At this time do not know who the UI design manager is, but this individual would likely be worthwhile talking to. I searched the DMPTool blog for “UI” <http://blog.dmptool.org/?s=UI> but found no reference to a user interface design manager.  A second search for ” ” brought up a blog entry stating “we now have a UI designer on-board at Purdue who will be working on the wireframe and UI aspects during this process.” It seems that the UI designer is based at Purdue.  At any rate, Andrew Sallans – signing the post as DMPTool2 Project Manager, will be able to connect us with past efforts concerning UI/UX testing on the DMPTool.

There is a full project proposal for DMPTool2 available online: https://bitbucket.org/dmptool/main/downloads/SloanDMPToolproposal.pdf

Objectives identified by Patricia Cruse, Director of the UC Curation Center at the California Digital Library include input on:

1. Workflow
2. Functions
3. User Roles

The expected benefit of this focus is to help assemble guidance for new and existing users.

Some information that will be forthcoming includes “wireframes” and a layout of graphics.  “UI Wireframes Review” was reported “Complete” to the DMPTool wiki on May 24, 2013, so this is a more recently completed section.

Phase 2 “User Interface Available (static pages) for user testing and feedback is scheduled for October 2013, with full functionality in developer environment in November, and Beta release in December. So, this user experience testing appears to be timely.

Also, from this post: https://bitbucket.org/dmptool/main/issue/26/usability-testing it is apparent that UVA was doing some usability testing (posted 2011-08-21).  It will be advantageous to coordinate with them to see what equipment they are using, what tests they have already done, and see if there is any particular unresolved issue that might inform testing for DMPTool2. (From recent correspondence and other information I have, the contact is likely Andrew Sallans with University of Virginia Libraries.  A quick google search reveals Andrew is “Head of Strategic Data Initiatives” and “Librarian for Digital Services” at Brown Science and Engineering Library.

I have a user account on DMPTool from setting up an account on DMPTool for me EIMI 2013 course, “Environmental Information Management.”

After logging in to my account, I can see the status of my plan.

I currently have one plan that I uploaded back in June – “Hobo Data Collection for 2013 EIM.” I guessed my user name, and fortunately found I had generated and saved a secure password for the DMPTool back in June with my online password manager (LastPass). Interestingly, this is the URL that I have available: https://dmp.cdlib.org/users/new?institution_id=48

At the time that I set this up, I may have entered my affiliation as a University of New Mexico. I may also have signed up with my UNM netID. That’s because, as I recall, there was not a “University of Tennessee” affiliation – which is my home institution. So, I think re-visiting the available institutions would be worthwhile (looking at “institution_id” code would likely be valuable, and a key usability question – what does a user do when they encounter that their institution is not available to select?). As an example, there is a highlighted portion of Text that I only noticed after viewing the page several times.  The text is located within a “text box” immediately to the right of the “login” portion of the site – it reads: “Anyone can use the DMP Tool” and instructs users who don’t see their institution to use “None of the Above.”

On the log-in screen, I am shown the option to log in as a “new user” or “returning user.” There is some text with some text apparently directed at new users (It would be interesting to see if existing users look at this text, using an eye-tracking user-experience evaluation):

If your institution is listed below the DMP tool will provide links to local data management resources and support available to you.

 

If you’re using the DMP tool for the first time you’ll be prompted to provide some additional information

 

There is also a drop-down menu with “Select Your Institution.”

Attempting to use Dreamweaver to extract the full list of institutions and save to an .xls file, via some transformations in Notepad++ to remove HTML markup. (Note: at present time I do not see the need to convert the file to an .XLS file, but it is now saved as a .txt file and archived at appropriate locations as described below).

After “viewing source” and copying out the portion of the HTML code pertinent to the drop-down, then pasting it into Notepad++, I can see that there are 102 listed option value choices for the drop down menu.

Note that “None of the Above” is an option, option value=”0″ and it is available at the very end of the drop-down menu.

Therefore, there are 101 institutions. This appears consistent with a recent news release of September 23 that “DMPTool adds 100th institution!” That article is linked online here: <http://blog.dmptool.org/2013/09/24/dmptool-adds-100th-institution/>.That article also lists to a wiki which apparently has updates for the institutions that participate: https://bitbucket.org/dmptool/main/wiki/Participants (which incidentally lists 101, not 100 institutions.  I surmise that UCOP does not count itself in the celebration of 100 institutions).

As I suspected, my home institution is “University of Tennessee” and the option value code is 91.   As I had suspected, University of New Mexico is option value 48. It would be an interesting user experience question to know if I can change my institution to a new location, even if I previously set up my Data Management Plan at another institution.  This might be something to try out. (Note: examining this more, it appears I am “locked” into using UNM, unless I change my institution using the “Edit DMPTool User information” tool available via  the “Manage Profile” link on the home page.

I saved this text file as “DMPTool-Inst-List-20130927.txt” on my local machine at “C:\Users\tjessel\Documents\DataONE Research\DMPTool-UX-2013,” e-mailed it to my UT Sharepoint repository folder, and uploaded it here: DMPTool-Inst-List-20130927

Upon logging in, I am taken to “My Data Management Plans”. I have the option to: edit, view, delete, share.

The option to export to Plain Text and Rich Text is also available.

I exported my class project data management plan as “Rich Text.” I am uploading that document to this research notebook: Hobo Data Collection available at <https://notebooks.dataone.org/wp-content/uploads/2013/09/Hobo-Data-Collection-.rtf>

For comparison to plain text, I also downloaded to my local machine and uploaded to this research notebook the .txt version: <https://notebooks.dataone.org/wp-content/uploads/2013/09/Hobo-Data-Collection-.txt>.

From the Web application page, some other basic information for my existing Hobo Plan is available.

That includes Plan Name; Solicitation Number (which I never supplied since my class project obviously did not have a solicitation number); Funder (I chose NSF-BIO: Biological Sciences (2013 – ); Status (I provided 5 of 5 questions – it would be worthwhile to look at those questions, I think).

I’m going to see if I can share my “Hobo Data Collection” using the [share] link. Clicking “share” appears to have generated a link:

http://dmp.cdlib.org/pdf/c4e0677e-2795-11e3-89eb-005056b30215.pdf

Note there is the option, after clicking [share] and generating a link, to [retract].

For the purposes of evaluating a “real” data management plan (versus my own data management plan that is far-from-professional or a bona fide examplar) it might be useful to search Google for the following:

“Data Management Plan” filetype:pdf and site:http://dmp.cdlib.org/pdf/

I’m surprised this yielded no results.

RE-execute, drop the extras in the URL (http)

“Data Management Plan” filetype:pdf and site:cdlib.org/pdf/

One result, and not of interest.  Possible any DMPTool plan created is not available online.

Trying something broader: “Data Management Plan” AND cdlib.org/pdf/

Unfortunately I’m not seeing any published Data Management Plans available. Possible they are not indexed by Google.

Note: later during the course of today’s research effort, I found a page with a variety of example data management plans keyed to the specific type of funder template. These plans are listed here: https://dmp.cdlib.org/pages/funder_requirements.There appear to be five sample data management plans for three types of funders. I am listing the direct hyperlinks below:

• http://www.neh.gov/files/grants/university_of_california_santa_cruzpreserving_cultural_software_objects_and_their_development.pdf
• https://dmp.cdlib.org/documents/NSF-BIO_sample1.pdf
• https://dmp.cdlib.org/documents/NSF-BIO_sample2.pdf
• https://dmp.cdlib.org/documents/NSF-GEN_sample1.pdf
• https://dmp.cdlib.org/documents/NSF-GEN_sample2.pdf

This one in particular seems to be an over-arching, generic example: https://dmp.cdlib.org/documents/Sample_Plan_DataOne.pdf

The “generic” plan was generated by the DMPTool in October 2011.  Like my class project plan generated in June 2013, the DMPTool output includes 5 main headings:

1. Types of data produced
2. Data and Metadata Standards
3. Policies for Access and Sharing
4. Policies for re-use, redistribution
5. Plans for archiving & preservation

It’s noteworthy that these differ from my class project DMPTool output’s headings:

1. Data Collected, Formats and Standards
2. Data Storage and Preservation
3. Dissemination Methods
4. Policies for Data Sharing and Public Access
5. Roles and Responsibilities

This may be because the two Data Management Plans were created with a different profile.  There is an institutional profile, and then there is the specific metadata profile.  I’ve already extracted the 101 institutions current as of September 27, 2013.  I think it’s worth looking at the various profiles available – this could be useful for understanding how various audiences might use the DMPTool.

Created a file.  There are 20 total profiles at the moment:

DMPTool-Profile-List-20130927 <https://notebooks.dataone.org/wp-content/uploads/2013/09/DMPTool-Profile-List-20130927.txt>

C:\Users\tjessel\Documents\DataONE Research\DMPTool-UX-2013

These are the 20 supported “funding templates” as they appear on the drop-down menu:

1. Gordon and Betty Moore Foundation
2. Gulf of Mexico Research Initiative
3. Institute of Education Sciences (US Dept of Education)
4. Institute of Museum and Library Services
5. National Institutes of Health
6. National Oceanic and Atmospheric Administration
7. NEH-ODH: Office of Digital Humanities
8. NSF-AGS: Atmospheric and Geospace Sciences
9. NSF-AST: Astronomical Sciences
10. NSF-BIO: Biological Sciences (2013- )
11. NSF-CHE: Chemistry Division
12. NSF-CISE: Computer and Information Science and Engineering
13. NSF-DMR: Materials Research
14. NSF-EAR: Earth Sciences
15. NSF-EFRI: Emerging Frontiers in Reseach and Innovation
16. NSF-EHR: Education and Human Resources
17. NSF-ENG: Engineering
18. NSF-GEN: Generic
19. NSF-PHY: Physics
20. NSF-SBE: Social, Behavioral, Economic Sciences

Question: will each of the sections in the plan description be different, depending on the profile used? If so, may be worth waiting to see / inquiring if there is existing documentation on each template, rather than wading through each of the 20 to compare and contrast the templates. (Note: further research effort indicated there is at least a funder template available saved as a .RTF file).

I also think it is useful to look at the prompts for the various sections – it would be interesting to know if the prompts are different for each of the 20 templates. The funding requirements are described in detail here: <https://dmp.cdlib.org/pages/funder_requirements>, and it seems to be the case that each funder has a unique text file associated with it. For example, “Gordon and Betty Moore Foundation” appears to have three requirements: Data description; Data management; Data sharing. In contrast, “NSF-Bio: Biological Sciences (2013- )” entails the 5 requirements as seen in my class project example using that template.  Note the descriptions in the template are different from the prompts in the Web application (Web application prompts are more extensive, to be specific).

From my class project example using “NSF-BIO: Biological Sciences (2013- ),” I have the following prompts when interacting with the “edit” component of my “Hobo Data Collection” plan accessed via <https://dmp.cdlib.org/plans/6578/edit>:

Data Collected, Formats and Standards

Describe the data to be collected (actual observations) during your research including amount (if known). Name the type of data, the instrument or collection approach, and how the data will be sampled. If actual data are interpreted, note the interpretation. Describe any quality control measures. Also describe the final derivative products (datasets and software or computer code) and the analysis used including analytical software packages that are required for replication, etc. Describe the format of your data; think about what details (metadata) someone else would need to be able to use these files. Describe the structural standards that you will apply in making data and metadata available. For example, for most ecological data, documentation should be structured in Ecological Metadata Language (EML). An example of metadata could also be as simple as a “readme file” to explain variables, structure of the files, etc. Consider these questions:

• What data will be generated in the research?
• What data types will you be creating or capturing? (e.g. experimental measures, qualitative, raw, processed)
• How will you capture or create the data? (This should cover content selection, instrumentation, technologies and approaches chosen, methods for naming, versioning, meeting user needs, etc, and should be sensitive to the location in which data capture is taking place.)
• If you will be using existing data, state that fact and include where you got it. What is the relationship between the data you are collecting and the existing data?
• Which file formats will you use for your data and why?
• What form will the metadata describing/documenting your data take?
• How will you create or capture these details?
• Which metadata standards will you use and why have you chosen them? (e.g. accepted domain-local standards, widespread usage)
• What contextual details (metadata) are needed to make the data you capture or collect meaningful?

Data Storage and Preservation:

Use this section to describe your long-term strategy for storing, archiving and preserving your data. Consider these questions:

Dissemination Methods:

Describe how and where you will make these data and metadata available to the community. Remember BIO is committed to timely and rapid data distribution; make sure you address how soon your data will be available. Indicate what data will be made available and preserved. Will data be accessible on a web page, by email request, via open-access repository, etc.? Consider these questions:

• What data will be made available from the study and preserved for the long-term?
• How and when will you make the data available? (Include resources needed to make the data available: equipment, systems, expertise, etc.)
• What transformations will be necessary to prepare data for preservation / data sharing?
• What metadata/ documentation will be submitted alongside the data or created on deposit/ transformation in order to make the data reusable?
• What related information will be deposited?
• What is the process for gaining access to the data?
• How long will the original data collector/creator/principal investigator retain the right to use the data before opening it up to wider use?
• Explain details of any embargo periods for political/commercial/patent or publisher reasons.

Policies for Data Sharing and Public Access:

Describe the policies under which these data will be made available. It is very important, the reason a DMP is required, that you specify how you will share your data with non-group members after the project is completed. If the data is of a sensitive nature—privacy or ecological endangerment concerns, for instance—and public access is inappropriate, address here the means by which granular control and access will be achieved (e.g. formal consent agreements, anonymized data, only available within a secure network, etc.). Consider these questions:

• Will any permission restrictions need to be placed on the data?
• Are there ethical and privacy issues? If so, how will these be resolved?
• What have you done to comply with your obligations in your IRB Protocol?
• Who will hold the intellectual property rights to the data and how might this affect data access?
• What and who are the intended or foreseeable uses/users of the data?
• Do you plan on publishing findings which rely on the data? If so, do your prospective publishers place any restrictions on other avenues of publication?

 Roles and Responsibilities

Explain how the responsibilities regarding the management of your data will be delegated. This should include time allocations, project management of technical aspects, training requirements, and contributions of non-project staff—individuals should be named where possible. Remember that those responsible for long-term decisions about your data will likely be the custodians of the repository/archive you choose to store your data. While the costs associated with your research (and the results of your research) must be specified in the Budget Justification portion of the proposal, you may want to reiterate who will be responsible for funding the management of your data. Don’t forget to include contingency plans for the departure of key personnel from the project. Consider these questions:

• Outline the staff/organizational roles and responsibilities for implementing this data management plan.
• Who will be responsible for data management and for monitoring the data management plan?
• How will adherence to this data management plan be checked or demonstrated?
• What process is in place for transferring responsibility for the data?
• Who will have responsibility over time for decisions about the data once the original personnel are no longer available?

Next steps.

At this point, I believe I have fully explored the DMPTool site’s features. I’ll be interested in how a usability study shapes up.

The next likely steps will be:

• Determine status of DMPTool2
• Review full project proposal for DMPTool2 at <https://bitbucket.org/dmptool/main/downloads/SloanDMPToolproposal.pdf>
• Review previously recorded DMPTool2 Webinar training materials
• Suggest attendance at remaining Web tutorials
• Consider the following:
• Workflow
• Functions
• User roles
• Review forthcoming “UI Wireframes”
• Get introduction to Purdue-based UI Designer, via Andrew Sallans
• Determine if there are some existing usability questions to address
• Determine if each of the individual DMPTool prompts will be the same or different user experience (with “experience” defined as the series of prompts the user is taken through, coupled with the “help” text they are provided).

Finally, I noticed that within the notebooks.dataone.org domain – there is a “Enriching DMPTool Content” category which is linked to about 20 notebook entries from the 2012 internships by user ramandell.  The entries can be accessed online: https://notebooks.dataone.org/dmptool-content/. It may be useful to review these notebook entries for insights.

From the first entry, ramandell references the “UVa Usability/UX team.”

Rachel suggests she would work on a DMPTool “kit” to send to target organizations.  Might be worthwhile in considering “personas” of potential DMPTool users for UX testing.

Mentions a Laura Miller at UVa. Mentions a NOAA template. Provides potential questions for contents of the DMPTool. Provides potential questions concerning Resources provided by the tool. Considers Design of the tool (e.g., layout and design).

Week 3 entry – did not see anything catch my attention.

Week 4 entry – Rachel mentions some other DMP related tools – Columbia University earth scientists have their own tool; there is also something called DMPonline.

Rachel discusses an early idea for a user test. She proposes a test of about 30 minutes, with a goal for 8 – 10 total tests.

She mentions potentially starting a “twitter feed” or other type of forum. She suggests that due to the limited duration of her appointment, it might not have “a chance to gain momentum and value to the community.”

Andrew Sallans replied in a comment “please do send us the specific protocol prior to testing” so it is likely that he will have some existing protocol from DMPTool1.

Week 4 post: – Mentions a User Test Protocol in Review – 10 questions, online survey, conducted virtually, and focusing on content rather than functionality.

Sallans mentioned potential interest from NOAA, Forest Service, and USGS.  Might be “newer” target audiences to consider (perhaps specific types of earth scientists).

Week 6 has some interesting back and forth with Sallans and Rachel concerning qualitative evaluation of DMPTool – https://notebooks.dataone.org/dmptool-content/the-medium-is-the-message-2/.

This post has some useful materials, specifically a September 2011 DMPTool Usability Report – Laura Miller has this.

Learning more about Usability Testing, Updates and New Ideas

Week 7 through 8 didn’t have anything that caught my eye.

Rachel’s Week 8 blog did interest me:

https://notebooks.dataone.org/dmptool-content/user-survey-responses-and-next-steps/

Particularly because it incorporated some feedback from survey respondents.  Depending on how much feedback from earlier studies is already incorporated into DMPTool2, it may be worth designing tasks around the earlier feedback.

Rachel and Sallans comment on the utility of the RTF template files in “Taking the Leap” <https://notebooks.dataone.org/dmptool-content/taking-the-leap/>.

After reviewing these, I think now I’ve looked at all there is to see.  I’ll forward this to the UTK-ORNL UEL Lab Coordinator later today.