Week 4
July 6, 2012: User Test Protocol in Review
This will be a brief post–as I have now sent off a draft of my user test protocol for review. I just wanted to add an explanation about why I chose to design the test this way.
After talking to Laura again yesterday, who again, has provided such a great resource and help, we came up with this format of 10 questions for an online survey. Since the test will be conducted virtually and we are focusing largely on content rather than functionality, Laura suggested that I create an online survey, in which I insert text from the DMPtool that I want to probe into the test itself, rather than requiring the user to interact with the tool. Though I think I should provide the link to the tool and ask for any additional feedback that the user wants to provide. Does everyone agree with this approach? This takes pressure and time off of the user and allows us to test the content of the tool. The only aspect that might be missing is feedback about the design or layout of the tool. Do you think I should include a few questions requiring the user to log into the tool?
I hope to make any necessary changes and create the actual survey online using SurveyMonkey either over the weekend or Monday in order to send it off to my first 4 participants who have agree to take the test. In total I have 5 researchers who have agreed to participate so far.
Secondly, I have been thinking more about Andrew’s comment about deciding on a few concrete goals/metrics for the marketing aspect of the internship. I agree that choosing a template within the scope of DataONE and hoping to increase usage by X amount of users within the time given is a good goal. Based on the latest statistics gathered and posted about the use of the tool in May 2012, there were 283 new users. That number seems incredibly high. What do you all think is a good goal for me to shoot for in terms of increasing the user base? Is 25 too low of an estimate? Is 50 too high? Also, I think I should focus on NSF-EAR and NSF-BIO as they seem to cover a large portion of the DataONE user community. Correct?
I want to add a second goal of getting a geo-science organization, such as USGS (?) more involved in the tool, as as far as I can tell, a presence from this type of organization is missing from the DMPtool. The NSF-GEO is not an included template either. So I would also like the focus on bringing them into the mix. Thoughts?
I also agree that this is going well, keep it up.
Regarding the questions about metrics, you are on track, but I think tying goals to specific templates or directorates would be best (rather than general user base), so I’ve asked for some specific stats in that area (copied you). Hopefully that will set us on the right course with this piece. As far as marketing/outreach goes, there are several particularly important things to think about: 1) get people in the door (ie. increase users), 2) get repeat customers (ie. quality content, functionality, etc. that makes people come back each time). We’re pretty good about getting people in for at least a first try, but now need to focus on keeping users and adding value for them to want to return each time. I think this is centered around high-quality content and guidance, local guidance and customization, and the service of keeping it all updated and current (that’s tough and expensive for an individual institution to do).
USGS is somewhat involved with our team so far (Lisa Zolly), but they are still working through their internal requirements. When they are done, they’ll likely join in fully. We also have interest from NOAA (also developing reqs) and the Forest Service (likewise). Finding other relevant requirements and bringing users into the DMPTool to take advantage of those is valuable. Getting any organizations to endorse the DMPTool and encourage users is valuable.
Great work, Rachel. Regarding the online survey – I think your approach is great. I suggest giving them a specific task (create a DMP) and then asking them your survey questions. They shouldn’t need to go back to the tool – they will remember what they liked and didn’t like.
EAR, BIO, GEO, OCE are the main dataONE-friendly NSF groups.
I think it would be really great to get more geoscience representation on the DMPTool.