(71) Engaging Colleagues for Feedback on a Journal Identification Decision Support Tool

Objectives: To describe how the Center for Knowledge Management team at Vanderbilt University Medical Center effectively engaged the knowledge and expertise of external colleagues for feedback on the Scholarly Publishing Information Hub (SPI-Hub™), a decision support tool that helps prospective authors identify journals in which to publish. We used the well-proven model employed by vendors to conduct live demonstrations and formal/informal evaluations of the tool at a professional library regional chapter meeting.

Methods: Information scientists attended an MLA chapter meeting as exhibitors and provided information about and live demonstrations of our decision support tool to attendees. Developed in-house, the tool provides prospective authors, through a variety of mechanisms (e.g., author topic of publication, known journal titles, colleagues’ journal preferences as viewed through their ORCID iD), a list of scholarly journals in which to publish their work. It retrieves 25 metadata record fields indicative of quality and transparency for each journal recommendation. Following each encounter, interested participants were provided remote access for one month to further evaluate and assess the tool as they share it with their local users, and provide feedback through an IRB-approved survey. In addition to these structured survey responses, commentary was highly encouraged on site by the team through informal conversations and more formal demonstrations of the tool’s capabilities.

Results: Survey respondents (n=10) gave a median score of 4 on a 5-point Likert scale for outcomes assessed: coverage, ease of navigation, relevance of search results, clarity of information, and overall usefulness. Although we received only 10 completed surveys, the live demonstrations allowed the team to discuss and demonstrate the project to the majority of meeting exhibit attendees. In total, we engaged in informal discussion with over 80 professional colleagues, and that provided a wealth of feedback greatly surpassing our expectations. Comments fell into four main categories: content coverage, overall functionality, ease of navigation, and ideas for future development.

Conclusions: The feedback indicated buy-in from our colleagues as major user stakeholders and affirmed the tool’s value. One faculty comment enthusiastically described how the tool facilitated discovery of new journals and citations that would otherwise have been overlooked. This was exciting, as the re-introduction of serendipity as a form of journal discovery was one of the motivating factors for the development of the tool. Our experience suggests leveraging the tested exhibitor model for a non-traditional project is a valuable method for gathering external feedback and establishing user engagement from expert target stakeholders.