User login
It is friday afternoon on a sunny July day. The last patient of the day, Ms. Connecticut, is an active hiker who has had Lyme disease previously. She found a tick on her ankle yesterday. She successfully removed the tick but has not brought the tick with her to the appointment. She had been hiking several times over the last week and is not certain when the tick bite occurred. Her question for you centers on the role of antibiotic prophylaxis and Lyme disease prevention.
TECHNOLOGY: PROBLEM AND SOLUTION
Physicians need to keep up with an ever-increasing stream of information—new guidelines, new medications, and updates in medical literature.1 They have to do this while seeing more patients with more chronic problems in less time and while meeting reporting requirements for meaningful use or quality measures for accountable care organizations.
Though some of these challenges are due to technology, one solution is to use technology to our advantage. While researching information in textbooks won’t drain a phone battery, carrying a textbook around is not feasible, and many textbooks (including their electronic versions) contain information that is outdated before they go to print or that is quickly outdated thereafter.2 Further, even online textbooks are currently more dense than the online resources that we review here.
Different types of resources can help task-saturated healthcare providers stay aware of new information while delivering evidence-based care. These tools—online textbooks, decision guides embedded within electronic health record systems, or even a Google search—are termed “point-of-care” resources when used at the time of patient care for decision-making in the moment.
Not all of these resources are of high quality, with reliable factual information. Researchers estimate that up to 70% of clinicians may use Wikipedia to research medical questions, and a comparison of 10 Wikipedia articles vs peer-reviewed sources on the 10 most costly diseases found that 9 of the 10 Wikipedia articles had errors.3,4
In an earlier article,5 we advocated a proactive approach to managing information, highlighting ways to scan for new information and to develop habits of extracting useful information that can then be stored and easily recovered. To complement this strategy and weed out erroneous information, physicians need reliable sources of unbiased information to efficiently answer clinical questions at the point of care.1,6
Here, to help busy clinicians choose which point-of-care resources to use, we review several of the most popular ones, examining their ease of use, key elements, strengths, and weaknesses.
WHAT MAKES A RESOURCE GOOD?
Key features that make point-of-care tools effective include:
Ease of use, with standard formats, a summary for each topic, or both
Links to original articles and concise, capsular summaries and syntheses of the data
Continuing medical education (CME) credit. Tip: when searching, add “CME” to the search string on the browser to access resources that provide this.
Institutional and individual accounts. For clinicians who work for large organizations, point-of-care products may be paid for already, or reimbursement may be available for your subscription. If unsure, ask your director of information technology or library services.
Freedom from advertisements. Many Internet sources have advertisements that either run alongside the information you want to see or, more annoyingly, pop up and require an action to move forward. There is also continuing concern about the effect of industry support on content.7 While not all of the resources that we use regularly and that we review here are ad-free, avoiding programs with high ad content helps limit the possibility of bias and the time it takes to access information. Although advertisements do bring up a risk of bias, resources with a low-level ad content can limit bias while providing free or low-cost access.
Evidence, not expert opinion. Many resources have an “about” page that explains their philosophy and the source of their information. It is vital to be sure that point-of-care databases are providing facts based on evidence.8 This page also typically addresses how authors and editors are selected and whether expert opinion is used when randomized trials are lacking.
Ease of access. Many tools can be accessed not only on computers but also through apps for smartphones and tablets. Some electronic medical records have clinical decision tools embedded in them, with varying capabilities.
Disclosure of conflict of interest. As conflicts of interest can shade recommendations, information sources should clearly disclose financial relationships that could be perceived as conflicts of interest—for example, authors writing about medications sold by companies with whom they have a financial relationship.
NO SINGLE RESOURCE DOES EVERYTHING
There are many types of tools for finding evidence-based medical information. Different tools serve different purposes. Table 1 lists “toolbox essentials” for clinicians needing to answer clinical questions during patient care.
For example, when a question about the need for a bone mineral density measurement comes up, it is useful to be able to quickly compare guidelines from different professional societies on the National Guideline Clearing House. For another example, if a patient brings in a medication in an unlabeled bottle, a pill identifier app can tell you what it is. Clinicians who can use these resources appropriately will be at an advantage in being able to use information to provide better care to their patients.
To date, no point-of-care summary source has been shown to be superior in all categories, and use may be driven by ease of navigation, clinician preference, clinical question, or past success.9,10
Reviewed below are several applications that can be used as point-of-care resources (Table 2).
CLINICAL EVIDENCE
Clinical Evidence provides systematic reviews on medical topics. Founded in 1999 by the British Medical Journal, it is available in print as the Clinical Evidence Handbook and in online desktop and smartphone formats.
More than any other source we reviewed, Clinical Evidence addresses not only the evidence that exists, but also the data that do not exist to guide decisions. Compared with 9 other point-of-care resources, Clinical Evidence was found to have the highest quality of evidence.11
Strengths of Clinical Evidence
- Uncommonly transparent in terms of source of evidence or disclosing when there is a lack of evidence.
- Clearly lists the strength and relevance of the evidence.
- Personalization. Users can add notes to articles, save personal searches, and bookmark pages for easy access later.
- Navigability. Users can easily access systematic reviews, key points, retracted papers, or guidelines.
- Intuitive organization, with information categorized as research, education, news, or campaigns.
- New content daily: podcasts, articles, videos.
Weaknesses of Clinical Evidence
- Limited topics (eg, Lyme disease was not available)
- The limited content is a challenge when needing quick information at the point of care and may cause most clinicians to use another source unless looking for comparisons of interventions.
- Cost. Subscribing to the service “on demand,” ie, to look up a single specific topic, costs $36 for 48 hours of access; monthly access or a “season ticket” allows 30 days of viewing of all content for $53. At over $600/year, this is one of the most costly of the sources we reviewed.
- Marketing of Clinical Evidence to academic institutions that support the service for faculty may limit its appeal to other clinicians.
DYNAMED
Dynamed, a clinical reference created by a group of physicians, was previously owned by the American College of Physicians and known as Smart Medicine; it is now owned by EBSCO.12 Reviewers investigate the literature for a given topic and create pithy summaries for busy clinicians. A top feature in Dynamed is its links to full articles cited for best practices or evidence-based guidelines. The company describes their content as free of expert opinion, while being unbiased and evidence-based.
Dynamed uses a 7-step algorithm for searched topics that identifies articles, assesses clinical relevance, evaluates validity of outcomes, compiles the evidence from multiple articles, and then updates the final recommendations daily.
Dynamed Plus, the new upgraded version, updates searched topics several times a day. Dynamed may be the most frequently updated point-of-care resource, with the least risk of conflict of interest, but it offers limited topics drawn from evidence-based findings.11,13–15
With the rapid doubling of the medical literature, frequent updates allow clinicians to be most current with practice guidelines. This potentially affects quality of care for antibiotic use, vaccination, health promotion, and screening as well as newly approved medications.
Strengths of Dynamed
- Large collection of topics, critically appraised, written for primary care physicians, presented in bulleted format
- The most frequently updated database11,14,15
- Can integrate with major electronic health records (eg, Epic, Allscripts, NextGen, Cerner)
- Has an area devoted to new information that changes current practice
- Chosen topic grouped with related topics in the differential diagnosis after the initial search
- Easy-to-read outline for quick access to information such as billing, diagnosis, and references
- Medical calculators
- No advertisements
- Helpful embedded tools
- Icons to print or email the article
- An icon to create a “perma-link” to topics, searches, and browse categories
- Graded evidence with a link to the grading model used
- Links to primary articles
- Patient information handouts
- Alerts for updated information
- CME credit
- Special consideration and features for medical education
- The upgraded version Dynamed Plus contains Micromedex for a medication database, expanded graphics, semantic search, concise overview for each topic, and expanded content.
Weaknesses of Dynamed
- Although the topic list is large, it is only about one-third the size of UpToDate.
- A subscription for a physician costs $395 a year. Residents can sign up for about $150, and students for just under $100.
- CME is obtainable but cumbersome; one submits the CME credits through Tufts Healthcare, which requires a second sign-on to access and track.
- Drug and nondrug treatments for diseases cannot be separated.
- Useful calculators include decision trees for clinical decision-making, but there is no way to search them—one must waste time scrolling through the topics and specialties looking for desired information.
- Major shortcoming: there is no medication reference tool unless you upgrade to Dynamed Plus.
- The expanded graphics of Dynamed Plus are difficult to view on mobile applications within the articles (they are brought up more reliably when searching just for the image).
- The use of strict evidence-based methodology without expert opinion is a strength, but limits the collection of topics without randomized controlled trials, for which turning to expert opinion may be the only option.
EVIDENCE ESSENTIALS
Evidence Essentials is a point-of-care resource from Wiley that offers a variety of content types. The website lists 13,000 medical topics; however, they are not all summary reviews as discussed in the other products above. Subject matter is reviewed 3 times a year. Comprehensive reviews number just under 800 individual topics, with the remaining content consisting of Cochrane reviews, calculators, decision support tools, POEMs (Patient-Oriented Evidence that Matters), evidence-based medical guidelines, and dermatology images (1,000).
Evidence Essentials provides some unique content including a quick evaluation and management (E/M) code-finder and calculators not only for the typical medical equations, but also for history and physical examination likelihood ratios and pretest probabilities, which are practical and an excellent teaching aid. It also offers CME along with POEMs, e-mail alerts, and a listing of upcoming topics.
Strengths of Evidence Essentials
- Relatively inexpensive at $85 a year.
- High-functioning filter system to choose to search one or multiple databases.
- Related results are listed for aid in differential diagnosis, similar to Dynamed.
- Authors, editors, and date of last review are highly visible. As in UpToDate, relevant medical calculators appear on the page.
- The likelihood and odds ratio calculators are a huge plus for clinical decision-making and putting guidelines into practice.
- “Overall bottom line” highlights key points
- Grading of evidence per topic.
- Bulleted and tabbed information for quick access.
- Tabs for information on background, prevention, diagnosis, treatment, references, guidelines, and special populations.
Weaknesses of Evidence Essentials
- Limited number of topics with comprehensive reviews.
- While you can click on any drug name and link to a choice of two drug databases, this is not included in the subscription and requires a second account.
- The resources tabs had some broken links. In our clinical example, the tab contained several videos at the top that were not related, followed by a map and tables that were relevant to Lyme disease.
- Likewise, some of the guideline references were disappointing. For example, the guideline link for Lyme disease is for the US Department of Labor Occupational Safety and Health Administration rather than a professional society.
- For the provider wanting a narrative, this is more of a bare-bones text.
FIRST CONSULT
First Consult is Elsevier’s point-of-care clinical decision product contained within ClinicalKey.
Unlike UpToDate and Dynamed, in which authors and editors read original articles and summarize or synthesize information for the learner, First Consult is a “smart” search engine that will research a question, together with associated terms and key words. Filters such as full-text availability, journal articles, and patient education can be applied.
You may need to read about your topic in a textbook first, and then, if you are looking for treatment information, find an original article through First Consult. It is available in mobile and desktop formats, and the point-of-care product, First Consult, has an app that can be downloaded and used for free for the first 60 days.
Importantly, the First Consult portion of ClinicalKey with the summary topics was rated by Shurtz and Foster13 as least current of the products we are discussing in this article. On the other hand, it was the only product that had an embedded program to assist the user in making presentations by allowing drag and drop of images and automatic citing of sources. Kim et al report that First Consult is one of the resources providers prefer.9
Strengths of First Consult
- Lengthy free trial
- Ability to access original articles from a list vs lengthy narrative
- Access to journals and books published by Elsevier
- Powerful search engine that applies associated terms automatically
- Patient education is available in different languages and font size with the ability to add instructions and even a local branding
- Can integrate with electronic health record
- Can filter results by guideline, patient education, topic overviews
- Presentation assistance.
Weaknesses of First Consult
- Time-intensive. A provider needing quick advice on treatment for a medical condition has to guess if an article or textbook will have the most up-to-date and digestible information, whereas this has already been summarized in other products. For the busy clinician, this may be prohibitive.
- Search results are limited to Elsevier products, and major journals such as the New England Journal of Medicine are not available.
- Inconsistent platform functionality. The app version was somewhat “sticky” to use, as pages did not always load efficiently, and the menu bar navigation is not ideal.
- Expensive, especially given cheaper alternatives. For example, subscribing to the specialty of internal medicine or family medicine costs $499 and provides access to 8 journals and 11 books. Extended access costs $998 and offers full-text access to 23 books and 45 journals. The complete service has a total of 400 journals, 700 books, and 2,500 procedural videos.
MEDSCAPE
Medscape, owned by parent company WebMD, has long been a popular resource. The most recent versions are available for both for Android and iOS mobile platforms. The desktop and mobile apps claim to be designed for point-of-care use, and can be downloaded at no cost after registering as a Medscape user.
Medscape has some interesting features, including a handy pill identifier tool that is new to Medscape and perfect for the “I take one blue pill for my cholesterol” moments. The drug information tools and other features work well offline.
Medscape contains a well-presented drug database and interaction checker, as well as a growing collection of evidence-based articles and videos with links to references in Medline. From the point-of-care standpoint, Medscape also offers a number of decision-making algorithms and a continuously updated medical literature and health-related newsfeed. It contains in-app medical calculators, searchable directories for providers, hospitals, and pharmacies, and CME that can be earned on the website or from the application.
The main Medscape website contains pop-up advertisements, but the mobile app has fewer. Among the occasional frustrations, updates are relatively infrequent, the content is slow to load, and the phone app can be cumbersome. Of note, in one review,11 Medscape was found to have the lowest quality of evidence.
Strengths of Medscape
- Free with registration
- Medical calculator
- Drug interaction checker
- Pill identifier
- Evidence-based information covering about 4,000 conditions with links to references in Medline
- Ability to e-mail articles for sharing or future reference
- CME
- Unique database of hospitals, providers, and pharmacies to aid in referrals or locating other healthcare professionals
- Algorithms for decision-making
- Images and videos for procedural review and learning
- Option for downloading certain databases for offline use
- Medical news helps you keep up with what patients are watching and reading.
Weaknesses of Medscape
- Advertisements (many of them pop-up)
- The content is updated less frequently than other products listed in this article
- The smartphone app can run slowly
- Quality of reviews may be a concern.
UpToDate
UpToDate (Wolters Kluwer) is used widely by medical students, residents, and fellows as well as practicing providers. It contains narrative reviews of topics written by respected experts directed at both clinicians and clinical staff. In hopes of appealing to many markets, it offers different subscription types so you can customize your choices with add-on features (UpToDate Desktop and UpToDate MobileComplete allow downloading of all content to be accessed offline), different service packages (1-, 2-, and 3-year subscriptions), and the traditional base product that provides online access.
Of the products we reviewed, UpToDate has the largest selection of medical topics, approaching 10,000.14 In some studies,10,15 it also had the fastest retrieval time for searches. It uses evidence-based graded recommendations that are updated regularly.
Some have lamented that there is too much information per topic.9 In response to early reviews, Wolters Kluwer has made significant changes in the platform and greatly improved the search engine. UpToDate has expanded to include CME and patient information, trying to become that Holy Grail of websites—a one-stop experience. For the lucky few, UpToDate integrates into some electronic health records and provides a relatively seamless experience at the point of care.
Strengths of UpToDate
- One-stop shopping for information, resources, and CME
- Patient information is easy to read and accessible from the same screen
- The largest repository of medical subject matter
- Ability to cull out only pediatric or adult topics
- Searching available within a medical topic
- Tabs for quick access
- The What’s New feature allows access to practice-changing medical updates
- Medical calculators
- Drug interactions
- CME is is tracked in the system, allowing for CME credit information for hospital privileges and board certification
- Flexibility of access: can use online or download content to mobile/desktop device (the online version is easy to use, although robust wireless reception is needed; offices with slow Internet benefit from the offline feature)
- Electronic health record integration is possible with the most popular systems, such as Epic, eClinicalWorks, NextGen, and Allscripts
- Patient education and medication interaction features embedded in the electronic health record; produced in collaboration with Lexicomp
- Integrated drug database
- Alerts for updates
- References have links to full-text articles
- The date of last update is easily found for verifying information accuracy
- May be provided free for clinicians who are a part of a university or large health system.
Weaknesses of UpToDate
- Articles can be lengthy, which is both a strength and a weakness. Searches can retrieve too much information.9 High volume of text can frustrate the user trying to find bulleted, easy-to-read facts. However, for the person looking for a narrative summary, the content is organized as narrative paragraphs with appropriate headers in the left margin, and the search function is robust and powerful.
- Each topic has a “Summary of Recommendations,” but answers here often require linking back to the main text.
- Patient information is sometimes at a high literacy level.
- Costs more than Dynamed. A 1-year subscription is $499 for a physician, but you have the option of paying $53 for a 30-day recurring subscription. Residents, fellows, and students can pay $199 for 1 year or $19 for a 30-day recurring subscription.
- The requirement to download means that users need to keep their version updated on all of their computers—in each of their examination rooms, for example.
- Concerns about conflict of interest arise because authors and editors may maintain financial relationships with companies that produce medications discussed in the articles they have written.
BUILDING YOUR OWN PERSONAL ONLINE REPOSITORY
Our previous article5 reviewed how to store information using tools such as Evernote and Diigo that allow information viewed on a web page to be exported to any online repository. This can be done using extensions for a web browser or by sending the information to a custom e-mail account for these services.
For information that a provider knows he or she will need repeatedly, storage in one system is actually the easiest method. Such a system can then incorporate key information from the summary tools we have reviewed here. The ideal “electronic filing cabinet” should have several features such as a the capability to label articles by topic, to separate or sort as you see fit, and a search function to find information quickly—making it a personalized and effective point-of-care resource.
STAYING AFLOAT
Clinicians make many decisions every day. In fact, the release of How Doctors Think (both publications) has led to increased research into how clinical decisions and diagnoses are formed.16,17
With the medical literature expected to double every 73 days by 2020,18 there is an ever-widening ocean of information to sift through. With this onslaught, clinicians can no longer remain fully current. Instead, refining skills in accessing, sorting, and interpreting accurate scientific evidence efficiently is crucial to time spent actually caring for patients and coordinating their care.
Guidelines, algorithms, and comprehensive databases can aid clinicians in all aspects of care, from generating more complete differential diagnoses to managing disease-specific treatment. Individuals can first think about and list the qualities of a tool that are most important to them (eg, breadth of topics, frequency of updates, integration within their electronic health record, and cost) before focusing on a few applications or websites that meet those goals. With practice, point-of-care knowledge can become part of the everyday visit.
Effective integration into electronic health records will require design input from front-line clinicians. Otherwise, systems are prone to add too much “support” and overly rely on orthodox metrics and guidelines, resulting in alarm fatigue and frustration rather than facilitation.19–23
OUR CONCLUSIONS
Comprehensive point-of-care resources can play a significant role in helping busy clinicians provide best evidence-based care to their patients. Embedded clinical decision guides within an electronic health record are ideal, but low topic coverage has limited the usefulness of these systems.24 Here are our conclusions:
Medscape, ePocrates, and Wikipedia are probably the most popular free resources. Dynamed has offered free subscriptions to Wikipedia’s top health editors with the hopes of correcting factual errors. Medscape has excellent features but is supported by sponsored content, which raises a concern about bias and potential time-consuming distractions.
Dynamed and UpToDate have both been reported to answer more questions than other sources.12
UpToDate has the largest repository, with each topic curated by an expert or experts in that subject. This content can be dense and difficult to scan quickly at the point of care, but this is balanced by the ability to search within a medical topic, which has given it the fastest retrieval time.15 It does, however, allow authors and editors to maintain financial relationships with companies that produce medications discussed in the article.22
Dynamed has the advantage in frequency of updates, clearest conflict-of-interest policy, and the least amount of conflict of interest. Its topic list is not as extensive as UpToDate’s due to the limitation of using only evidence-based medicine without expert opinion.
First Consult has high user satisfaction, but as a point-of-care resource it can be time-consuming to find the best source for the clinical question at hand, and its expanded access is costly.9
ART AND SCIENCE
Point-of-care resources do not solve all the complicated problems of patient care, and no single resource is ideal for all situations. A busy clinician has limited time to process the evolving literature to practice the best evidence-based medicine. Effective information access, quality of care provided, and the marginal time cost required create a complex calculus. Clinical decision-making remains an art and a science,25 but these technologies help define a new era in its pursuit.
Ultimately, a clinician’s choice needs to correlate with a provider’s resources and style. This article has detailed several options available on the market today. This is a quickly evolving area of products and services. Longer term, users might consider a tool’s preferred key features when evaluating any current or future resource in order to choose the right ones for their practice.
CASE REVISITED
Before we leave for the weekend, we need a plan for Ms. Connecticut. To find appropriate recommendations for our patient, we search several of our point-of-care resources: UpToDate and Dynamed. Both resources have correct information according to the Infectious Disease Society of America (IDSA) guidelines.
UpToDate has a monograph of approximately 2,000 words on Lyme disease, which is lengthy but adds to clinical-decision making skills for a learner thinking through the decision. This service also has a patient handout highlighting the recommendations. The topic was last updated in 2016, but states that it is current with literature through January 2017.
Dynamed has bulleted information that is quicker to digest, but essentially highlights the IDSA recommendations without the thought process behind them. It too, has patient resources with links to a variety of handouts from professional organizations such as the US Centers for Disease Control and Prevention. They last updated the topic January 31, 2017.
When searching for the topic on both sites, a clinician can see the breadth of information in each program. However, this is also a detractor. Searching for Lyme disease prophylaxis on Dynamed brought up related data (that doxycycline is not FDA-approved for prophylaxis), but not the primary information. Likewise, the search under UpToDate first brought us to the patient information. Both articles have helpful tables and links to associated topics.
My partner chose the UpToDate article, in part to review the topic with a medical student. However, I used Dynamed for its quick bulleted information, as I was on call that evening and needed to return to the hospital. We both came to the same conclusion, and Ms. Connecticut chose no prophylaxis even though her home is in an endemic area. She has done well.
- Worster A, Haynes RB. How do I find a point-of-care answer to my clinical question? CJEM 2012; 14:31–35.
- Jeffery R, Navarro T, Lokker C, Haynes RB, Wilczynski NL, Farjou G. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res 2012; 14:e175.
- ClinicalKey. Errors found in nine out of ten Wikipedia health entries. www.clinicalkey.com/info/blog/errors-in-wikipedia-health/. Accessed February 9, 2017.
- Hasty RT, Garbalosa RC, Barbato VA, et al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc 2014; 114:368–373.
- Mehta NB, Martin SA, Maypole J, Andrews R. Information management for clinicians. Cleve Clin J Med 2016; 83:589–595.
- Cook DA, Sorensen KJ, Hersh W, Berger RA, Wilkinson JM. Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One 2013; 8:e80318.
- Steinbrook R. Future directions in industry funding of continuing medical education. Arch Intern Med 2011; 171:257–258.
- Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999; 319:1618.
- Kim S, Noveck H, Galt J, Hogshire L, Willett L, O’Rourke K. Searching for answers to clinical questions using Google versus evidence-based summary resources: a randomized controlled crossover study. Acad Med 2014; 89:940–943.
- Ahmadi SF, Faghankhani M, Javanbakht A, et al. A comparison of answer retrieval through four evidence-based textbooks (ACP PIER, Essential Evidence Plus, First Consult, and UpToDate): a randomized controlled trial. Med Teach 2011; 33:724–730.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Shurtz S, Foster MJ. Developing and using a rubric for evaluating evidence-based medicine point-of-care tools. J Med Libr Assoc 2011; 99:247–254.
- Ketterman E, Besaw M. An evaluation of citation counts, search results, and frequency of updates in Dynamed and UpToDate. J Electron Res in Med Libr 2010; 7:273–280.
- Amber KT, Dhiman G, Goodman KW. Conflict of interest in online point-of-care clinical support websites. J Med Ethics 2014; 40:578–580.
- Montgomery K. How Doctors Think: Clinical Judgment and the Practice of Medicine. New York, NY: Oxford University Press; 2005.
- Groopman J. How Doctors Think. Boston, MA: Houghton Mifflin; 2008.
- Densen P. Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc 2010; 122:48–58.
- Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30:2310–2317.
- Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human-computer interaction. Int J Med Inform 2012; 81:232–243.
- Fraccaro P, Arguello Castelerio M, Ainsworth J, Buchan I. Adoption of clinical decision support in multimorbidity: a systematic review. JMIR Med Informatics 2015; 3:e4.
- McLeod W, Eidus R, Stewart EE. Clinical decision support: using technology to identify patients’ unmet needs. Fam Pract Manag 2012; 19:22–28.
- Colla CH. Swimming against the current—what might work to reduce low-value care? N Engl J Med 2014; 371:1280–1283.
- Cook DA, Sorensen KJ, Nishimura RA, Ommen SR, Lloyd FJ. A comprehensive information technology system to support physician learning at the point of care. Acad Med 2015; 90:33–39.
- Woolever DR. The art and science of clinical decision making. Fam Pract Manag 2008; 15:31–36.
It is friday afternoon on a sunny July day. The last patient of the day, Ms. Connecticut, is an active hiker who has had Lyme disease previously. She found a tick on her ankle yesterday. She successfully removed the tick but has not brought the tick with her to the appointment. She had been hiking several times over the last week and is not certain when the tick bite occurred. Her question for you centers on the role of antibiotic prophylaxis and Lyme disease prevention.
TECHNOLOGY: PROBLEM AND SOLUTION
Physicians need to keep up with an ever-increasing stream of information—new guidelines, new medications, and updates in medical literature.1 They have to do this while seeing more patients with more chronic problems in less time and while meeting reporting requirements for meaningful use or quality measures for accountable care organizations.
Though some of these challenges are due to technology, one solution is to use technology to our advantage. While researching information in textbooks won’t drain a phone battery, carrying a textbook around is not feasible, and many textbooks (including their electronic versions) contain information that is outdated before they go to print or that is quickly outdated thereafter.2 Further, even online textbooks are currently more dense than the online resources that we review here.
Different types of resources can help task-saturated healthcare providers stay aware of new information while delivering evidence-based care. These tools—online textbooks, decision guides embedded within electronic health record systems, or even a Google search—are termed “point-of-care” resources when used at the time of patient care for decision-making in the moment.
Not all of these resources are of high quality, with reliable factual information. Researchers estimate that up to 70% of clinicians may use Wikipedia to research medical questions, and a comparison of 10 Wikipedia articles vs peer-reviewed sources on the 10 most costly diseases found that 9 of the 10 Wikipedia articles had errors.3,4
In an earlier article,5 we advocated a proactive approach to managing information, highlighting ways to scan for new information and to develop habits of extracting useful information that can then be stored and easily recovered. To complement this strategy and weed out erroneous information, physicians need reliable sources of unbiased information to efficiently answer clinical questions at the point of care.1,6
Here, to help busy clinicians choose which point-of-care resources to use, we review several of the most popular ones, examining their ease of use, key elements, strengths, and weaknesses.
WHAT MAKES A RESOURCE GOOD?
Key features that make point-of-care tools effective include:
Ease of use, with standard formats, a summary for each topic, or both
Links to original articles and concise, capsular summaries and syntheses of the data
Continuing medical education (CME) credit. Tip: when searching, add “CME” to the search string on the browser to access resources that provide this.
Institutional and individual accounts. For clinicians who work for large organizations, point-of-care products may be paid for already, or reimbursement may be available for your subscription. If unsure, ask your director of information technology or library services.
Freedom from advertisements. Many Internet sources have advertisements that either run alongside the information you want to see or, more annoyingly, pop up and require an action to move forward. There is also continuing concern about the effect of industry support on content.7 While not all of the resources that we use regularly and that we review here are ad-free, avoiding programs with high ad content helps limit the possibility of bias and the time it takes to access information. Although advertisements do bring up a risk of bias, resources with a low-level ad content can limit bias while providing free or low-cost access.
Evidence, not expert opinion. Many resources have an “about” page that explains their philosophy and the source of their information. It is vital to be sure that point-of-care databases are providing facts based on evidence.8 This page also typically addresses how authors and editors are selected and whether expert opinion is used when randomized trials are lacking.
Ease of access. Many tools can be accessed not only on computers but also through apps for smartphones and tablets. Some electronic medical records have clinical decision tools embedded in them, with varying capabilities.
Disclosure of conflict of interest. As conflicts of interest can shade recommendations, information sources should clearly disclose financial relationships that could be perceived as conflicts of interest—for example, authors writing about medications sold by companies with whom they have a financial relationship.
NO SINGLE RESOURCE DOES EVERYTHING
There are many types of tools for finding evidence-based medical information. Different tools serve different purposes. Table 1 lists “toolbox essentials” for clinicians needing to answer clinical questions during patient care.
For example, when a question about the need for a bone mineral density measurement comes up, it is useful to be able to quickly compare guidelines from different professional societies on the National Guideline Clearing House. For another example, if a patient brings in a medication in an unlabeled bottle, a pill identifier app can tell you what it is. Clinicians who can use these resources appropriately will be at an advantage in being able to use information to provide better care to their patients.
To date, no point-of-care summary source has been shown to be superior in all categories, and use may be driven by ease of navigation, clinician preference, clinical question, or past success.9,10
Reviewed below are several applications that can be used as point-of-care resources (Table 2).
CLINICAL EVIDENCE
Clinical Evidence provides systematic reviews on medical topics. Founded in 1999 by the British Medical Journal, it is available in print as the Clinical Evidence Handbook and in online desktop and smartphone formats.
More than any other source we reviewed, Clinical Evidence addresses not only the evidence that exists, but also the data that do not exist to guide decisions. Compared with 9 other point-of-care resources, Clinical Evidence was found to have the highest quality of evidence.11
Strengths of Clinical Evidence
- Uncommonly transparent in terms of source of evidence or disclosing when there is a lack of evidence.
- Clearly lists the strength and relevance of the evidence.
- Personalization. Users can add notes to articles, save personal searches, and bookmark pages for easy access later.
- Navigability. Users can easily access systematic reviews, key points, retracted papers, or guidelines.
- Intuitive organization, with information categorized as research, education, news, or campaigns.
- New content daily: podcasts, articles, videos.
Weaknesses of Clinical Evidence
- Limited topics (eg, Lyme disease was not available)
- The limited content is a challenge when needing quick information at the point of care and may cause most clinicians to use another source unless looking for comparisons of interventions.
- Cost. Subscribing to the service “on demand,” ie, to look up a single specific topic, costs $36 for 48 hours of access; monthly access or a “season ticket” allows 30 days of viewing of all content for $53. At over $600/year, this is one of the most costly of the sources we reviewed.
- Marketing of Clinical Evidence to academic institutions that support the service for faculty may limit its appeal to other clinicians.
DYNAMED
Dynamed, a clinical reference created by a group of physicians, was previously owned by the American College of Physicians and known as Smart Medicine; it is now owned by EBSCO.12 Reviewers investigate the literature for a given topic and create pithy summaries for busy clinicians. A top feature in Dynamed is its links to full articles cited for best practices or evidence-based guidelines. The company describes their content as free of expert opinion, while being unbiased and evidence-based.
Dynamed uses a 7-step algorithm for searched topics that identifies articles, assesses clinical relevance, evaluates validity of outcomes, compiles the evidence from multiple articles, and then updates the final recommendations daily.
Dynamed Plus, the new upgraded version, updates searched topics several times a day. Dynamed may be the most frequently updated point-of-care resource, with the least risk of conflict of interest, but it offers limited topics drawn from evidence-based findings.11,13–15
With the rapid doubling of the medical literature, frequent updates allow clinicians to be most current with practice guidelines. This potentially affects quality of care for antibiotic use, vaccination, health promotion, and screening as well as newly approved medications.
Strengths of Dynamed
- Large collection of topics, critically appraised, written for primary care physicians, presented in bulleted format
- The most frequently updated database11,14,15
- Can integrate with major electronic health records (eg, Epic, Allscripts, NextGen, Cerner)
- Has an area devoted to new information that changes current practice
- Chosen topic grouped with related topics in the differential diagnosis after the initial search
- Easy-to-read outline for quick access to information such as billing, diagnosis, and references
- Medical calculators
- No advertisements
- Helpful embedded tools
- Icons to print or email the article
- An icon to create a “perma-link” to topics, searches, and browse categories
- Graded evidence with a link to the grading model used
- Links to primary articles
- Patient information handouts
- Alerts for updated information
- CME credit
- Special consideration and features for medical education
- The upgraded version Dynamed Plus contains Micromedex for a medication database, expanded graphics, semantic search, concise overview for each topic, and expanded content.
Weaknesses of Dynamed
- Although the topic list is large, it is only about one-third the size of UpToDate.
- A subscription for a physician costs $395 a year. Residents can sign up for about $150, and students for just under $100.
- CME is obtainable but cumbersome; one submits the CME credits through Tufts Healthcare, which requires a second sign-on to access and track.
- Drug and nondrug treatments for diseases cannot be separated.
- Useful calculators include decision trees for clinical decision-making, but there is no way to search them—one must waste time scrolling through the topics and specialties looking for desired information.
- Major shortcoming: there is no medication reference tool unless you upgrade to Dynamed Plus.
- The expanded graphics of Dynamed Plus are difficult to view on mobile applications within the articles (they are brought up more reliably when searching just for the image).
- The use of strict evidence-based methodology without expert opinion is a strength, but limits the collection of topics without randomized controlled trials, for which turning to expert opinion may be the only option.
EVIDENCE ESSENTIALS
Evidence Essentials is a point-of-care resource from Wiley that offers a variety of content types. The website lists 13,000 medical topics; however, they are not all summary reviews as discussed in the other products above. Subject matter is reviewed 3 times a year. Comprehensive reviews number just under 800 individual topics, with the remaining content consisting of Cochrane reviews, calculators, decision support tools, POEMs (Patient-Oriented Evidence that Matters), evidence-based medical guidelines, and dermatology images (1,000).
Evidence Essentials provides some unique content including a quick evaluation and management (E/M) code-finder and calculators not only for the typical medical equations, but also for history and physical examination likelihood ratios and pretest probabilities, which are practical and an excellent teaching aid. It also offers CME along with POEMs, e-mail alerts, and a listing of upcoming topics.
Strengths of Evidence Essentials
- Relatively inexpensive at $85 a year.
- High-functioning filter system to choose to search one or multiple databases.
- Related results are listed for aid in differential diagnosis, similar to Dynamed.
- Authors, editors, and date of last review are highly visible. As in UpToDate, relevant medical calculators appear on the page.
- The likelihood and odds ratio calculators are a huge plus for clinical decision-making and putting guidelines into practice.
- “Overall bottom line” highlights key points
- Grading of evidence per topic.
- Bulleted and tabbed information for quick access.
- Tabs for information on background, prevention, diagnosis, treatment, references, guidelines, and special populations.
Weaknesses of Evidence Essentials
- Limited number of topics with comprehensive reviews.
- While you can click on any drug name and link to a choice of two drug databases, this is not included in the subscription and requires a second account.
- The resources tabs had some broken links. In our clinical example, the tab contained several videos at the top that were not related, followed by a map and tables that were relevant to Lyme disease.
- Likewise, some of the guideline references were disappointing. For example, the guideline link for Lyme disease is for the US Department of Labor Occupational Safety and Health Administration rather than a professional society.
- For the provider wanting a narrative, this is more of a bare-bones text.
FIRST CONSULT
First Consult is Elsevier’s point-of-care clinical decision product contained within ClinicalKey.
Unlike UpToDate and Dynamed, in which authors and editors read original articles and summarize or synthesize information for the learner, First Consult is a “smart” search engine that will research a question, together with associated terms and key words. Filters such as full-text availability, journal articles, and patient education can be applied.
You may need to read about your topic in a textbook first, and then, if you are looking for treatment information, find an original article through First Consult. It is available in mobile and desktop formats, and the point-of-care product, First Consult, has an app that can be downloaded and used for free for the first 60 days.
Importantly, the First Consult portion of ClinicalKey with the summary topics was rated by Shurtz and Foster13 as least current of the products we are discussing in this article. On the other hand, it was the only product that had an embedded program to assist the user in making presentations by allowing drag and drop of images and automatic citing of sources. Kim et al report that First Consult is one of the resources providers prefer.9
Strengths of First Consult
- Lengthy free trial
- Ability to access original articles from a list vs lengthy narrative
- Access to journals and books published by Elsevier
- Powerful search engine that applies associated terms automatically
- Patient education is available in different languages and font size with the ability to add instructions and even a local branding
- Can integrate with electronic health record
- Can filter results by guideline, patient education, topic overviews
- Presentation assistance.
Weaknesses of First Consult
- Time-intensive. A provider needing quick advice on treatment for a medical condition has to guess if an article or textbook will have the most up-to-date and digestible information, whereas this has already been summarized in other products. For the busy clinician, this may be prohibitive.
- Search results are limited to Elsevier products, and major journals such as the New England Journal of Medicine are not available.
- Inconsistent platform functionality. The app version was somewhat “sticky” to use, as pages did not always load efficiently, and the menu bar navigation is not ideal.
- Expensive, especially given cheaper alternatives. For example, subscribing to the specialty of internal medicine or family medicine costs $499 and provides access to 8 journals and 11 books. Extended access costs $998 and offers full-text access to 23 books and 45 journals. The complete service has a total of 400 journals, 700 books, and 2,500 procedural videos.
MEDSCAPE
Medscape, owned by parent company WebMD, has long been a popular resource. The most recent versions are available for both for Android and iOS mobile platforms. The desktop and mobile apps claim to be designed for point-of-care use, and can be downloaded at no cost after registering as a Medscape user.
Medscape has some interesting features, including a handy pill identifier tool that is new to Medscape and perfect for the “I take one blue pill for my cholesterol” moments. The drug information tools and other features work well offline.
Medscape contains a well-presented drug database and interaction checker, as well as a growing collection of evidence-based articles and videos with links to references in Medline. From the point-of-care standpoint, Medscape also offers a number of decision-making algorithms and a continuously updated medical literature and health-related newsfeed. It contains in-app medical calculators, searchable directories for providers, hospitals, and pharmacies, and CME that can be earned on the website or from the application.
The main Medscape website contains pop-up advertisements, but the mobile app has fewer. Among the occasional frustrations, updates are relatively infrequent, the content is slow to load, and the phone app can be cumbersome. Of note, in one review,11 Medscape was found to have the lowest quality of evidence.
Strengths of Medscape
- Free with registration
- Medical calculator
- Drug interaction checker
- Pill identifier
- Evidence-based information covering about 4,000 conditions with links to references in Medline
- Ability to e-mail articles for sharing or future reference
- CME
- Unique database of hospitals, providers, and pharmacies to aid in referrals or locating other healthcare professionals
- Algorithms for decision-making
- Images and videos for procedural review and learning
- Option for downloading certain databases for offline use
- Medical news helps you keep up with what patients are watching and reading.
Weaknesses of Medscape
- Advertisements (many of them pop-up)
- The content is updated less frequently than other products listed in this article
- The smartphone app can run slowly
- Quality of reviews may be a concern.
UpToDate
UpToDate (Wolters Kluwer) is used widely by medical students, residents, and fellows as well as practicing providers. It contains narrative reviews of topics written by respected experts directed at both clinicians and clinical staff. In hopes of appealing to many markets, it offers different subscription types so you can customize your choices with add-on features (UpToDate Desktop and UpToDate MobileComplete allow downloading of all content to be accessed offline), different service packages (1-, 2-, and 3-year subscriptions), and the traditional base product that provides online access.
Of the products we reviewed, UpToDate has the largest selection of medical topics, approaching 10,000.14 In some studies,10,15 it also had the fastest retrieval time for searches. It uses evidence-based graded recommendations that are updated regularly.
Some have lamented that there is too much information per topic.9 In response to early reviews, Wolters Kluwer has made significant changes in the platform and greatly improved the search engine. UpToDate has expanded to include CME and patient information, trying to become that Holy Grail of websites—a one-stop experience. For the lucky few, UpToDate integrates into some electronic health records and provides a relatively seamless experience at the point of care.
Strengths of UpToDate
- One-stop shopping for information, resources, and CME
- Patient information is easy to read and accessible from the same screen
- The largest repository of medical subject matter
- Ability to cull out only pediatric or adult topics
- Searching available within a medical topic
- Tabs for quick access
- The What’s New feature allows access to practice-changing medical updates
- Medical calculators
- Drug interactions
- CME is is tracked in the system, allowing for CME credit information for hospital privileges and board certification
- Flexibility of access: can use online or download content to mobile/desktop device (the online version is easy to use, although robust wireless reception is needed; offices with slow Internet benefit from the offline feature)
- Electronic health record integration is possible with the most popular systems, such as Epic, eClinicalWorks, NextGen, and Allscripts
- Patient education and medication interaction features embedded in the electronic health record; produced in collaboration with Lexicomp
- Integrated drug database
- Alerts for updates
- References have links to full-text articles
- The date of last update is easily found for verifying information accuracy
- May be provided free for clinicians who are a part of a university or large health system.
Weaknesses of UpToDate
- Articles can be lengthy, which is both a strength and a weakness. Searches can retrieve too much information.9 High volume of text can frustrate the user trying to find bulleted, easy-to-read facts. However, for the person looking for a narrative summary, the content is organized as narrative paragraphs with appropriate headers in the left margin, and the search function is robust and powerful.
- Each topic has a “Summary of Recommendations,” but answers here often require linking back to the main text.
- Patient information is sometimes at a high literacy level.
- Costs more than Dynamed. A 1-year subscription is $499 for a physician, but you have the option of paying $53 for a 30-day recurring subscription. Residents, fellows, and students can pay $199 for 1 year or $19 for a 30-day recurring subscription.
- The requirement to download means that users need to keep their version updated on all of their computers—in each of their examination rooms, for example.
- Concerns about conflict of interest arise because authors and editors may maintain financial relationships with companies that produce medications discussed in the articles they have written.
BUILDING YOUR OWN PERSONAL ONLINE REPOSITORY
Our previous article5 reviewed how to store information using tools such as Evernote and Diigo that allow information viewed on a web page to be exported to any online repository. This can be done using extensions for a web browser or by sending the information to a custom e-mail account for these services.
For information that a provider knows he or she will need repeatedly, storage in one system is actually the easiest method. Such a system can then incorporate key information from the summary tools we have reviewed here. The ideal “electronic filing cabinet” should have several features such as a the capability to label articles by topic, to separate or sort as you see fit, and a search function to find information quickly—making it a personalized and effective point-of-care resource.
STAYING AFLOAT
Clinicians make many decisions every day. In fact, the release of How Doctors Think (both publications) has led to increased research into how clinical decisions and diagnoses are formed.16,17
With the medical literature expected to double every 73 days by 2020,18 there is an ever-widening ocean of information to sift through. With this onslaught, clinicians can no longer remain fully current. Instead, refining skills in accessing, sorting, and interpreting accurate scientific evidence efficiently is crucial to time spent actually caring for patients and coordinating their care.
Guidelines, algorithms, and comprehensive databases can aid clinicians in all aspects of care, from generating more complete differential diagnoses to managing disease-specific treatment. Individuals can first think about and list the qualities of a tool that are most important to them (eg, breadth of topics, frequency of updates, integration within their electronic health record, and cost) before focusing on a few applications or websites that meet those goals. With practice, point-of-care knowledge can become part of the everyday visit.
Effective integration into electronic health records will require design input from front-line clinicians. Otherwise, systems are prone to add too much “support” and overly rely on orthodox metrics and guidelines, resulting in alarm fatigue and frustration rather than facilitation.19–23
OUR CONCLUSIONS
Comprehensive point-of-care resources can play a significant role in helping busy clinicians provide best evidence-based care to their patients. Embedded clinical decision guides within an electronic health record are ideal, but low topic coverage has limited the usefulness of these systems.24 Here are our conclusions:
Medscape, ePocrates, and Wikipedia are probably the most popular free resources. Dynamed has offered free subscriptions to Wikipedia’s top health editors with the hopes of correcting factual errors. Medscape has excellent features but is supported by sponsored content, which raises a concern about bias and potential time-consuming distractions.
Dynamed and UpToDate have both been reported to answer more questions than other sources.12
UpToDate has the largest repository, with each topic curated by an expert or experts in that subject. This content can be dense and difficult to scan quickly at the point of care, but this is balanced by the ability to search within a medical topic, which has given it the fastest retrieval time.15 It does, however, allow authors and editors to maintain financial relationships with companies that produce medications discussed in the article.22
Dynamed has the advantage in frequency of updates, clearest conflict-of-interest policy, and the least amount of conflict of interest. Its topic list is not as extensive as UpToDate’s due to the limitation of using only evidence-based medicine without expert opinion.
First Consult has high user satisfaction, but as a point-of-care resource it can be time-consuming to find the best source for the clinical question at hand, and its expanded access is costly.9
ART AND SCIENCE
Point-of-care resources do not solve all the complicated problems of patient care, and no single resource is ideal for all situations. A busy clinician has limited time to process the evolving literature to practice the best evidence-based medicine. Effective information access, quality of care provided, and the marginal time cost required create a complex calculus. Clinical decision-making remains an art and a science,25 but these technologies help define a new era in its pursuit.
Ultimately, a clinician’s choice needs to correlate with a provider’s resources and style. This article has detailed several options available on the market today. This is a quickly evolving area of products and services. Longer term, users might consider a tool’s preferred key features when evaluating any current or future resource in order to choose the right ones for their practice.
CASE REVISITED
Before we leave for the weekend, we need a plan for Ms. Connecticut. To find appropriate recommendations for our patient, we search several of our point-of-care resources: UpToDate and Dynamed. Both resources have correct information according to the Infectious Disease Society of America (IDSA) guidelines.
UpToDate has a monograph of approximately 2,000 words on Lyme disease, which is lengthy but adds to clinical-decision making skills for a learner thinking through the decision. This service also has a patient handout highlighting the recommendations. The topic was last updated in 2016, but states that it is current with literature through January 2017.
Dynamed has bulleted information that is quicker to digest, but essentially highlights the IDSA recommendations without the thought process behind them. It too, has patient resources with links to a variety of handouts from professional organizations such as the US Centers for Disease Control and Prevention. They last updated the topic January 31, 2017.
When searching for the topic on both sites, a clinician can see the breadth of information in each program. However, this is also a detractor. Searching for Lyme disease prophylaxis on Dynamed brought up related data (that doxycycline is not FDA-approved for prophylaxis), but not the primary information. Likewise, the search under UpToDate first brought us to the patient information. Both articles have helpful tables and links to associated topics.
My partner chose the UpToDate article, in part to review the topic with a medical student. However, I used Dynamed for its quick bulleted information, as I was on call that evening and needed to return to the hospital. We both came to the same conclusion, and Ms. Connecticut chose no prophylaxis even though her home is in an endemic area. She has done well.
It is friday afternoon on a sunny July day. The last patient of the day, Ms. Connecticut, is an active hiker who has had Lyme disease previously. She found a tick on her ankle yesterday. She successfully removed the tick but has not brought the tick with her to the appointment. She had been hiking several times over the last week and is not certain when the tick bite occurred. Her question for you centers on the role of antibiotic prophylaxis and Lyme disease prevention.
TECHNOLOGY: PROBLEM AND SOLUTION
Physicians need to keep up with an ever-increasing stream of information—new guidelines, new medications, and updates in medical literature.1 They have to do this while seeing more patients with more chronic problems in less time and while meeting reporting requirements for meaningful use or quality measures for accountable care organizations.
Though some of these challenges are due to technology, one solution is to use technology to our advantage. While researching information in textbooks won’t drain a phone battery, carrying a textbook around is not feasible, and many textbooks (including their electronic versions) contain information that is outdated before they go to print or that is quickly outdated thereafter.2 Further, even online textbooks are currently more dense than the online resources that we review here.
Different types of resources can help task-saturated healthcare providers stay aware of new information while delivering evidence-based care. These tools—online textbooks, decision guides embedded within electronic health record systems, or even a Google search—are termed “point-of-care” resources when used at the time of patient care for decision-making in the moment.
Not all of these resources are of high quality, with reliable factual information. Researchers estimate that up to 70% of clinicians may use Wikipedia to research medical questions, and a comparison of 10 Wikipedia articles vs peer-reviewed sources on the 10 most costly diseases found that 9 of the 10 Wikipedia articles had errors.3,4
In an earlier article,5 we advocated a proactive approach to managing information, highlighting ways to scan for new information and to develop habits of extracting useful information that can then be stored and easily recovered. To complement this strategy and weed out erroneous information, physicians need reliable sources of unbiased information to efficiently answer clinical questions at the point of care.1,6
Here, to help busy clinicians choose which point-of-care resources to use, we review several of the most popular ones, examining their ease of use, key elements, strengths, and weaknesses.
WHAT MAKES A RESOURCE GOOD?
Key features that make point-of-care tools effective include:
Ease of use, with standard formats, a summary for each topic, or both
Links to original articles and concise, capsular summaries and syntheses of the data
Continuing medical education (CME) credit. Tip: when searching, add “CME” to the search string on the browser to access resources that provide this.
Institutional and individual accounts. For clinicians who work for large organizations, point-of-care products may be paid for already, or reimbursement may be available for your subscription. If unsure, ask your director of information technology or library services.
Freedom from advertisements. Many Internet sources have advertisements that either run alongside the information you want to see or, more annoyingly, pop up and require an action to move forward. There is also continuing concern about the effect of industry support on content.7 While not all of the resources that we use regularly and that we review here are ad-free, avoiding programs with high ad content helps limit the possibility of bias and the time it takes to access information. Although advertisements do bring up a risk of bias, resources with a low-level ad content can limit bias while providing free or low-cost access.
Evidence, not expert opinion. Many resources have an “about” page that explains their philosophy and the source of their information. It is vital to be sure that point-of-care databases are providing facts based on evidence.8 This page also typically addresses how authors and editors are selected and whether expert opinion is used when randomized trials are lacking.
Ease of access. Many tools can be accessed not only on computers but also through apps for smartphones and tablets. Some electronic medical records have clinical decision tools embedded in them, with varying capabilities.
Disclosure of conflict of interest. As conflicts of interest can shade recommendations, information sources should clearly disclose financial relationships that could be perceived as conflicts of interest—for example, authors writing about medications sold by companies with whom they have a financial relationship.
NO SINGLE RESOURCE DOES EVERYTHING
There are many types of tools for finding evidence-based medical information. Different tools serve different purposes. Table 1 lists “toolbox essentials” for clinicians needing to answer clinical questions during patient care.
For example, when a question about the need for a bone mineral density measurement comes up, it is useful to be able to quickly compare guidelines from different professional societies on the National Guideline Clearing House. For another example, if a patient brings in a medication in an unlabeled bottle, a pill identifier app can tell you what it is. Clinicians who can use these resources appropriately will be at an advantage in being able to use information to provide better care to their patients.
To date, no point-of-care summary source has been shown to be superior in all categories, and use may be driven by ease of navigation, clinician preference, clinical question, or past success.9,10
Reviewed below are several applications that can be used as point-of-care resources (Table 2).
CLINICAL EVIDENCE
Clinical Evidence provides systematic reviews on medical topics. Founded in 1999 by the British Medical Journal, it is available in print as the Clinical Evidence Handbook and in online desktop and smartphone formats.
More than any other source we reviewed, Clinical Evidence addresses not only the evidence that exists, but also the data that do not exist to guide decisions. Compared with 9 other point-of-care resources, Clinical Evidence was found to have the highest quality of evidence.11
Strengths of Clinical Evidence
- Uncommonly transparent in terms of source of evidence or disclosing when there is a lack of evidence.
- Clearly lists the strength and relevance of the evidence.
- Personalization. Users can add notes to articles, save personal searches, and bookmark pages for easy access later.
- Navigability. Users can easily access systematic reviews, key points, retracted papers, or guidelines.
- Intuitive organization, with information categorized as research, education, news, or campaigns.
- New content daily: podcasts, articles, videos.
Weaknesses of Clinical Evidence
- Limited topics (eg, Lyme disease was not available)
- The limited content is a challenge when needing quick information at the point of care and may cause most clinicians to use another source unless looking for comparisons of interventions.
- Cost. Subscribing to the service “on demand,” ie, to look up a single specific topic, costs $36 for 48 hours of access; monthly access or a “season ticket” allows 30 days of viewing of all content for $53. At over $600/year, this is one of the most costly of the sources we reviewed.
- Marketing of Clinical Evidence to academic institutions that support the service for faculty may limit its appeal to other clinicians.
DYNAMED
Dynamed, a clinical reference created by a group of physicians, was previously owned by the American College of Physicians and known as Smart Medicine; it is now owned by EBSCO.12 Reviewers investigate the literature for a given topic and create pithy summaries for busy clinicians. A top feature in Dynamed is its links to full articles cited for best practices or evidence-based guidelines. The company describes their content as free of expert opinion, while being unbiased and evidence-based.
Dynamed uses a 7-step algorithm for searched topics that identifies articles, assesses clinical relevance, evaluates validity of outcomes, compiles the evidence from multiple articles, and then updates the final recommendations daily.
Dynamed Plus, the new upgraded version, updates searched topics several times a day. Dynamed may be the most frequently updated point-of-care resource, with the least risk of conflict of interest, but it offers limited topics drawn from evidence-based findings.11,13–15
With the rapid doubling of the medical literature, frequent updates allow clinicians to be most current with practice guidelines. This potentially affects quality of care for antibiotic use, vaccination, health promotion, and screening as well as newly approved medications.
Strengths of Dynamed
- Large collection of topics, critically appraised, written for primary care physicians, presented in bulleted format
- The most frequently updated database11,14,15
- Can integrate with major electronic health records (eg, Epic, Allscripts, NextGen, Cerner)
- Has an area devoted to new information that changes current practice
- Chosen topic grouped with related topics in the differential diagnosis after the initial search
- Easy-to-read outline for quick access to information such as billing, diagnosis, and references
- Medical calculators
- No advertisements
- Helpful embedded tools
- Icons to print or email the article
- An icon to create a “perma-link” to topics, searches, and browse categories
- Graded evidence with a link to the grading model used
- Links to primary articles
- Patient information handouts
- Alerts for updated information
- CME credit
- Special consideration and features for medical education
- The upgraded version Dynamed Plus contains Micromedex for a medication database, expanded graphics, semantic search, concise overview for each topic, and expanded content.
Weaknesses of Dynamed
- Although the topic list is large, it is only about one-third the size of UpToDate.
- A subscription for a physician costs $395 a year. Residents can sign up for about $150, and students for just under $100.
- CME is obtainable but cumbersome; one submits the CME credits through Tufts Healthcare, which requires a second sign-on to access and track.
- Drug and nondrug treatments for diseases cannot be separated.
- Useful calculators include decision trees for clinical decision-making, but there is no way to search them—one must waste time scrolling through the topics and specialties looking for desired information.
- Major shortcoming: there is no medication reference tool unless you upgrade to Dynamed Plus.
- The expanded graphics of Dynamed Plus are difficult to view on mobile applications within the articles (they are brought up more reliably when searching just for the image).
- The use of strict evidence-based methodology without expert opinion is a strength, but limits the collection of topics without randomized controlled trials, for which turning to expert opinion may be the only option.
EVIDENCE ESSENTIALS
Evidence Essentials is a point-of-care resource from Wiley that offers a variety of content types. The website lists 13,000 medical topics; however, they are not all summary reviews as discussed in the other products above. Subject matter is reviewed 3 times a year. Comprehensive reviews number just under 800 individual topics, with the remaining content consisting of Cochrane reviews, calculators, decision support tools, POEMs (Patient-Oriented Evidence that Matters), evidence-based medical guidelines, and dermatology images (1,000).
Evidence Essentials provides some unique content including a quick evaluation and management (E/M) code-finder and calculators not only for the typical medical equations, but also for history and physical examination likelihood ratios and pretest probabilities, which are practical and an excellent teaching aid. It also offers CME along with POEMs, e-mail alerts, and a listing of upcoming topics.
Strengths of Evidence Essentials
- Relatively inexpensive at $85 a year.
- High-functioning filter system to choose to search one or multiple databases.
- Related results are listed for aid in differential diagnosis, similar to Dynamed.
- Authors, editors, and date of last review are highly visible. As in UpToDate, relevant medical calculators appear on the page.
- The likelihood and odds ratio calculators are a huge plus for clinical decision-making and putting guidelines into practice.
- “Overall bottom line” highlights key points
- Grading of evidence per topic.
- Bulleted and tabbed information for quick access.
- Tabs for information on background, prevention, diagnosis, treatment, references, guidelines, and special populations.
Weaknesses of Evidence Essentials
- Limited number of topics with comprehensive reviews.
- While you can click on any drug name and link to a choice of two drug databases, this is not included in the subscription and requires a second account.
- The resources tabs had some broken links. In our clinical example, the tab contained several videos at the top that were not related, followed by a map and tables that were relevant to Lyme disease.
- Likewise, some of the guideline references were disappointing. For example, the guideline link for Lyme disease is for the US Department of Labor Occupational Safety and Health Administration rather than a professional society.
- For the provider wanting a narrative, this is more of a bare-bones text.
FIRST CONSULT
First Consult is Elsevier’s point-of-care clinical decision product contained within ClinicalKey.
Unlike UpToDate and Dynamed, in which authors and editors read original articles and summarize or synthesize information for the learner, First Consult is a “smart” search engine that will research a question, together with associated terms and key words. Filters such as full-text availability, journal articles, and patient education can be applied.
You may need to read about your topic in a textbook first, and then, if you are looking for treatment information, find an original article through First Consult. It is available in mobile and desktop formats, and the point-of-care product, First Consult, has an app that can be downloaded and used for free for the first 60 days.
Importantly, the First Consult portion of ClinicalKey with the summary topics was rated by Shurtz and Foster13 as least current of the products we are discussing in this article. On the other hand, it was the only product that had an embedded program to assist the user in making presentations by allowing drag and drop of images and automatic citing of sources. Kim et al report that First Consult is one of the resources providers prefer.9
Strengths of First Consult
- Lengthy free trial
- Ability to access original articles from a list vs lengthy narrative
- Access to journals and books published by Elsevier
- Powerful search engine that applies associated terms automatically
- Patient education is available in different languages and font size with the ability to add instructions and even a local branding
- Can integrate with electronic health record
- Can filter results by guideline, patient education, topic overviews
- Presentation assistance.
Weaknesses of First Consult
- Time-intensive. A provider needing quick advice on treatment for a medical condition has to guess if an article or textbook will have the most up-to-date and digestible information, whereas this has already been summarized in other products. For the busy clinician, this may be prohibitive.
- Search results are limited to Elsevier products, and major journals such as the New England Journal of Medicine are not available.
- Inconsistent platform functionality. The app version was somewhat “sticky” to use, as pages did not always load efficiently, and the menu bar navigation is not ideal.
- Expensive, especially given cheaper alternatives. For example, subscribing to the specialty of internal medicine or family medicine costs $499 and provides access to 8 journals and 11 books. Extended access costs $998 and offers full-text access to 23 books and 45 journals. The complete service has a total of 400 journals, 700 books, and 2,500 procedural videos.
MEDSCAPE
Medscape, owned by parent company WebMD, has long been a popular resource. The most recent versions are available for both for Android and iOS mobile platforms. The desktop and mobile apps claim to be designed for point-of-care use, and can be downloaded at no cost after registering as a Medscape user.
Medscape has some interesting features, including a handy pill identifier tool that is new to Medscape and perfect for the “I take one blue pill for my cholesterol” moments. The drug information tools and other features work well offline.
Medscape contains a well-presented drug database and interaction checker, as well as a growing collection of evidence-based articles and videos with links to references in Medline. From the point-of-care standpoint, Medscape also offers a number of decision-making algorithms and a continuously updated medical literature and health-related newsfeed. It contains in-app medical calculators, searchable directories for providers, hospitals, and pharmacies, and CME that can be earned on the website or from the application.
The main Medscape website contains pop-up advertisements, but the mobile app has fewer. Among the occasional frustrations, updates are relatively infrequent, the content is slow to load, and the phone app can be cumbersome. Of note, in one review,11 Medscape was found to have the lowest quality of evidence.
Strengths of Medscape
- Free with registration
- Medical calculator
- Drug interaction checker
- Pill identifier
- Evidence-based information covering about 4,000 conditions with links to references in Medline
- Ability to e-mail articles for sharing or future reference
- CME
- Unique database of hospitals, providers, and pharmacies to aid in referrals or locating other healthcare professionals
- Algorithms for decision-making
- Images and videos for procedural review and learning
- Option for downloading certain databases for offline use
- Medical news helps you keep up with what patients are watching and reading.
Weaknesses of Medscape
- Advertisements (many of them pop-up)
- The content is updated less frequently than other products listed in this article
- The smartphone app can run slowly
- Quality of reviews may be a concern.
UpToDate
UpToDate (Wolters Kluwer) is used widely by medical students, residents, and fellows as well as practicing providers. It contains narrative reviews of topics written by respected experts directed at both clinicians and clinical staff. In hopes of appealing to many markets, it offers different subscription types so you can customize your choices with add-on features (UpToDate Desktop and UpToDate MobileComplete allow downloading of all content to be accessed offline), different service packages (1-, 2-, and 3-year subscriptions), and the traditional base product that provides online access.
Of the products we reviewed, UpToDate has the largest selection of medical topics, approaching 10,000.14 In some studies,10,15 it also had the fastest retrieval time for searches. It uses evidence-based graded recommendations that are updated regularly.
Some have lamented that there is too much information per topic.9 In response to early reviews, Wolters Kluwer has made significant changes in the platform and greatly improved the search engine. UpToDate has expanded to include CME and patient information, trying to become that Holy Grail of websites—a one-stop experience. For the lucky few, UpToDate integrates into some electronic health records and provides a relatively seamless experience at the point of care.
Strengths of UpToDate
- One-stop shopping for information, resources, and CME
- Patient information is easy to read and accessible from the same screen
- The largest repository of medical subject matter
- Ability to cull out only pediatric or adult topics
- Searching available within a medical topic
- Tabs for quick access
- The What’s New feature allows access to practice-changing medical updates
- Medical calculators
- Drug interactions
- CME is is tracked in the system, allowing for CME credit information for hospital privileges and board certification
- Flexibility of access: can use online or download content to mobile/desktop device (the online version is easy to use, although robust wireless reception is needed; offices with slow Internet benefit from the offline feature)
- Electronic health record integration is possible with the most popular systems, such as Epic, eClinicalWorks, NextGen, and Allscripts
- Patient education and medication interaction features embedded in the electronic health record; produced in collaboration with Lexicomp
- Integrated drug database
- Alerts for updates
- References have links to full-text articles
- The date of last update is easily found for verifying information accuracy
- May be provided free for clinicians who are a part of a university or large health system.
Weaknesses of UpToDate
- Articles can be lengthy, which is both a strength and a weakness. Searches can retrieve too much information.9 High volume of text can frustrate the user trying to find bulleted, easy-to-read facts. However, for the person looking for a narrative summary, the content is organized as narrative paragraphs with appropriate headers in the left margin, and the search function is robust and powerful.
- Each topic has a “Summary of Recommendations,” but answers here often require linking back to the main text.
- Patient information is sometimes at a high literacy level.
- Costs more than Dynamed. A 1-year subscription is $499 for a physician, but you have the option of paying $53 for a 30-day recurring subscription. Residents, fellows, and students can pay $199 for 1 year or $19 for a 30-day recurring subscription.
- The requirement to download means that users need to keep their version updated on all of their computers—in each of their examination rooms, for example.
- Concerns about conflict of interest arise because authors and editors may maintain financial relationships with companies that produce medications discussed in the articles they have written.
BUILDING YOUR OWN PERSONAL ONLINE REPOSITORY
Our previous article5 reviewed how to store information using tools such as Evernote and Diigo that allow information viewed on a web page to be exported to any online repository. This can be done using extensions for a web browser or by sending the information to a custom e-mail account for these services.
For information that a provider knows he or she will need repeatedly, storage in one system is actually the easiest method. Such a system can then incorporate key information from the summary tools we have reviewed here. The ideal “electronic filing cabinet” should have several features such as a the capability to label articles by topic, to separate or sort as you see fit, and a search function to find information quickly—making it a personalized and effective point-of-care resource.
STAYING AFLOAT
Clinicians make many decisions every day. In fact, the release of How Doctors Think (both publications) has led to increased research into how clinical decisions and diagnoses are formed.16,17
With the medical literature expected to double every 73 days by 2020,18 there is an ever-widening ocean of information to sift through. With this onslaught, clinicians can no longer remain fully current. Instead, refining skills in accessing, sorting, and interpreting accurate scientific evidence efficiently is crucial to time spent actually caring for patients and coordinating their care.
Guidelines, algorithms, and comprehensive databases can aid clinicians in all aspects of care, from generating more complete differential diagnoses to managing disease-specific treatment. Individuals can first think about and list the qualities of a tool that are most important to them (eg, breadth of topics, frequency of updates, integration within their electronic health record, and cost) before focusing on a few applications or websites that meet those goals. With practice, point-of-care knowledge can become part of the everyday visit.
Effective integration into electronic health records will require design input from front-line clinicians. Otherwise, systems are prone to add too much “support” and overly rely on orthodox metrics and guidelines, resulting in alarm fatigue and frustration rather than facilitation.19–23
OUR CONCLUSIONS
Comprehensive point-of-care resources can play a significant role in helping busy clinicians provide best evidence-based care to their patients. Embedded clinical decision guides within an electronic health record are ideal, but low topic coverage has limited the usefulness of these systems.24 Here are our conclusions:
Medscape, ePocrates, and Wikipedia are probably the most popular free resources. Dynamed has offered free subscriptions to Wikipedia’s top health editors with the hopes of correcting factual errors. Medscape has excellent features but is supported by sponsored content, which raises a concern about bias and potential time-consuming distractions.
Dynamed and UpToDate have both been reported to answer more questions than other sources.12
UpToDate has the largest repository, with each topic curated by an expert or experts in that subject. This content can be dense and difficult to scan quickly at the point of care, but this is balanced by the ability to search within a medical topic, which has given it the fastest retrieval time.15 It does, however, allow authors and editors to maintain financial relationships with companies that produce medications discussed in the article.22
Dynamed has the advantage in frequency of updates, clearest conflict-of-interest policy, and the least amount of conflict of interest. Its topic list is not as extensive as UpToDate’s due to the limitation of using only evidence-based medicine without expert opinion.
First Consult has high user satisfaction, but as a point-of-care resource it can be time-consuming to find the best source for the clinical question at hand, and its expanded access is costly.9
ART AND SCIENCE
Point-of-care resources do not solve all the complicated problems of patient care, and no single resource is ideal for all situations. A busy clinician has limited time to process the evolving literature to practice the best evidence-based medicine. Effective information access, quality of care provided, and the marginal time cost required create a complex calculus. Clinical decision-making remains an art and a science,25 but these technologies help define a new era in its pursuit.
Ultimately, a clinician’s choice needs to correlate with a provider’s resources and style. This article has detailed several options available on the market today. This is a quickly evolving area of products and services. Longer term, users might consider a tool’s preferred key features when evaluating any current or future resource in order to choose the right ones for their practice.
CASE REVISITED
Before we leave for the weekend, we need a plan for Ms. Connecticut. To find appropriate recommendations for our patient, we search several of our point-of-care resources: UpToDate and Dynamed. Both resources have correct information according to the Infectious Disease Society of America (IDSA) guidelines.
UpToDate has a monograph of approximately 2,000 words on Lyme disease, which is lengthy but adds to clinical-decision making skills for a learner thinking through the decision. This service also has a patient handout highlighting the recommendations. The topic was last updated in 2016, but states that it is current with literature through January 2017.
Dynamed has bulleted information that is quicker to digest, but essentially highlights the IDSA recommendations without the thought process behind them. It too, has patient resources with links to a variety of handouts from professional organizations such as the US Centers for Disease Control and Prevention. They last updated the topic January 31, 2017.
When searching for the topic on both sites, a clinician can see the breadth of information in each program. However, this is also a detractor. Searching for Lyme disease prophylaxis on Dynamed brought up related data (that doxycycline is not FDA-approved for prophylaxis), but not the primary information. Likewise, the search under UpToDate first brought us to the patient information. Both articles have helpful tables and links to associated topics.
My partner chose the UpToDate article, in part to review the topic with a medical student. However, I used Dynamed for its quick bulleted information, as I was on call that evening and needed to return to the hospital. We both came to the same conclusion, and Ms. Connecticut chose no prophylaxis even though her home is in an endemic area. She has done well.
- Worster A, Haynes RB. How do I find a point-of-care answer to my clinical question? CJEM 2012; 14:31–35.
- Jeffery R, Navarro T, Lokker C, Haynes RB, Wilczynski NL, Farjou G. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res 2012; 14:e175.
- ClinicalKey. Errors found in nine out of ten Wikipedia health entries. www.clinicalkey.com/info/blog/errors-in-wikipedia-health/. Accessed February 9, 2017.
- Hasty RT, Garbalosa RC, Barbato VA, et al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc 2014; 114:368–373.
- Mehta NB, Martin SA, Maypole J, Andrews R. Information management for clinicians. Cleve Clin J Med 2016; 83:589–595.
- Cook DA, Sorensen KJ, Hersh W, Berger RA, Wilkinson JM. Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One 2013; 8:e80318.
- Steinbrook R. Future directions in industry funding of continuing medical education. Arch Intern Med 2011; 171:257–258.
- Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999; 319:1618.
- Kim S, Noveck H, Galt J, Hogshire L, Willett L, O’Rourke K. Searching for answers to clinical questions using Google versus evidence-based summary resources: a randomized controlled crossover study. Acad Med 2014; 89:940–943.
- Ahmadi SF, Faghankhani M, Javanbakht A, et al. A comparison of answer retrieval through four evidence-based textbooks (ACP PIER, Essential Evidence Plus, First Consult, and UpToDate): a randomized controlled trial. Med Teach 2011; 33:724–730.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Shurtz S, Foster MJ. Developing and using a rubric for evaluating evidence-based medicine point-of-care tools. J Med Libr Assoc 2011; 99:247–254.
- Ketterman E, Besaw M. An evaluation of citation counts, search results, and frequency of updates in Dynamed and UpToDate. J Electron Res in Med Libr 2010; 7:273–280.
- Amber KT, Dhiman G, Goodman KW. Conflict of interest in online point-of-care clinical support websites. J Med Ethics 2014; 40:578–580.
- Montgomery K. How Doctors Think: Clinical Judgment and the Practice of Medicine. New York, NY: Oxford University Press; 2005.
- Groopman J. How Doctors Think. Boston, MA: Houghton Mifflin; 2008.
- Densen P. Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc 2010; 122:48–58.
- Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30:2310–2317.
- Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human-computer interaction. Int J Med Inform 2012; 81:232–243.
- Fraccaro P, Arguello Castelerio M, Ainsworth J, Buchan I. Adoption of clinical decision support in multimorbidity: a systematic review. JMIR Med Informatics 2015; 3:e4.
- McLeod W, Eidus R, Stewart EE. Clinical decision support: using technology to identify patients’ unmet needs. Fam Pract Manag 2012; 19:22–28.
- Colla CH. Swimming against the current—what might work to reduce low-value care? N Engl J Med 2014; 371:1280–1283.
- Cook DA, Sorensen KJ, Nishimura RA, Ommen SR, Lloyd FJ. A comprehensive information technology system to support physician learning at the point of care. Acad Med 2015; 90:33–39.
- Woolever DR. The art and science of clinical decision making. Fam Pract Manag 2008; 15:31–36.
- Worster A, Haynes RB. How do I find a point-of-care answer to my clinical question? CJEM 2012; 14:31–35.
- Jeffery R, Navarro T, Lokker C, Haynes RB, Wilczynski NL, Farjou G. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res 2012; 14:e175.
- ClinicalKey. Errors found in nine out of ten Wikipedia health entries. www.clinicalkey.com/info/blog/errors-in-wikipedia-health/. Accessed February 9, 2017.
- Hasty RT, Garbalosa RC, Barbato VA, et al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc 2014; 114:368–373.
- Mehta NB, Martin SA, Maypole J, Andrews R. Information management for clinicians. Cleve Clin J Med 2016; 83:589–595.
- Cook DA, Sorensen KJ, Hersh W, Berger RA, Wilkinson JM. Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One 2013; 8:e80318.
- Steinbrook R. Future directions in industry funding of continuing medical education. Arch Intern Med 2011; 171:257–258.
- Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999; 319:1618.
- Kim S, Noveck H, Galt J, Hogshire L, Willett L, O’Rourke K. Searching for answers to clinical questions using Google versus evidence-based summary resources: a randomized controlled crossover study. Acad Med 2014; 89:940–943.
- Ahmadi SF, Faghankhani M, Javanbakht A, et al. A comparison of answer retrieval through four evidence-based textbooks (ACP PIER, Essential Evidence Plus, First Consult, and UpToDate): a randomized controlled trial. Med Teach 2011; 33:724–730.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol 2012; 65:1289–1295.
- Shurtz S, Foster MJ. Developing and using a rubric for evaluating evidence-based medicine point-of-care tools. J Med Libr Assoc 2011; 99:247–254.
- Ketterman E, Besaw M. An evaluation of citation counts, search results, and frequency of updates in Dynamed and UpToDate. J Electron Res in Med Libr 2010; 7:273–280.
- Amber KT, Dhiman G, Goodman KW. Conflict of interest in online point-of-care clinical support websites. J Med Ethics 2014; 40:578–580.
- Montgomery K. How Doctors Think: Clinical Judgment and the Practice of Medicine. New York, NY: Oxford University Press; 2005.
- Groopman J. How Doctors Think. Boston, MA: Houghton Mifflin; 2008.
- Densen P. Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc 2010; 122:48–58.
- Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30:2310–2317.
- Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human-computer interaction. Int J Med Inform 2012; 81:232–243.
- Fraccaro P, Arguello Castelerio M, Ainsworth J, Buchan I. Adoption of clinical decision support in multimorbidity: a systematic review. JMIR Med Informatics 2015; 3:e4.
- McLeod W, Eidus R, Stewart EE. Clinical decision support: using technology to identify patients’ unmet needs. Fam Pract Manag 2012; 19:22–28.
- Colla CH. Swimming against the current—what might work to reduce low-value care? N Engl J Med 2014; 371:1280–1283.
- Cook DA, Sorensen KJ, Nishimura RA, Ommen SR, Lloyd FJ. A comprehensive information technology system to support physician learning at the point of care. Acad Med 2015; 90:33–39.
- Woolever DR. The art and science of clinical decision making. Fam Pract Manag 2008; 15:31–36.
KEY POINTS
- Today, it seems impossible to keep up with all the information we need, but we can refine our skills in accessing, sorting, and interpreting accurate scientific evidence.
- The resources reviewed in this article require paid subscriptions except for Medscape, which is supported by advertising.
- Each of the resources has strengths and weaknesses. For example, UpToDate offers the most topics, but its articles tend to be too long to be practical to read at the point of care.
- Physicians should familiarize themselves with these resources and use the ones that best suit their needs.
