We developed the KnowledgeMap (KM) system as an online, concept-based database of medical school curriculum documents. It uses the KM concept indexer to map full-text documents and match search queries to concepts in the Unified Medical Language System (UMLS). In this paper, we describe the design of KM and report the first seven months of its implementation into a medical school. Despite being emphasized in only two first year courses and one fourth year course, students from all four classes used KM to search and browse documents.

KM is a Web-accessible, comprehensive database that organizes course materials (at the level of full lectures, not just outlines or syllabi) from the Vanderbilt School of Medicine curriculum. KM uses natural language processing techniques to analyze educational documents for biomedical concepts. Lecture handouts and Microsoft PowerPoint presentations are indexed and available online for students, faculty and administrators to search for individual or interrelated concepts across the medical school curriculum.

Often, medical educators and students do not know where important concepts are taught and learned in medical school. Manual efforts to identify and track concepts covered across the curriculum are inaccurate and resource intensive.

Many authors have reported on the use of the Unified Medical Language System (UMLS) to match concepts in free text. Unmatched search strings may be due to misspellings, concepts not in the UMLS, or searches for words not expected to be in the UMLS (e.g., names of people or places). We mapped search strings from a full-text, concept-based curriculum database to UMLSconcepts and performed a failure analysis. The majority of unmatched text strings were medically related (71.7%).

To determine whether the integration of an automated electronic clinical portfolio into clinical clerkships can improve the quality of feedback given to students on their patient write-ups and the quality of students' write-ups.

Currently, many medical educators track trainee clinical experience using student-created manual logs. Using a web-based portfolio system that captures all notes written by trainees in the electronic medical record, we examined a graduating medical student's clinical notes to determine if we could automatically assess exposure to 10 institution-defined core clinical topics. We located all biomedical concepts in his clinical notes, divided by note section, using the KnowledgeMap concept identifier. Notes were ranked according to the concepts matching each core topic's concept list.

Graduate medical students must demonstrate competency in clinical skills. Current tracking methods rely either on manual efforts or on simple electronic entry to record clinical experience. We evaluated automated methods to locate 10 institution-defined core clinical problems from three medical students' clinical notes (n=290). Each note was processed with section header identification algorithms and the KnowledgeMap concept identifier to locate Unified Medical Language System (UMLS) concepts.

Accurate assessment and evaluation of medical curricula has long been a goal of medical educators. Current methods rely on manually-entered keywords and trainee-recorded logs of case exposure. In this study, we used natural language processing to compare the clinical content coverage in a four-year medical curriculum to the electronic medical record notes written by clinical trainees. The content coverage was compared for each of 25 agreed-upon core clinical problems (CCPs) and seven categories of infectious diseases. Most CCPs were covered in both corpora.