Evaluating Expert-authored Rules for Military Reasoning
Mike Pool, Kenneth S. Murray, Julie Fitzgerald, Mala Mehrotra,
Robert Schrag, Jim Blythe, Jihie Kim, Hans Chalupsky, Pierluigi
Miraglia, Thomas Russ and Dave Schneider
Abstract
Eliciting complex logical rules directly from logic-naive subject
matter experts (SMEs) is a challenging knowledge capture task. We
describe a large-scale experiment to evaluate tools designed to
produce SME-authored rule bases. We assess the quality of the rule
bases with respect to the: 1) performance on the addressed functional
task (military course of action (COA) critiquing); and 2) intrinsic
knowledge representation quality. In the course of this assessment, we
note both strengths and weaknesses in the state of the art, and
accordingly suggest some foci for future development in this important
technology area.
In John H. Gennari, Bruce W. Porter and Yolanda Gil
(eds), Proceedings of the 2nd International Conference on
Knowledge Capture (K-CAP 2003), Sanibel Island, FL, USA,
pp. 96-104, October 23-25, 2003.
Back to Paper List