An Achievement Test for Knowledge-Based Systems: QUEM

This paper describes QUEM, a method for assessing the skill level of a knowledge-based system based on the quality of the solutions it produces. QUEM is demonstrated by using it to assess the performance of a particular knowledge-based system, P3 . QUEM can be viewed as an achievement or job placement test given to know ledge-based systems to help system designers determine how the system should be used, and in what capacity by what level of users. In general, it is difficult to find useful metrics for assessing a system's overall performance. Most literature on evaluation deals with validation, verification and testing in which the primary concern is the correctness and consistency in the databases and rulebases. However, these properties alone may not be sufficient to determine how well a system performs its task. QUEM allows software developers to assess their system's performance by constructing a skill function based on human performance data that relates experience and...

• Type: Conference
• Year: 1997
• Where: IJCAI
• Authors: Caroline C. Hayes, Michael I. Parzen
• Keywords: IJCAI 1997, IJCAI 2007, Knowledge-based System, Particular Knowledge-based System, QUEM Allows Software