|
General
The Halo Pilot was a six-month effort to investigate the state-of-the-art in logical inference-based
question answering.
Programmatics
Vulcan Inc. employed rigorous program management practices, using a DARPA/ARDA- inspired selective
call for proposal process to solicit competitive bids from 15 organizations. The program
adhered to a strict schedule with numerous meetings. For more details see the
Programmatics section.
Teams
Vulcan Inc. contracted three of the leaders in knowledge-based representation and reasoning (KRR):
Cycorp, based in Austin, Texas; SRI International, based in Menlo Park, California, with significant contributions from Boeing and the University of Texas; and Ontoprise, based in Karlsruhe,
Germany. For more details on the teams see the
Team Documentation section.
The Halo teams, to help bring Vulcan Inc. up to speed in this challenging field, assembled an
extensive training program. It consisted of a primer from a well-known artificial intelligence
(AI) textbook, followed by team-oriented algorithmic training, and Tools and Admin training (TAT)
to enable the running of the Halo applications. The TAT also included instruction on question
encoding/translation methodologies. For more details on our extensive training program, see the
Team Documentation section.
Challenge
Each team was given four months to independently encode 50 pages out of a chemistry syllabus
into their respective KRR technology platforms. At the end of this time, all the systems were
sequestered and a challenge exam consisting of 100 mostly novel questions was released to the teams.
The teams had two weeks to translate the questions into their respective formal logical languages.
These translations were run as batch jobs on the systems, which produced documents containing
English answers and justifications.
The final step was an impartial evaluation conducted by three separate chemistry professors,
who graded the “exams” for accuracy and justification quality.
All three teams did remarkably well on the challenge, leaving Vulcan Inc. extremely
encouraged that novel question-answering is technically feasible.
For more details see the Challenge section.
Final Reports
All the Halo teams assembled a comprehensive final report.
For more details see the Final Reports section.
Publications
For a list of Halo-related publications, please see the
Publications section.
Downloads
All the Halo applications are available for download and install. For more details see the
Downloads section.
|
