~~NOTOC~~


[[http://dialrc.org/sdc/|{{  :projs:sdc2010:header.jpg?740  |}}]]



===== The Spoken Dialog Challenge 2010 =====
== Yi ZHU ==
{{ people:zhuyi2.jpg:?80|}}
  * Department of Computer Science and Engineering
  * The Chinese University of Hong Kong
  * Homepage: [[http://wiki.cse.cuhk.edu.hk/user/zhuyi|http://wiki.cse.cuhk.edu.hk/user/zhuyi]]
  * Email: zhuyi@cse.cuhk.edu.hk
== Baichuan Li ==
  * Department of Computer Science and Engineering
  * The Chinese University of Hong Kong
  * Email: tobcli@gmail.com

==== Introduction ====
The Dialog Research Center at Carnegie Mellon (DialRC) build a telephone-based spoken dialog system that provides bus schedule information for the City of Pittsburgh, PA (USA). The DialRC establish this challenge to call for participation with three fold: 1). build a system with the similar function; 2). build simulate user for these system and 3). build a machine judge to evaluate the user satisfication of the system.

==== Objective ====
Our goal is to evaluate the system based on user satisfication.

==== Ideas ====
We analyze the dialogs with Natural Language Processing and statistical techniques to extract useful features from the dialogs, which is basically considering its question answer pairs. We could do regression with those features. Based on the user's query and system response, we can construct a state diagram to improve the regression step by divide the dialog by the state transition.

==== Previous and On Going Work ====
  - **(done)**: Find the most popular path on the state diagram that the CMU's system works;
  - **(done)**: Find out the states and construct the state diagram according to the dialog;
  - **(done)**: Put part of the dialog onto the  [[https://www.mturk.com/mturk/welcome|Amazon Mechanical Turk]] for labelling;
  - **(done)**: Feature selection;
  - **(on going)**: Find appropriate regression method;
  - **(on going)**: Decide the weight on the state diagram;
  - **(on going)**: Test experiment.

==== Presentation slides ====
{{projs:sdc2010:presentation_01--a_graph-based_semi-supervised_learning_for_question-answering.pdf|A_Graph-based_Semi-Supervised_Learning_for_Question-Answering}}

==== Papers ====
{{:projs:sdc2010:a_graph-based_semi-supervised_learning_for_question-answering.pdf|A_Graph-based_Semi-Supervised_Learning_for_Question-Answering}}
{{:projs:sdc2010:paradise-a_framework_for_evaluating_spoken_dialogue_agents.pdf|paradise:a_framework_for_evaluating_spoken_dialogue_agents}} \\
{{:projs:sdc2010:feature_selection.pdf|Feature Selection for Evaluating Spoken Dialog System}}


==== Some results ====
{{:projs:sdc2010:result.pdf|Status Statistic Result}}

{{:projs:sdc2010:status_transition_graph_01.rar|Status Transition Graph}}

{{:projs:sdc2010:status_transition_graph_02.rar|Status Transition Graph (Add all the capital word in the System Response as the location)}}

==== Dialog labelling instruction ====
{{:projs:sdc2010:instruction_of_dialog_labelling.doc|For amazon's mechanical turk}}