DELOS:Meeting in Duisburg in June 2005

From Wiki

Jump to: navigation, search

Contents

First Meeting of JPA 2 Task 7.5

Agenda

  1. Presentation of Evaluation Computer Model
  2. Presentation of Interaction Evaluation Model
  3. Presentation of Risoe
  4. Presentation of Daffodil
  5. ... to come

Meeting

  • Introduction of Norbert, concerning DELOS JPA2
    1. Review in March 2006
    2. SB decision on which activity will continue: latest January 2006
    3. EU Digital Library: End of September 2005 Demonstration of concepts for DLs
  • ECDL Poster presentation of founded activities
  • Delos DL Reference Model (WP 1+2)
  • Andras
    1. Evaluation Computer
    2. Store Evaluation, store primary log data
    3. Links to Meta Evaluationdescriptions
    4. Testbed: [http:]
  • Sarantos
    1. New Interaction Evaluation
      1. User, Content, System have different attributes
      2. Interaction: Control, Shift management, lucidity
      3. Requirements: Who, What, Where, When, How (Methods and Criterion)
      4. Combination of Interaction & Evaluation Computer
      5. Goals: Stabalize model
  • Hanne
    1. Taxonomy of WP4

Proposal Goals

Report: Description of the evaluation framework based on Evaluation Computer and the Interaction Evaluation models model

    1. Read [1]
    2. Integrate empirical user eval + Evaluation Computer + Interaction Evaluation

Evaluation Framework

      1. Daffodil + Logging + Tutorial
      2. Conceptional Framework vs. Implementation & Requirements
        1. Saracevic [2]
      3. Means end analysis of user domain of current Daffodil
    2. Report: Agree on the following steps
    3. Start from evaluation phases:
      • PLAN
      • COLLECT
      • ANALYSE
      1. Provide solution through ...

Implementation of the testbed framework

    1. Daffodil is an instantiation of the framework
    2. DL usability collaboration (col-)laboratory
    3. Log repository of primary data
    4. Classification of kind of evaluation (based on our frameworks)
    5. Evalued: [3]
    6. Book: [4]

Implementation of two applications of the testbed framework

    1. Application 1
      1. Archeological Data from Sarantos (Usergroups)
      2. Perseus (Tufts: Greg Crane)
      3. Think about more nice services by spatial data
    2. Application 2
      1. Economics: Budget: user behavior on constraints
      2. Annotation 4.10
      3. Fulltext
    3. Application 3
      1. Evaluate different graphical tools

Report: A standard for describing events in a digital library on different levels, along with a corresponding XML Schema specifying a logging format.

  1. Pejtersen: Strategy model [5]
  2. Prepare some suggestion

Implementation of the logging standard within the testbed framework.

  1. Storing events in standard format
  2. Providing all events possible

Report: Verification technique for the comparison of event specifications and logs.

  1. protocol verification
  2.  ???
  3. Laszlo

Scientific primary data repository for DL interaction logs

  1. privacy issue
  2. Ask user about log
  3. Tool for finding log information (need to have context)
  4. Context: User, Country, etc.

A set of tools for statistical analysis of logs and for conversion into the ARFF format.

  1. ARFF: Standard format for data mining (linear record)
  2. Need: Different transformations of XML Schema
  3. Filtering (Projection) & Transformation
  4. Implement & Gather tools

Report: Evaluation of two applications of the testbed framework based on the logging standard and the analysis tools.

Next Steps

  1. Evaluation Framework
    1. Conceptual Framework
      1. Condensed Versions/Comparison/Compilation of (due 24. June 2005)
        • Hanne: Onion model
        1. Compressed version of the Onion model. Media:Onion.pdf
        • Us: Evaluation Computer
        • IU: Interaction model
        1. IU deposited a condensed version of the Interaction Framework. The PDF file contains also a short paragraph about the potential unification of the two models (Interaction Framework and Evaluation Computer). In case further clarification or analysis is required, IU will return with a revised version of the file. Media:InteractionFramework_CondVersion.pdf
        • Hanne: W4 - Taxonomy
        • Peter: Saracevic
        • Us: 5S
        • create an integrated version (due 22. July 2005)
      2. Goals, Usecases, Ideas -> ECDL 2005 Poster (due 24. June 2005)
      3. Move from general evaluation model like "Evalued" to our evaluation model (due 26. August 2005)
        1. Take Ann Blandford's papers and compare/evaluate our evaluation framework
    2. Schema description of events (due 24. June 2005)
      1. Extract sample set from logging
      2. Think about structure
  2. Application
    1. Get archeological metadata, evaluate and integrate into Daffodil
    2. Search for more data source
    3. Find/ask scenarios
    4. Contact Tufts
    5. Externalize Strings in Daffodil to have language support
  3. Evaluation
    1. Planning
    2. Talk to persons: Contact Preben Hansen; Peter Ingwersen; Kield Schmidt; Peter Carstensen; Morten Hertzum
    3. Use of logging mechanism to compare evaluations
      1. One old system vs. Daffodil, same goals
    4. Scenarios for Evaluations: Possibilites, Drawbacks, Todos
  4. Overall Deadlines
    1. December 2005: Report for review.

Deadlines

  • 24. June 2005: Condensed Versions/Comparison/Compilation of
    • Hanne: Onion model
    • Us: Evaluation Computer
    • IU: Interaction model
    • Hanne: W4 - Taxonomy
    • Peter: Saracevic
    • Us: 5S
  • 24. June 2005: Goals, Usecases, Ideas for ECDL Poster
  • 24. June 2005: Begin of Schema description of events
    • Extract sample set from logging
  • 26. August 2005: Integrated version
  • 26. August 2005: Move from general evaluation model like "Evalued" to our evaluation model
  • December 2005: Report for review
Retrieved from "http://www.is.informatik.uni-duisburg.de/wiki/index.php/DELOS:Meeting_in_Duisburg_in_June_2005"
Personal tools