de.unidu.is.evaluation.trec
Class TRECEval

java.lang.Object
  de.unidu.is.evaluation.trec.TRECEval

public class TRECEval

extends java.lang.Object

An ecapsulation for the trec_eval program for evaluating TREC results.

The relevance judgement file must have the format [query id] * [document id] [0/1]

The result file must be in TREC format, e.g. created by de.unidu.is.evaluation.trec.TRECResult.

Since:

2004-01-06

Version:

$Revision: 1.6 $, $Date: 2005/03/04 11:19:18 $

Author:

Henrik Nottelmann

Constructor Summary

TRECEval(java.io.File trecEvalFile, java.io.File qrelFile)
Creates a new instance.

Method Summary

java.io.File	getQrelFile() Returns the file for the relevance judgements.
java.io.File	getTrecEvalFile() Returns the file for the trec_eval program.
double[]	run(java.io.File resultFile, java.io.File dataOutFile, java.io.File evalOutFile) Runs trec_eval file on the specified result file (in TREC format), and optionally writes the precision in the 11 standard recall points and the output of trec_eval in two different files.
double[]	run(java.io.File resultFile, java.io.Writer dataOutWriter, java.io.Writer evalOutWriter) Runs trec_eval file on the specified result file (in TREC format), and optionally writes the precision in the 11 standard recall points and the output of trec_eval in two different streams.
int[]	runAccuracy(java.io.File resultFile) Computes the number of retrieved documents, number of total relevant documents, number the retrieved relevant documents and the number of queries.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TRECEval

public TRECEval(java.io.File trecEvalFile,
                java.io.File qrelFile)

Creates a new instance.

Parameters:

trecEvalFile - file for the trec_eval program

qrelFile - file for the relevance judgements

Method Detail

run

public double[] run(java.io.File resultFile,
                    java.io.File dataOutFile,
                    java.io.File evalOutFile)
             throws java.io.IOException

Runs trec_eval file on the specified result file (in TREC format), and optionally writes the precision in the 11 standard recall points and the output of trec_eval in two different files. This method returns the precision at ranks 5, 10, 15, 20 and 30 and the average precision.

Parameters:

resultFile - file with the TREC result

dataOutFile - unless null, file for the precision points

evalOutFile - unless null, file for the trec_eval output

Returns:

array with 6 elements (precision at 5, 10, 15, 20, 30, AP)

Throws:

java.io.IOException

run

public double[] run(java.io.File resultFile,
                    java.io.Writer dataOutWriter,
                    java.io.Writer evalOutWriter)
             throws java.io.IOException

Runs trec_eval file on the specified result file (in TREC format), and optionally writes the precision in the 11 standard recall points and the output of trec_eval in two different streams. This method returns the precision at ranks 5, 10, 15, 20 and 30 and the average precision.

Parameters:

resultFile - file with the TREC result

dataOutWriter - unless null, stream for the precision points

evalOutWriter - unless null, stream for the trec_eval output

Returns:

array with 6 elements (precision at 5, 10, 15, 20, 30, AP)

Throws:

java.io.IOException

runAccuracy

public int[] runAccuracy(java.io.File resultFile)
                  throws java.io.IOException

Computes the number of retrieved documents, number of total relevant documents, number the retrieved relevant documents and the number of queries.

Parameters:

resultFile - result file

Returns:

array with 4 elements (order see above)

Throws:

java.io.IOException

getTrecEvalFile

public java.io.File getTrecEvalFile()

Returns the file for the trec_eval program.

Returns:

file for the trec_eval program

getQrelFile

public java.io.File getQrelFile()

Returns the file for the relevance judgements.

Returns:

file for the relevance judgements