Adding feature for search result diversity re-ranking and evaluation - submission for the IRDM group project

[yiweichen]

Adding feature for search result diversity re-ranking and evaluation - submission for the IRDM group project.

Group 6: Yiwei Chen, Yifei Rong, Tianxiang Li

This is a submission of code for our group project of COMPGI15 Information Retrieval and Data Mining. We forked the original repository and added two features onto the Panda platform:

• The -r mode: re-ranking evaluation, format: -r <approach>. We have implemented two re-ranking approaches, MMR and Portfolio, for our comparison experiment. But it is not limited to this particular two approaches. As long as fitting the Rerank interface, more re-ranking approaches can be added into Panda. Also, -m can be used together to specify the underlying document scoring model.
• The -br mode: batch re-ranking evaluation, format: -br <approach>:a:i:b. As in some re-ranking approaches, a parameter (e.g. λ in MMR or b in Portfolio) can be tuned, this mode is used to run evaluations over various parameters, similar to the -b mode. Same as -r, the -m option can be used together to specify the underlying document scoring model.

Usage Examples

• -r mmr -m 2 runs a evaluation using MMR as the re-ranking approach and BM25 (numbered 2) as the underlying document scoring model.
• -br portfolio:0:0.1:1 -m 2 runs 11 evaluations with b = 0, 0.1, 0.2, ... , 0.9, 1 using Portfolio as the re-ranking approach and BM25 (numbered 2) as the underlying document scoring model.

About the details of implementation and our comparison experiments, please see our report submitted.