Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checking
Abstract
The claim matching (CM) task can benefit an
automated fact-checking pipeline by putting
together claims that can be resolved with the
same fact-check. In this work, we are the
first to explore zero-shot and few-shot learning approaches to the task. We consider
CM as a binary classification task and experiment with a set of instruction-following
large language models (GPT-3.5-turbo, Gemini1.5-flash, Mistral-7B-Instruct, and Llama-3-
8B-Instruct), investigating prompt templates.
We introduce a new CM dataset, ClaimMatch,
which will be released upon acceptance. We
put LLMs to the test in the CM task and find
that it can be tackled by leveraging more mature yet similar tasks such as natural language
inference or paraphrase detection. We also propose a pipeline for CM, which we evaluate on
texts of different lengths.