LLMs One-shot Learning from Human Demonstration on Inductive Reasoning Tasks

Simple item page
dc.contributor.advisorHu, Xiaen_US
dc.contributor.advisorSegarra, Santiagoen_US
dc.creatorWang, Jeffen_US
dc.date.accessioned2025-01-16T20:00:32Zen_US
dc.date.available2025-01-16T20:00:32Zen_US
dc.date.created2024-12en_US
dc.date.issued2024-08-29en_US
dc.date.submittedDecember 2024en_US
dc.date.updated2025-01-16T20:00:32Zen_US
dc.description.abstractLarge Language Models (LLMs) have shown impressive proficiency across a range of natural language tasks. However, recent research has highlighted their limitations in inductive reasoning, a key aspect of human cognitive ability. While chain-of-thought demonstration have improved LLM performance on various tasks, there has been limited exploration into their effectiveness for inductive reasoning tasks specifically. Since inductive reasoning rules can vary widely, providing a tailored demonstration for every question is impractical. This study aims to investigate how LLMs perform with minimal demonstration and whether they can generalize in different regimes. To support this research, we design a programmable dataset with inputs of varying lengths and complexities to test LLMs' generalizability. Our findings suggest that a single human demonstration can enable LLMs to achieve perfect in-distribution generalization performance in some tasks. LLMs sometimes exhibit comparable or even better out-of-distribution generalization performance relative to their in-distribution performance.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.urihttps://hdl.handle.net/1911/118180en_US
dc.language.isoenen_US
dc.subjectLLMen_US
dc.subjectAIen_US
dc.titleLLMs One-shot Learning from Human Demonstration on Inductive Reasoning Tasksen_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentElectrical and Computer Engineeringen_US
thesis.degree.disciplineArtificial Intelligenceen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelMastersen_US
thesis.degree.nameMaster of Scienceen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
WANG-DOCUMENT-2024.pdf
Size:
1.12 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.97 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Theses and Dissertations