Generation and human-expert evaluation of interesting research ideas using 
knowledge graphs and large language models

Gu, Xuemei; Krenn, Mario

アイテム詳細

登録内容を編集ファイル形式で保存

一時保存へ追加

タグ情報を表示リリース履歴を表示詳細要約

公開

成果報告書

Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language models

MPS-Authors

Gu, Xuemei
Krenn Research Group, Marquardt Division, Max Planck Institute for the Science of Light, Max Planck Society;

Krenn, Mario
Krenn Research Group, Marquardt Division, Max Planck Institute for the Science of Light, Max Planck Society;

External Resource

There are no locators available

Fulltext (restricted access)

There are currently no full texts shared for your IP range.

フルテキスト (公開)

2405.17044.pdf
(全文テキスト（全般）), 9MB

付随資料 (公開)

Bildschirmfoto 2024-05-31 um 13.37.20.png
(付録資料), 39KB

引用

Gu, X., & Krenn, M. (2024). Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language models. arXiv,.

引用: https://hdl.handle.net/21.11116/0000-000F-5B65-A

要旨

Advanced artificial intelligence (AI) systems with access to millions of research papers could inspire new research ideas that may not be conceived by humans alone. However, how interesting are these AI-generated ideas, and how can we improve their quality? Here, we introduce SciMuse, a system that uses an evolving knowledge graph built from more than 58 million scientific papers to generate personalized research ideas via an interface to GPT-4. We conducted a large-scale human evaluation with over 100 research group leaders from the Max Planck Society, who ranked more than 4,000 personalized research ideas based on their level of interest. This evaluation allows us to understand the relationships between scientific interest and the core properties of the knowledge graph. We find that data-efficient machine learning can predict research interest with high precision, allowing us to optimize the interest-level of generated research ideas. This work represents a step towards an artificial scientific muse that could catalyze unforeseen collaborations and suggest interesting avenues for scientists.