hide
Free keywords:
-
Abstract:
Introduction: The programming of MRI sequences remains a chal-
lenging and time-consuming task for researchers, technicians and
students alike. With the Pulseq standard, this process recently became
simpler. In this study, we investigated the potential of GPT-4, an
advanced large language model (LLM) capable of programming and
producing natural language, as an MRI sequence programming
assistant using the Pulseq framework, accelerating MR prototyping in
development and education.
Methods: We used ChatGPT [1] with the models GPT-3.5 and GPT-4.
We adapted GPT for MRI sequence programming by adding a custom
prompt [2] to turn it into an MRI and PyPulseq [3] coding assistant
abbreviated here by GPT4MR. The prompt contained general
instructions as well as PyPulseq function definitions and examples,
principles known as in-context few-shot learning [4,5] and Chain-of-
Thought Prompting [6]. We tested the AI model’s ability to generate
simple pulse sequences, composite binomial pulses, a spin echo EPI
sequence with reconstruction, and a Lissajous-EPI. To facilitate user
experimentation, we share an open Colab notebook [2] containing the
GPT4MR prompt, comprehensive examples, guidance, and a platform
to test GPT-4 as an MR coding assistant. This abstract was revised by
GPT-4.
Results: In our initial attempts, native GPT models often generated
erroneous code, mainly using non-existing PyPulseq subfunc-
tions. The performance is considerably improved using our
tailored GPT4MR prompt, allowing it to generate MRI sequences
with fewer or no errors (Fig. 1). GPT-4 outperformed GPT-3.5 in
terms of number of bugs and reasoning/explaining. However, our
study also revealed GPT-4’s limitations in handling more complex
sequence ideas or fully replicating existing sequence concepts.
Prompts like ’’Code a spin echo EPI‘‘ lead to running code, but
conceptual sequence errors (Fig. 2). Interestingly, GPT4MR was able
to correct its own errors when problems were pointed out. When
instructed with step-by-step instructions of the sequence implemen-
tation as plain text, GPT4MR was able to generate correct and
running code in a single try for a spin echo EPI (Fig. 3), and a
Lissajous EPI (Fig. 4). While timings, gradient moments etc. were not
always 100% correct or optimal, running codes were produced and an
easy-to-alter base sequence as well as a correct EPI FFT recon-
struction (Fig. 3), as well as a non-uniform FFT reconstruction using
the advanced torchkbnufft package, which yields a better outcome
compared to linear re-gridding, were generated (Fig. 4).
Discussion: Our findings indicate that LLMs have the potential to
serve as a valuable MRI sequence programming assistant, enabling
faster development of novel MRI sequences, reconstruction, or
building blocks. Our last two chosen examples cannot be found on the
internet (i.e. in the training set of GPT-4), demonstrating GPT4MR’s
capacity to accelerate the realization of new MRI sequence ideas.
However, its limitations in dealing with complex ideas and sequences
necessitate further research and improvement. A well designed
prompt including PyPulseq documentation can improve the perfor-
mance considerably. GPT4MR understood programming hints and
altered the code accordingly, forming a sparring partner for fast MR
prototyping.
Conclusion We propose a versatile prompt that enables GPT-4 to act
as a Pulseq coding assistant for MRI sequence/recon development and
prototyping, streamlining the process.
As a future outlook, integrating a PyPulseq plugin into (free, leight-
weight, open source) [7] LLMs could create a powerful tool for MRI
sequence development and prototyping.
