Graphs for HPC and LLMs

LLM Techniques

Exploring the integration of graphs into large language models (LLMs) reveals innovative reasoning techniques. Maciej Besta highlights the use of structured prompts, such as chain of thought and graph of thought, to enhance LLM performance by decomposing problems into sub-steps 1. He explains that refining prompts through iterative learning allows LLMs to develop more abstract and powerful problem-solving capabilities 2.

Learning is not just about training weights, but refining prompts to solve more general problems.

--- Maciej Besta

Additionally, Besta discusses program-aided prompting, where LLMs create pseudocode to solve problems, demonstrating their effectiveness in planning complex tasks 3.

   

Enhancing Outputs

Enhancing LLM outputs through graph-based approaches offers a transformative potential. Maciej Besta suggests applying graphs at various stages, such as fine-tuning and retrieval, to improve reasoning and reduce hallucinations 4. He emphasizes the importance of balancing cost and quality in problem-solving, noting that while chain of thought is cost-effective, it may not always yield the best results 5.

You need to decide what is most important: quality or cost.

--- Maciej Besta

Besta also discusses the advantages of using trees over graphs in certain scenarios, highlighting the need to tailor approaches based on problem complexity 6.