local-llm-example

README

Local LLM Example 🚀

Welcome to the Local LLM Example! This nifty little Go program demonstrates how to use a local language model with the langchaingo library. It's perfect for those who want to run AI models on their own machines or servers. Let's dive in!

What Does This Example Do? 🤔

This example shows you how to:

1. Set up a local LLM client
2. Generate text using a simple prompt
3. Customize the LLM configuration (with some cool commented-out options)

The Magic Explained ✨

Here's what's happening in our main function:

1. We create a new local LLM client using local.New(). This uses default settings from your environment.

2. We set up a context for our LLM operations.

3. We generate text by asking the LLM a simple question: "How many sides does a square have?"

4. Finally, we print the LLM's response!

Cool Features to Explore 🕵️‍♀️

While the example uses default settings, it also shows you how to customize your LLM:

• You can specify a custom binary and arguments for your local LLM.
• There are options to set top-k, top-p, and seed values for text generation.
• You can even use global arguments as part of your LLM configuration!

Running the Example 🏃‍♂️

Just compile and run the Go file, and you'll see the LLM's response to the square question. It's that simple!

Have Fun! 🎉

This example is a great starting point for experimenting with local LLMs. Feel free to uncomment the additional options and play around with different configurations. Happy coding!