What is Spring AI, and how to use it: A beginner's guide
SpringAI is a powerful tool that helps Java application developers work with LLMs in a platform-agnostic way. Here's how to start using it.
Feb 20, 2024 • 6 Minute Read
Applications everywhere are investigating how artificial intelligence can enhance their application. In this article, you’ll learn about a new Spring project that affords Java developers a convenient API abstraction for working with large-language models (LLMs) like ChatGPT. By the end, we’ll have a simple program that can answer questions about the Swiftie Bowl based on information retrieved from Wikipedia.
The Origin of Spring AI
Spring AI --- a Spring project, not quite GA as of this writing --- represents a groundbreaking initiative aimed at simplifying the integration of artificial intelligence features into applications across the JVM family of languages. It was introduced officially to the world in the fall of 2023 at SpringOne by Mark Pollack, its creator. In true Spring fashion, the project introduces a common Spring-like pane of glass across the disparate LLM APIs cropping up, among them OpenAI and Azure OpenAI.
While it is in the experimental phase, it’s not considered production-ready; however, it is certainly worth a look so you can get a jump on learning the kinds of problems AI can help you solve.
AI Concepts
Before creating our sample application, let’s go over a few basic terms:
Models: AI models are algorithms that mimic human cognitive functions to generate predictions, text, images, or other outputs from large datasets.
Prompts: Prompts are language-based inputs that guide AI models to produce specific outputs; often this requires skillful crafting (sometimes called Prompt Engineering) to be effective.
Prompt Templates: These templates use text-based engines to create prompts by substituting parts of the request with user-specific values.
Embeddings: Embeddings transform text into numerical vectors, allowing AI models to process and understand language.
Tokens: Tokens are the basic units of language processed by AI models, with usage directly linked to the cost of AI services.
We’ll refer to each of these concepts while creating the application.
Initializing a Spring AI Application
Spring AI is so new that it isn’t part of the Spring Intializr. While we could use https://start.spring.io to create a skeleton and then manually add the starters afterward, I’m going to use the Spring CLI which already has Spring AI support.
Once you’ve installed the CLI, navigate to a directory where you want the application to be, start the Spring Shell, and run:
spring:> boot new spring-ai-solar-eclipse ai
Getting project from https://github.com/rd-1-2022/ai-openai-helloworld
Created project in directory 'spring-ai-solar-eclipse'
spring:> exit
This creates a Hello World project for interacting with ChatGPT.
Then, take your ChatGPT API key and export it like so:
export SPRING_AI_OPENAI_API_KEY=your-api-key
Finally, run the application using mvnw:
./mvnw spring-boot:run
Then you can send it a command like “Write a haiku”:
http :8080/ai/simple?message=”Please write me a haiku”
and see ChatGPT’s response:
{
"completion": "Beneath moon's soft glow,\nWhispers of the gentle breeze,\nNature's calm, bestowed."
}
Inspecting the Code
Now that we’ve seen how to run the application, let’s dive into some of the details of the application to learn about Spring AI’s API.
In SimpleAiController, you can see AiClient as an injected dependency:
@RestController
public class SimpleAiController {
private final ChatClient chatClient;
@Autowired
public SimpleAiController(ChatClient chatClient) {
this.chatClient = chatClient;
}
//…
}
Similar to other XXXClient classes in the Spring portfolio like WebClient and LdapClient, ChatClient represents the interface by which you’ll interact with any LLM.
Its simplest usage is similar to what you may have already experienced when using the ChatGPT web interface; you give it a string and it gives one back. You can see the above controller doing just that in its one method:
@GetMapping("/ai/simple")
public Completion completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return new Completion(chatClient.call(message));
}
So, when we sent “Please write me a haiku”, Spring received that, sent it to this rest controller which subsequently transmitted it to OpenAI via its AIClient implementation.
But, because the model is trained only periodically on recent content, it doesn’t know the answer to recent questions like the following:
http :8080/ai/simple?”Who won the Swiftie Bowl? If you don’t know, just say ‘I don’t know’”
It will give the simple answer:
{
"completion": "I don’t know"
}
Can Spring AI help us with this? Let’s take a look.
Stuffing Prompt with Additional Information
One technique as far as effective prompting is stuffing it with information in addition to the question or message that the user is sending.
At the most basic (and often inefficient) level, this looks like giving OpenAI the text of an article along with a question like so:
?message=”Based on this article: ‘{text of the article}’, please answer me this question: {your question}”
The OpenAI API can handle this just fine, but as I said, it’s inefficient since OpenAI charges you by the token. Since it’s quite likely that the answer to your question exists in only a portion of that article, sending fewer parts will result in you spending less money to get the same answer.
Pulling Relevant Information from Wikipedia
If we download the latest Wikipedia article about the Super Bowl, it should help ChatGPT to be able to answer our question.
Place it in the src/main/resources directory of the project.
Storing in a Vector Database
Next, you’ll load the contents of that pdf into a VectorStore. You can use a local store like Redis, but for the purposes of this intro article, you’ll use SimpleVectorStore, which is an in-memory store.
NOTE: This demo uses the `OpenAIEmbeddingClient` by default, using your OpenAI credits. See the documentation for other embedding clients that run locally. Additionally, since we are using `SimpleVectorStore`, the embedding happens again at startup each time. For a production application, you would certainly want to store these vectors somewhere outside of the application.
Since we’re working with a PDF, add the following dependency to your project:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-pdf-document-reader</artifactId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
This adds the needed APIs for parsing the PDF file.
Then in Application, publish the vector store, including the PDF as its contents:
@Bean
VectorStore vectors(EmbeddingClient embedding, @Value("Super_Bowl_LVIII.pdf") Resource pdf) {
SimpleVectorStore vectors = new SimpleVectorStore(embedding);
var reader = new PagePdfDocumentReader(pdf);
var splitter = new TokenTextSplitter();
var documents = splitter.apply(reader.get());
vectors.accept(documents);
return vectors;
}
What this does is split up the document into smaller chunks and then store them locally as a set of vectors. In the next step, we’ll have Spring AI do some math to figure out which of those document chunks to send to OpenAI as part of our question.
Then, let’s update SimpleAiController to also be dependent on the VectorStore like so:
private final VectorStore vectors;
@Autowired
public SimpleAiController(ChatClient chat, VectorStore vectors) {
this.chat = chat;
this.vectors = vectors;
}
Finding Relevant Information to Include
Now we’re ready to add a new endpoint to SimpleAiController. This time, it will have two parameters, one for the stuffed prompt and one for the message:
@GetMapping("/ai/stuffed")
public Completion completion(
@RequestParam(value = "prompt", defaultValue = "Based on the following: {documents}") String prompt,
@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
// … content to follow
}
But instead of simply calling chatClient.call like the first time, we’re going to:
- Search the vector store for relevant chunks, and
- Construct a prompt that includes those chunks
You can search the vector store by the user’s message like this:
var documents = this.vectors.similaritySearch(message);
var inlined = documents.stream().map(Document::getContent).collect(Collectors.joining(System.lineSeparator()));
Then you can send both those documents and the user’s message so OpenAI has more context to work with:
var system = new SystemPromptTemplate(prompt).createMessage(Map.of("documents", inlined));
var user = new UserMessage(message);
return new Completion(this.chat.call(new Prompt(List.of(system, user))).getResult().getOutput().getContent());
Trying It Out
With that endpoint in place, start up the application and try it out!
http :8080/ai/stuffed?message=”Who won the Swiftie Bowl? If you don’t know, just say ‘I don’t know’”
{
“completion”: "The Kansas City Chiefs won the Swiftie Bowl, defeating the San Francisco 49ers with a final score of 17-10."
}
Conclusion
Spring AI is a powerful tool for working with LLMs in a platform-agnostic way. Today, you caught a glimpse of what the ChatClient and VectorStore APIs are capable of, allowing you in a few keystrokes to create a chatbot that references a set of data that you provide.
Choosing Spring AI allows you to work within the Spring programming model. Looking ahead, choosing something platform-agnostic like Spring AI allows you either to switch platforms or use a multiplicity of platforms without changing the way you interface with them.
Other learning resources
Other learning resources
Want to learn more about using the Spring Framework? Pluralsight offers a wide range of learning paths that can help you become a master of all things Spring. Here's some you should check out:
