Unlocking the Power of OpenAI Text Embeddings in .NET Applications

Introduction

Embeddings, in the realm of natural language processing, offer a robust method to quantify the relatedness of text strings. OpenAI's text embeddings, as highlighted in a previous article, bring immense capabilities to applications demanding search, clustering, and classification features. The ConnectingApps.Refit.OpenAI NuGet page, as showcased here, further simplifies the integration of this feature into .NET applications.

Background

So, what exactly are text embeddings? Think of them as a method to convert words or sentences into numerical vectors while preserving their meaning. If two pieces of text have similar meanings, their vectors will be closer in this multi-dimensional space. For a more tangible example, consider a music streaming service that aims to recommend songs based on lyrics. By utilizing embeddings, the service can analyze the meaning behind the lyrics of your favorite songs and recommend others with similar thematic content. Diving a bit deeper, say you often listen to songs about "overcoming challenges". Even if the new songs don't use the exact phrase "overcoming challenges", as long as they convey a similar sentiment, they will be recommended to you. This magic is made possible by text embeddings. For an in-depth understanding, the official OpenAI documentation offers valuable insights into embeddings and their mechanics.

Using the code

The ConnectingApps.Refit.OpenAI NuGet package provides a seamless interface for harnessing OpenAI's API. To use text embeddings with this package:

1. Secure the package from its designated NuGet page.

2. Retrieve the necessary API keys from OpenAI and set them up as environment variables.

3. Integrate with OpenAI's embedding endpoint as done in the code example hosted on the NuGet package's GitHub repository.

using ConnectingApps.Refit.OpenAI;
using ConnectingApps.Refit.OpenAI.Embeddings;
using ConnectingApps.Refit.OpenAI.Embeddings.Request;
using Refit;

var apiKey = Environment.GetEnvironmentVariable("OPENAI_KEY");
var completionApi = RestService.For<IEmbedding>(new HttpClient
{
    BaseAddress = new Uri("https://api.openai.com")
}, OpenAiRefitSettings.RefitSettings);

var response = await completionApi.GetEmbeddingAsync(new EmbeddingRequest
    {
        Input = "The food was delicious",
        Model = "text-embedding-ada-002"
}, $"Bearer {apiKey}");

Console.WriteLine($"Returned response status code {response.StatusCode}");
Console.WriteLine($"Vector length {response.Content!.Data.First().Embedding.Length}");
Console.WriteLine($"Vector {string.Join(", ", response.Content!.Data.First().Embedding.Take(10))}...");

Upon executing the code, you will receive an output similar to:

Returned response status code OK
Vector length 1536
Vector 0,022599462, -0,0008510616, -0,005139073, -0,010128645, -0,0023203692, 0,0057370737, -0,01077648, -0,03453457, 0,008851663, -0,044576004...

This output signifies that the input text "The food was delicious" was converted into an embedding vector of length 1536. The printed vector values are just a snippet of the entire output, showcasing how the sentiment and meaning of the input text is represented numerically.

Points of Interest

The exploration of the ConnectingApps.Refit.OpenAI package brought to light the immense power of OpenAI’s capabilities when combined with the strengths of .NET. By abstracting complexities, this NuGet package makes fetching embeddings as easy as calling a routine function. This remarkable integration is a testament to how AI technologies are becoming increasingly accessible to developers across different platforms.

History

19th October, 2023: Revised article focusing on the application of embeddings in .NET using the ConnectingApps.Refit.OpenAI package. For those seeking a comprehensive overview, the detailed article dives deeper into this package's features and benefits.