Google Memorystore for Redis

Google Memorystore for Redis is a fully-managed service that is powered by the Redis in-memory data store to build application caches that provide sub-millisecond data access. Extend your database application to build AI-powered experiences leveraging Memorystore for Redis’s Langchain integrations.

​

Pre-reqs

​

Before You Begin

• Create a Google Cloud Project
• Enable the Memorystore for Redis API
• Create a Memorystore for Redis instance. Ensure that the version is greater than or equal to 7.2.

​

🦜🔗 Library Installation

%pip install -upgrade --quiet langchain-google-memorystore-redis langchain

# # Automatically restart kernel after installs so that your environment can access the new packages
# import IPython

# app = IPython.Application.instance()
# app.kernel.do_shutdown(True)

​

☁ Set Your Google Cloud Project

• Run gcloud config list.
• Run gcloud projects list.
• See the support page: Locate the project ID.

# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.

PROJECT_ID = "my-project-id"  # @param {type:"string"}

# Set the project id
!gcloud config set project {PROJECT_ID}

​

🔐 Authentication

• If you are using Colab to run this notebook, use the cell below and continue.
• If you are using Vertex AI Workbench, check out the setup instructions here.

from google.colab import auth

auth.authenticate_user()

​

Basic Usage

​

Initialize a Vector Index

import redis
from langchain_google_memorystore_redis import (
    DistanceStrategy,
    HNSWConfig,
    RedisVectorStore,
)

# Connect to a Memorystore for Redis instance
redis_client = redis.from_url("redis://127.0.0.1:6379")

# Configure HNSW index with descriptive parameters
index_config = HNSWConfig(
    name="my_vector_index", distance_strategy=DistanceStrategy.COSINE, vector_size=128
)

# Initialize/create the vector store index
RedisVectorStore.init_index(client=redis_client, index_config=index_config)

​

Prepare Documents

• Loading Text: The TextLoader obtains text data from a file (e.g., “state_of_the_union.txt”).
• Text Splitting: The CharacterTextSplitter breaks the text into smaller chunks for embedding models.

from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import CharacterTextSplitter

loader = TextLoader("./state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

​

Add Documents to the Vector Store

​

Method 1: Classmethod for Direct Insertion

from langchain_community.embeddings.fake import FakeEmbeddings

embeddings = FakeEmbeddings(size=128)
redis_client = redis.from_url("redis://127.0.0.1:6379")
rvs = RedisVectorStore.from_documents(
    docs, embedding=embeddings, client=redis_client, index_name="my_vector_index"
)

​

Method 2: Instance-Based Insertion

• [Optional] Create a RedisVectorStore Instance: Instantiate a RedisVectorStore object for customization. If you already have an instance, proceed to the next step.
• Add Text with Metadata: Provide raw text and metadata to the instance. Embedding generation and insertion into the vector store are handled automatically.

rvs = RedisVectorStore(
    client=redis_client, index_name="my_vector_index", embeddings=embeddings
)
ids = rvs.add_texts(
    texts=[d.page_content for d in docs], metadatas=[d.metadata for d in docs]
)

​

Perform a Similarity Search (KNN)

• Formulate the Query: A natural language question expresses the search intent (e.g., “What did the president say about Ketanji Brown Jackson”).
• Retrieve Similar Results: The similarity_search method finds items in the vector store closest to the query in meaning.

import pprint

query = "What did the president say about Ketanji Brown Jackson"
knn_results = rvs.similarity_search(query=query)
pprint.pprint(knn_results)

​

Perform a Range-Based Similarity Search

• Formulate the Query: A natural language question defines the search intent.
• Set Similarity Threshold: The distance_threshold parameter determines how close a match must be considered relevant.
• Retrieve Results: The similarity_search_with_score method finds items from the vector store that fall within the specified similarity threshold.

rq_results = rvs.similarity_search_with_score(query=query, distance_threshold=0.8)
pprint.pprint(rq_results)

​

Perform a Maximal Marginal Relevance (MMR) Search

• Formulate the Query: A natural language question defines the search intent.
• Balance Relevance and Diversity: The lambda_mult parameter controls the trade-off between strict relevance and promoting variety in the results.
• Retrieve MMR Results: The max_marginal_relevance_search method returns items that optimize the combination of relevance and diversity based on the lambda setting.

mmr_results = rvs.max_marginal_relevance_search(query=query, lambda_mult=0.90)
pprint.pprint(mmr_results)

​

Use the Vector Store as a Retriever

• LangChain Compatibility: Many LangChain tools and methods are designed to directly interact with retrievers.
• Ease of Use: The as_retriever() method converts the vector store into a format that simplifies querying.

retriever = rvs.as_retriever()
results = retriever.invoke(query)
pprint.pprint(results)

​

Clean up

​

Delete Documents from the Vector Store

rvs.delete(ids)

​

Delete a Vector Index

• Index Configuration Changes: If index parameters need modification, it’s often required to delete and recreate the index.
• Storage Management: Removing unused indices can help free up space within the Redis instance.

# Delete the vector index
RedisVectorStore.drop_index(client=redis_client, index_name="my_vector_index")