Client: Streamlit
Server Side: LangChain 🦜🔗
Vectorstore: PGVector
Embeddings: GCP VertexAI
LLMS: PaLM 2, AI21 Contextual Answers
Runtime: Cloud Run
To run this project, you will need to add the following environment variables to your .env file
AI21_API_KEY, CONNECTION_STRING
Clone the project
git clone https://github.com/g-emarco/advanced-rag.gitGo to the project directory
cd advanced-ragInstall dependencies
pipenv installStart the Streamlit server
streamlit run app.pyNOTE: When running locally make sure GOOGLE_APPLICATION_CREDENTIALS is set to a service account with permissions to use VertexAI
CI/CD via Cloud build is availale in cloudbuild.yaml
Please replace $PROJECT_ID with your actual Google Cloud project ID.
To deploy manually:
- Make sure you enable GCP APIs:
gcloud services enable cloudbuild.googleapis.com
gcloud services enable run.googleapis.com
gcloud services enable secretmanager.googleapis.com
gcloud services enable artifactregistry.googleapis.com
gcloud services enable vertexai.googleapis.com
- Create a service account
rag-app-sawith the following roles:
gcloud iam service-accounts create rag-app-sa \
--display-name="SA For Application"
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:rag-app-sa@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/run.invoker"
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:rag-app-sa@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/serviceusage.serviceUsageConsumer"
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:rag-app-sa@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/ml.admin"
gcloud projects add-iam-policy-binding PROJECT_ID \
--member=<
63AD
span class="pl-s">"serviceAccount:rag-app-sa@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/vertexai.admin"
- Create the secrets:
AI21_API_KEY
and for each secret grant the SA rag-app-sa@$PROJECT_ID.iam.gserviceaccount.com Secret Manager Secret Accessor
role to th secrets
- Build Image
docker build . -t us-east1-docker.pkg.dev/$PROJECT_ID/app/documentation-assistant:latest- Push to Artifact Registry
docker push us-east1-docker.pkg.dev/$PROJECT_ID/app/documentation-assistant:latest- Deploy to cloud run
--image=us-east1-docker.pkg.dev/PROJECT_ID/app/documentation-assistant:latest \
--region=us-east1 \
--service-account=rag-app-sa@$PROJECT_ID.iam.gserviceaccount.com \
--allow-unauthenticated \
--set-secrets="GOOGLE_API_KEY=projects/PROJECT_ID/secrets/AI21_API_KEY/versions/latest
Eden Marco, LLM Lead @ Google Cloud, Tel Aviv🇮🇱
