Scaling NL-to-SQL Agents with GraphRAG on AWS

Introduction This article addresses a specific problem in natural language to SQL generation: avoiding context overflow and LLM confusion when generating SQL queries against databases with a large number of tables and complex foreign key dependencies. The described solution uses GraphRAG to improve LLM accuracy by eliminating guesswork and reducing context pollution. The Problem Consider: an RDS PostgreSQL instance […]

Experimenting with the Databricks Mosaic AI Vector Search using python: a beginner example

Introduction After exploring how vector databases work and trying the AWS OpenSearch vector database in this blog post, the goal of this article is to experiment with a similar product from Databricks – Mosaic AI Vector Search. We will create a new Vector Search endpoint and index, load the toy dataset, try to perform similarity […]

Augment Your Retrieval: LLMs with Python LangChain and AWS OpenSearch VectorSearch database

Introduction In this blog post we will introduce vector databases and some of the algorithms used for indexing and show examples of how to work with AWS OpenSearch vector database using python and LangChain library. Vector databases A vector database is a type of database that stores high-dimensional vectors for fast retrieval and similarity search. […]

DBT and Databricks part 3: Loading noSQL data (from MongoDB) into Databricks

This series of blog posts will illustrate how to use DBT with Azure Databricks: set up a connection profile, work with python models, and copy noSQL data into Databricks(from MongoDB). In the third part, we will talk about one specific example of how to load noSQL data into Databricks(originally coming from MongoDB). Task: We have […]

DBT and Databricks Part 2: Working with python models

This series of blog posts will illustrate how to use DBT with Azure Databricks: set up a connection profile, work with python models, and copy noSQL data into Databricks(from MongoDB). In the second part, we will talk about working with python models. Starting from version 1.3 python support is added to DBT. As for now […]

DBT and Databricks Part 1: Setting up DBT profile for connecting to Azure Databricks using…

This series of blog posts will illustrate how to use DBT with Azure Databricks: set up a connection profile, work with python models, and copy noSQL data into Databricks(from MongoDB). In the first part, we will talk about how to set up a profile when using dbt-databricks python package. Install python package dbt-databricks using pip […]