Databricks AI Playground
Your Quick Start Guide
Introduction
Databricks provides an AI Playground that is a chat-like environment for testing and comparing models. It supports foundation models like Databricks DBRX Instruct or Meta Llama 3, on a pay-per-token-basis. Also, it supports models you’ve created (via the Model Registry with a Serving Endpoint).
The AI Playground reduces time spent on infrastructure setup and model deployment so that data teams can go faster when:
- Comparing models
- Learning and developing skills
- Prototyping to test AI/ML ideas
- Demoing models for clients or stakeholders
As of July 20, AI Playground is in preview and is available in limited regions. For this article, the us-west-2 region was used.
YOUR FIRST PROMPT
To launch AI Playground, select “Playground” from the left navigation menu. The tool launches with the DBRX Instruct model selected and an input box for typing, just like the interface of Gemini or ChatGPT in a browser.
To get started, you can choose a suggestion or enter your own prompt.
ADVANCED FEATURES
There are additional UI components to help with common tasks.
If you want to compare models, click the plus icon on the right and select the model. The sync checkbox keeps the prompts the same for both models.
If you are looking for pricing, hover over the model name. For example, DBRX Instruct is listed as 32.143 DBUs per 1M tokens.
If you are looking for performance metrics, check the gray text below a response. Here’s an example of the metrics after a response is generated:
0.48s to first token + 1.23s | 110 tokens/sec | 136 tokens
If you are looking for a Python or curl code snippet to use the model API, click the bracket icon (next to the gear icon).
If you need to control model parameters, click the gear icon. You can control parameters such as Temperature, Top P, Top K, and Max tokens.
If you need to change the system prompt, enter your text in the System Prompt box. The default system prompt is listed in the Databricks documentation. Interestingly, it includes:
“You avoid stereotyping and provide balanced perspectives on controversial topics. You do not provide song lyrics, poems, or news articles and do not divulge details of your training data.”
EXPERIMENTS
I experimented with two models: DBRX Instruct and Meta Llama 3. I used a prompt to summarize an abstract from a research paper and compared the outputs and the processing metrics.
I also experimented with the system prompt and changed it for both models to the text below. This resulted in different outputs and processing times.
This is the alternate system prompt that I used for testing:
“You are a science communicator. You explain things in an accessible way for someone who does not have a background in medical terminology or signal processing.”
CONCLUSION
In conclusion, the Databricks AI Playground is a great way to compare models without having to set up and maintain infrastructure. I hope you find it helpful for faster experiments and learning.