Introduction

Firebase Genkit is an innovative framework designed to help developers seamlessly integrate generative artificial intelligence (genAI) capabilities into their mobile and web applications, which is an exponentially growing need today. Launched by Google in May 2024, it aims to simplify the process of building, deploying, and monitoring AI-powered features, making it particularly beneficial for developers familiar with JavaScript, TypeScript, and Go. 

This article delves into understanding the core capabilities of Firebase Genkit and covers practical implementation examples through a case study.

What is Firebase Genkit and Why is it Needed?

Firebase Genkit is an open-source framework that provides a structured environment for developers to harness the power of genAI. It is built on the Firebase platform, leveraging its robust infrastructure while allowing flexibility in integrating various AI models and services. Genkit supports both Google’s models and third-party open models, and it is designed to be extensible, allowing developers to customize it as needed. It also integrates with vector databases like Pinecone and PostgreSQL’s pgvector.

Genkit itself is a free framework, but pricing depends on the models and services used in conjunction with it. Costs are determined by the specific AI models, Firebase services, and third-party tools integrated within your application.

Currently, Genkit for JavaScript and TypeScript is in its beta version while Go support is in its alpha phase. The framework is designed to support a wide range of applications, including content creation, text summarization, translation, and more.

Key Features of Firebase Genkit

Firebase Genkit offers several powerful features that enhance the development of AI applications:

Unified API for AI Generation: Developers can use a single API to generate or stream content from various AI models, supporting multimodal input and output, not just text-based large language models (LLMs).

Structured Generation: Genkit allows for the generation of structured objects (like JSON) with built-in validation, simplifying integration with applications. Strongly-typed data can be generated to analyze unstructured text, create content, select tasks, and return results to applications as structured, type-safe objects.

Tool Calling: AI models can invoke functions and APIs as tools, allowing for dynamic task completion based on context.

Retrieval-Augmented Generation (RAG): This feature improves the relevance and accuracy of generated outputs by integrating user data and providing simple APIs for embedding, indexing, and retrieving information.

Prompt Templating: Genkit supports creating effective prompts that include rich text templating, model settings, and tool integration, all within a compact and runnable prompt file.

Local Development and Debugging: The framework includes a browser-based UI for local testing and debugging, enabling developers to experiment with AI components and view traces for observability.

Deployment Flexibility: Genkit can be deployed on Firebase Cloud Functions or Google Cloud Run, allowing for scalable and serverless applications. It also supports any Node.js-compatible deployment platform.

Open Ecosystem of Plugins: Access to pre-built components and integrations through community and Google-built plugins enhances functionality and ease of use.

Setting Up Firebase Genkit

Setting up Firebase Genkit in a Node.js project requires Node.js 20 or later. To get started with Firebase Genkit, follow these steps:

1. Install the Genkit CLI

Install the Command Line Interface (CLI) by running the following command:

npm i -g

This command installs the Genkit CLI in the Node installation directory, enabling its use outside of a Node project.

2. Create a new Node project

mkdir genkit-intro && cd genkit-intro
npm init -y

The ‘main’ field in package.json must be set to ‘lib/index.js’.

3. Initialize a Genkit project

4. Select model

When setting up you can choose one of two models:

Gemini (Google AI): Generate an API key for the Gemini API using Google AI Studio and set the GOOGLE_GENAI_API_KEY environment variable. This is recommended as the simplest way to set up but might be restricted depending on location.

Gemini (Vertex AI): Ensure you have a billing-enabled Google Cloud project, enable AI Platform API, and set additional environment variables like GCLOUD_PROJECT and GCLOUD_LOCATION.

The default options can be chosen for the rest of the questions during setup, which will create a project folder with some sample code.

5. Run and explore Genkit features locally

The following command starts the Genkit Developer UI, allowing developers to test and refine their AI functions, prompts, and queries, as well as debug issues with detailed execution traces:

genkit start

Firebase Genkit can also be used in Next.js apps.

Building AI Workflows with Firebase Genkit

Genkit provides various components for building AI workflows, allowing developers to create complex applications that leverage generative AI capabilities efficiently.

1. Models

Genkit supports the integration of various AI models, enabling developers to choose the most suitable one for their application needs. Models can be utilized for different tasks, such as text and image generation, summarization, and conversation. 

Genkit offers plugins for integrating with models from Vertex AI, Google Generative AI, and Ollama, and also supports several community plugins. Key models include the Gemini family of LLMs (via Google Cloud Vertex AI and Google AI plugins), Imagen2/3 image generation models, Anthropic's Claude 3 LLMs (available through both Google Cloud Vertex AI’s model garden and the Anthropic plugin), the GPT family of LLMs (via OpenAI and Azure OpenAI plugins), the Command R family of LLMs (via the Cohere plugin), the Mistral family of LLMs (via the Mistral plugin), and open models like Gemma 2 and Llama 3 (accessible through Ollama and Groq plugins, with Ollama requiring self-hosting).  Discover more packages by searching for "genkit-model" on npmjs.org.

When integrating a model, parameters like temperature, max tokens, and more can be configured to control the behavior of the model's responses. This flexibility allows developers to fine-tune the AI’s output according to the specific requirements of their applications.

This is how to define a function to generate content using a chosen model:

import { gemini15Flash } from '@genkit-ai/googleai';
const result = await generate({
    model: gemini15Flash,
    prompt: 'What is Product Engineering as a Service?',
});

2. Flows

Flows in Genkit represent a sequence of operations or steps that can be executed together to achieve a specific outcome. They allow developers to define how data moves through their application and how different components interact.

Each flow can have input and output schemas defined using libraries like Zod, which ensures type safety and validation throughout the workflow. This structured approach helps prevent errors and ensures that the data being processed meets the expected formats.

For instance, one can create a flow that takes user input, processes it through one or more AI models, and returns a structured response:

import { defineFlow } from '@genkit-ai/flow';
import { z } from 'zod';

export const menuSuggestionFlow = defineFlow(
  {
    name: 'menuSuggestionFlow',
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (subject) => {
    const llmResponse = await generate({
      prompt: `Suggest an item for the menu of a ${subject} themed restaurant...`,
      model: gemini15Flash,
    });
    return llmResponse.text();
  }
);

This example illustrates how to define a flow that processes input and generates a menu suggestion based on the specified theme.

3. Prompts and Dotprompt

Prompts are essential for communicating with AI models, as they define the context and instructions for the model to generate responses. Prompt engineering goes beyond simply adjusting text. The quality of the output is influenced by the model chosen, the parameters set, and the format requested. Genkit’s dotprompt file format allows developers to organize prompts alongside their code, making it easier to manage and iterate on them.

Dotprompt files can include structured input and output schemas, enabling developers to define the expected data formats clearly. This organization helps streamline the development process and ensures that prompts are easily accessible and modifiable.

Here is an example of a dotprompt file:

---
model: vertexai/gemini-1.5-flash
config:
  temperature: 0.9
input:
  schema:
    properties:
      location: {type: string}
      style: {type: string}
      name: {type: string}
    required: [location]
  default:
    location: a restaurant
---

This file defines a prompt that generates a greeting based on the specified location and style, showcasing the flexibility of dotprompt in managing complex prompts.

4. Tool calling

Tool calling allows AI models to invoke functions and APIs as tools to complete tasks dynamically. This capability enhances the interactivity and functionality of AI applications, enabling models to perform actions beyond simple text generation.

Developers can define tools with specific input and output schemas, ensuring that the model understands how to use them effectively. This feature is particularly useful for applications that require real-time data access or external API calls.

This is an example of defining a tool for looking up weather information:

import { defineTool } from '@genkit-ai/ai';
import { z } from 'zod';

const lookupWeather = defineTool({
  name: 'lookupWeather',
  description: 'Get the current weather in a location.',
  inputSchema: z.object({
    location: z.string().describe('The location to get the weather for.'),
  }),
  outputSchema: z.object({
    temperature: z.number().describe('The current temperature in Fahrenheit.'),
    condition: z.string().describe('A brief description of the weather conditions.'),
  }),
  async (input) => {
    // Insert weather lookup API code
  },
});

In this example, the `lookupWeather` tool is defined to fetch weather data based on a specified location, demonstrating how Genkit facilitates the integration of external functionalities.

5. Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) enhances the relevance and accuracy of generated outputs by integrating user data. This feature allows applications to access and leverage existing information, improving the quality of responses generated by AI models.

Genkit provides APIs for embedding, indexing, and retrieving information from various sources, making it easier to implement RAG in applications. By combining generative AI with retrieval capabilities, developers can create more contextually aware and informative responses.

This is an example of how to implement a simple retrieval mechanism:

import { retrieve } from '@genkit-ai/ai';
import { devLocalRetrieverRef } from '@genkit-ai/dev-local-vectorstore';

const retriever = devLocalRetrieverRef('genkitQA');
const query = 'How do I retrieve relevant documents in Genkit?';

const docs = await retrieve({
  retriever: retriever,
  query: query,
  options: { limit: 5 },
});

6. Evaluation and Monitoring

Evaluation and monitoring are critical components of deploying AI workflows, as they help ensure that the models are performing as expected and delivering high-quality outputs. Genkit provides tools for evaluating generated results across multiple test cases, allowing developers to assess the accuracy and relevance of the AI’s responses.

Monitoring capabilities enable developers to track the performance of AI workflows in real time, providing insights into usage patterns, response times, and potential issues. By exporting telemetry data to Google Cloud Logging and Monitoring, developers can maintain oversight of their applications and make data-driven decisions for improvements.

Deploying AI Workflows with Firebase Genkit

Genkit provides flexibility in deploying AI workflows, allowing the usage of Firebase, Google Cloud Run, or any Node.js platform.

1. Firebase

To deploy a Genkit application on Firebase, its own Firebase Cloud Functions can be used. Genkit integrates seamlessly with Firebase, enabling leverage of its infrastructure and authentication services.

2. Google Cloud Run

Google Cloud Run is another deployment option for Genkit applications. It allows for scalable and serverless deployments, ensuring the AI workflows can handle varying user demands.

3. Node.js Platforms

Genkit is designed to work with any Node.js platform, providing flexibility in choosing a deployment environment. Genkit applications can be deployed on any Node.js-compatible platform that suits the developers’ needs.

4. Authorization and Integrity

When building any public-facing application, protecting the data stored in your system is critical. With large language models (LLMs), extra diligence is necessary to ensure that the model only accesses data it should, tool calls are properly scoped to the user invoking the LLM, and the flow is invoked only by verified client applications.

Firebase Genkit provides mechanisms for managing authorization policies and contexts. All flows can define an `authPolicy` in their configuration. An `authPolicy` is a function that tests if certain criteria (defined by the developers) are met and throws an exception if any test fails. For more details on implementation, check out the documentation.

Implementation Case Study: Adding AI to the Compass App

The Compass app, a travel planning tool, serves as a practical implementation case for Firebase Genkit. Initially offering a standard form-based experience, the team sought to enhance it with AI capabilities. Here is how they utilized Genkit:

Embedding Existing Content: The team used the pgvector extension for Postgres and the textembedding-gecko API from Vertex AI to create embeddings for existing content. This enabled users to search based on what each location is known for or general descriptions.

Semantic Search: They implemented a retriever to search for semantically relevant data based on user queries. Using Genkit’s embed function, they generated embeddings for user queries, which were then compared against the "knownFor" attributes in their database.

Prompt Management: Prompts were organized as dotprompt files within a dedicated directory, allowing for easy testing and iteration through the developer UI. This streamlined the process of refining prompts to improve output quality.

Deployment: The team opted for Cloud Run for deployment, utilizing Genkit’s defineFlow to create HTTPS endpoints for their flows. This integration ensured smooth operation and scalability.

Performance Monitoring: By using the Google Cloud plugin, they exported telemetry data to Google Cloud Logging and Monitoring, ensuring robust performance tracking.

Through these steps, Firebase Genkit enabled the Compass team to efficiently develop and deploy AI-enhanced features, significantly improving user experience.

Conclusion

Firebase Genkit represents a significant advancement in the integration of generative AI into applications. Its comprehensive capabilities, from unified APIs to structured generation and robust deployment options, empower developers to create sophisticated AI features with ease. By leveraging Firebase’s infrastructure, Genkit not only simplifies the development process but also enhances the performance and scalability of AI applications. Its successful implementation in the Compass app showcases its capabilities in enhancing user experience through AI. As the landscape of AI continues to evolve, frameworks like Firebase Genkit will play a crucial role in enabling developers to innovate and deliver powerful solutions.