GENIE

A safe place for state employees to try generative AI.

What I ownedOwnership: Core contributor on a team build

Core engineer on the Burnes team’s build, and the author of Smart Model, the router that picks a model for each request.

44,000+State employees with access

Year: 2024
Role: Co‑op engineer, author of Smart Model
With: The Burnes Center for Social Change
Category: Government AI

GENIE launch film: text alternative Length: 19.5 seconds. Landscape and vertical cuts share one timeline. Audio: a quiet instrumental music bed with soft interface sounds. No voiceover and no spoken words. [0:00 - 0:03] Hook On screen: "MASSACHUSETTS STATE EMPLOYEES" On screen: "A safe place to try generative AI." (the word "safe" is set in an italic serif) On screen, typed live into a chat box: "help me write a LinkedIn post announcing the bill, make sure to include the summary (keep the reading level to grade 6)" Chat box labels: "Send a message", "Smart Model", "No workspace (RAG data source)" (landscape only), "Send" Visual: a dark screen with a violet glow. The GENIE message box types a prompt letter by letter while the headline rises into place above it. [0:03 - 0:08] Multi-chat On screen: "MULTI-CHAT" On screen: "Two to four models, side by side." App labels: "Llama 3 70B Instruct", "Mixtral 8x7B Instruct", "Command", "State-Bills-Recent" (landscape), "Welcome to the AI Sandbox" (landscape), "Connected", "Show Metadata", "Add model", "Clear messages" (landscape), "Metadata" Sample answers streaming in word by word, one per model: Llama 3 70B Instruct: "We have news to share. A new bill was announced this week, and it aims to make state services easier to use." Mixtral 8x7B Instruct: "Today we are sharing a new bill. In plain words, it is about making state services easier to find and use. Summary below." Command: "Announcing the bill. In short, it makes services easier to reach for everyone. Read the summary and share your thoughts." Visual: a cursor clicks Send. The same prompt goes to three models, shown side by side (landscape) or stacked one per row (vertical). The three answers appear at different speeds. The sample answers are illustrative text, not recorded model output. [0:08 - 0:13] Smart Model On screen: "SMART MODEL" On screen: "Smart Model picks the model so you don’t have to." Diagram labels: "REQUEST", "help me write a LinkedIn post announcing the bill" (landscape adds ", make sure to include the summary"), "Smart Model", "CHOOSES BY", "Task", "Cost", "Token size", "Llama 3 70B Instruct", "Mixtral 8x7B Instruct", "Command" On screen: "17", "Bedrock models, one picks for you" Visual: a request card flows into a panel that ticks off three criteria one at a time. One model, Mixtral 8x7B Instruct, lights up with a check mark. The routing diagram is a schematic of how Smart Model chooses, not a recording of a real request. Then "17" counts up from zero. [0:13 - 0:16.5] Reach and recognition On screen: "ACCESS" On screen: "44,000+" On screen: "State employees with access." On screen: "GOVERNOR’S CITATION · JUNE 26, 2024" Visual: the number counts up to 44,000+ beside the real Governor’s Citation, a certificate from the Commonwealth of Massachusetts, which tilts slowly while a soft band of light passes across it. [0:16.5 - 0:19.5] End card On screen: "GENIE" On screen: "Built by Dhruv Kamalesh Kumar" Visual: the name GENIE in large white-to-lavender letters on a dark violet glow, a small orange dot beside the credit line, then a fade to black.

01/The problem

State employees wanted AI. The state wanted it safe.

Massachusetts state employees wanted to use generative AI, but pasting government work into a public chatbot is a security and cost problem. They needed a sandbox with guardrails.

02/What I built

One sandbox, many models, and a router that picks for you.

01
Built with the Executive Office of Technology Services and Security and AWS on top of the open source AWS GenAI chatbot.
02
A chat playground, a multi-chat view that compares two to four models side by side, a task carousel of ready-made prompts, and RAG workspaces where staff upload their own documents.
03
My own piece was Smart Model, an LLM router. One Claude call reads each prompt beside a sheet of every model’s languages, token limits, use cases and prices, and names the model to use, so beginners never have to know which to pick.
04
The sandbox also hosted the RIDE Guide and MassHealth Helper builds from the same co-op cohort.

03/Architecture

Every request finds the right model.

Sign in, send a prompt, a Claude call picks the model from a capability sheet, the answer streams back, and RAG workspaces ground it in uploaded documents.

04/Outcomes

Forty-four thousand people with access.

44,000+
State employees with access
The Burnes Center reports that the sandbox enabled over 44,000 state employees to explore AI. That is access, not measured daily use.
Employer-reportedSource: burnes.northeastern.edu
17
Bedrock models Smart Model routes across
One router call reads the prompt against a capability sheet of task fit, cost and token size, and picks the model. Simple requests can go to models up to 12 times cheaper per request than the default: Claude 3 Haiku against Claude 3.5 Sonnet at Bedrock list prices, for 1,500 tokens in and 500 out. The list is in the router's own code.
From the repoSource: github.com
1
Governor’s Citation
Signed by Governor Healey and Lieutenant Governor Driscoll on June 26, 2024.
Third party

Check these in the ledger

05/Production notes

Behind the numbers

All receipts for this project

Mechanism

Smart Model is an LLM router, not a rules table or a trained classifier. In the version I wrote, one Claude 3 Sonnet call on Bedrock reads the user’s prompt beside a sheet on each model (token limit, languages, use cases, price per 1,000 tokens, benchmark scores) and answers with one model name from a fixed list. A rule in that prompt sends any language other than English to Claude 3 Sonnet.

Fallback

In that version, a failed router call, a name that is not on the list, or an unreadable sheet all send the request to Claude 3 Sonnet. The chosen model is logged for every request.

Cost

Not measured

I never measured a saving, so none is claimed. Picking a cheaper model for simple prompts saves money, but the router call itself runs on Claude Sonnet and reads the whole capability sheet, so on a typical request it costs about as much as it saves. A small, cheap router model is the first fix I would make.

Evaluation

Not measured

The repo has no tests and no routing evaluation, so nothing shows that routing kept answers as good as always using one model.

Evidence

Private repo

The Burnes Center repo is private, so the router is described here and not linked.

06/Stack