# How to build a Meta Muse gadget with the open-source SDK

Source: https://www.humanstoknow.com/blog/meta-muse-gadget-sdk

Turn a Raspberry Pi or supported ESP32 board into a Muse gadget. Understand what runs locally, pair the SDK, and choose community skills for the devices you already own.

Author: Akshay Apsingi
Published: 2026-10-03T01:17:55.735Z
Updated: 2026-10-03T01:18:07.545Z

## TL;DR

- Choose Linux for a Raspberry Pi workflow and supported ESP32 firmware for custom hardware.
- Local gadget software still connects to Muse; the public SDK does not supply a self-hosted Muse model.
- The skills folder contains 43 community recipes, including a shared Google Cast procedure.
- Use the device recipe’s capability checks and observed state to verify the requested result.

![Diagram showing Linux and ESP32 gadget runtimes, the Muse app/token/VM connection, and the collection of 43 community device recipes.](https://www.humanstoknow.com/brand/muse-gadget-cover-architecture.png)

The device code runs locally. The documented connection still uses the Muse app, an SDK token and a Muse VM.

Your Raspberry Pi already runs the backup job. You still open a terminal to find out whether it finished. A useful first Muse gadget could close that small gap: let your agent inspect the machine and return the answer where you already talk to it.

Meta's open-source **Muse Gadget SDK** gives builders two starting points: firmware for supported ESP32 boards and a Linux client for machines such as a Raspberry Pi. Both connect local hardware to Muse. The repository is a better starting point than a shopping list because it shows what each device can actually do, how pairing works, and where to add your own behavior. [Official SDK overview](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/README.md).

This guide is based on source inspection at commit `468e5ba`, checked on 3 October 2026. The setup below follows the upstream documentation; it has not been tested on a physical device.

## What runs locally, and what stays with Muse?

The gadget firmware or Linux client runs on your hardware. It pairs with the Muse app and connects to your Muse VM, which is the agent's execution environment. The public repository documents that connection; it doesn't provide a supported independent installation of the Muse AI service. [Linux architecture](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/AGENTS.md#what-this-is).

The similar names describe different things:

| Name | What it means for your build |
|---|---|
| Muse | Meta's personal AI agent, with its own cloud VM and browser. |
| Muse Spark | The model family powering Muse; opening the gadget SDK does not release Spark model weights. |
| Muse Gadget SDK | The local device code you can inspect and modify. |
| Muse Home Link | Meta's USB-C bridge to compatible devices and APIs on your home network. Its shipped firmware cannot be reflashed. |
| Muse Charm | Meta's separately announced pocket device for interacting with Muse. |

Sources: [Meta's Muse announcement](https://about.fb.com/news/2026/09/introducing-muse-personal-ai-agent/), [SDK README](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/README.md), [Home Link](https://gadgets.muse.ai/home-link), [Muse Charm announcement](https://www.meta.com/blog/meta-connect-2026-everything-we-announced/).

If you searched for a self-hosted Muse gadget, that distinction matters: you control the local device software, while the documented agent connection still depends on Muse, an SDK token, and app pairing.

## Choose the route that fits the task

Start with the hardware you have and one result you want it to produce.

| Starting point | A sensible first task | Read first |
|---|---|---|
| Raspberry Pi or Linux computer with Bluetooth LE | Report device health or inspect an existing job | `linux/README.md`, then `linux/AGENTS.md` |
| Supported ESP32 board | Build a small display, voice interface, or board-specific sensor gadget | `esp32/README.md` and the device feature table |
| Muse Home Link and an existing compatible device | Read its state, then try one supported control | The community skills catalog and the matching recipe |

ESP32 support varies by board. The repository lists 11 profiles, with different display, audio, sensor and home-network tunnel capabilities. For example, the Waveshare ESP32-C6 display profile has no network tunnel and returns voice replies as scrolling text. Check that table before buying a board for a specific role. [Supported devices and features](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/esp32/devices/README.md#features).

For an existing light or plug, choose a recipe that matches the exact hardware. The [Philips Hue bridge skill](/company/meta/skill/gadget-philips-hue-bridges) covers CLIP v2 lights and existing scenes, while the [Shelly plug skill](/company/meta/skill/gadget-shelly-plugs) checks device generation and switching or metering capabilities. A familiar brand name does not establish that your device exposes the required local API.

Every DIY gadget built with this SDK needs an SDK token. Source licensing and service access are separate: the project uses Apache 2.0 with listed exceptions, while the current token terms restrict access to personal, non-commercial use. Review those terms before planning a product around it. [License scope](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/README.md#license), [SDK token terms](https://gadgets.muse.ai/sdk-terms).

## Pair a Raspberry Pi before adding custom behavior

The Linux route makes the connection easiest to inspect. Upstream supports a Bluetooth-equipped Pi 3B+, 4, 5 or Zero 2 W, or another Linux machine with Bluetooth LE. It names Raspberry Pi OS Bullseye+, Debian 11+ and Ubuntu 22.04+, with the machine already online and an account that can install the service using sudo. You also need a Muse SDK token and the phone app. [Linux prerequisites](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/README.md#what-you-need).

1. **Choose the account before installing.** Shell and file operations receive that account's permissions. If it can use sudo, Muse can too. Run the commands below as the account you intend Muse to use, or select another account with the installer's documented `--run-as` option. [Linux access model](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/README.md#what-muse-can-do).

2. **Inspect the version you intend to install.** The quickstart fetches from `main`. For the source version reviewed here, clone the repository, select the commit, and use the installer's documented local-source option:

   ```sh
   git clone https://github.com/facebookincubator/muse-gadget-sdk.git
   cd muse-gadget-sdk
   git checkout 468e5ba629f988ad107ba472555a8b632faac0d3
   cd linux
   less install.sh
   bash install.sh --from . --sdk-token mgst_REPLACE_WITH_YOUR_TOKEN
   ```

   The final command installs a persistent service. Read the installer before running it and replace the placeholder only on your chosen machine. [Installer and local-source option](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/install.sh).

3. **Pair in the app.** Enable **Settings → Devices → Developer mode**, then **Add Device**. Select the `MuseGadgetXXXXXX` name printed by the installer. Confirm the community device is yours. When asked about Wi-Fi, select the network shown; Linux is already connected. Pairing stays open for ten minutes, and `sudo musegadget pair` reopens it. Pair on a network you trust; community-device pairing has no manufacturer verification. [Pairing steps](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/README.md#set-it-up-with-muse).

4. **Prove one read works.** Ask Muse to read the Pi's device health and report its uptime, memory and disk status. The client advertises a built-in `device.health` command for those readings. If the request fails, inspect `musegadget info`, `sudo systemctl status musegadget`, and the service log with `sudo journalctl -u musegadget -f`. Record the actual result before adding another component. [Built-in commands and management](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/README.md).

## Turn the connection into a useful workflow

For the backup example, define what “finished” means before asking the agent to check it. A successful exit code, a recent output file, and a completed-job timestamp answer different questions. Give Muse the actual job and output location, then ask for a read-only check of the signal you trust. The Linux SDK's existing `system.run` and file commands provide the building blocks; the repository doesn't supply a universal backup checker. [Command implementation](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/src/musegadget/executor.py).

The same read-first approach applies to equipment already on the network. The [Moonraker 3D printer skill](/company/meta/skill/gadget-moonraker-3d-printers) starts from an existing, commissioned Moonraker service and reads printer and job state. A printer-status request needs no new firmware or arbitrary G-code, and the recipe keeps those tasks outside its scope.

If the check becomes a repeatable tool, the extension point is concrete: add its command specification to `COMMAND_SPECS` and a handler in `Executor.run`, test the returned result and errors, then restart the service so the command is registered. Upstream's `linux/AGENTS.md` explains that path for a coding agent. [Adding a command](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/linux/AGENTS.md#adding-a-command).

The useful output is “the last backup completed at this time, based on this evidence.” Decide what the tool returns when that evidence is missing, stale, or unreadable. A clear failure is more useful than an agent confidently calling an unchecked job successful.

## What the SDK's skills folder actually contains

At the reviewed commit, `skills/` contains **43 community recipes**: 42 device or device-family recipes and one shared Google Cast procedure. They are explicitly unofficial integrations. Each Markdown file describes how to identify a compatible device, establish prerequisites, perform supported operations, verify the outcome, and respect its limits. They aren't general instructions for building an ESP32 gadget. [Skills README](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/skills/README.md), [catalog](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/skills/CATALOG.md).

Upstream suggests giving Muse the repository link, or copying the matching `SKILL.md` into chat with any shared skills it references. The Google Cast recipe shows why those dependencies matter: it starts from a freshly identified receiver, reads its current state, checks which controls it supports, then verifies the requested result. A “smart TV” label alone doesn't establish Cast support, and Cast doesn't provide general TV navigation or input selection. Our [shared Google Cast skill](/company/meta/skill/gadget-google-cast) retains that protocol procedure and adds the SDK context needed to reuse it. [Google Cast recipe](https://github.com/facebookincubator/muse-gadget-sdk/blob/468e5ba629f988ad107ba472555a8b632faac0d3/skills/gadget-google-cast/SKILL.md).

There is a public-source gap to check before reusing these files: all 43 recipes reference a shared `home_link.md`, but that file is absent from the reviewed repository. The recipe text alone therefore doesn't supply all its referenced networking and safety instructions. If your current Muse environment supplies that guidance, read it; don't invent missing commands.

Our [Muse Gadget SDK collection](/company/meta/collection/muse-gadget-sdk) preserves the device workflows and adds a repository-context reference to each package. That editorial supplement explains a concrete alternative: run a compatible local protocol client on a paired Linux Home Link through its actual `system.run` command, using that account's network access and permissions. It links to pinned SDK documentation and identifies evidence needed to verify the result. It does not reproduce the absent upstream document or establish an official integration.

## Choose the device skill by the result you need

The collection is useful when it helps an agent choose a narrower workflow. A gateway recipe, a media protocol and a TV remote are different tools, even when the devices share your home network.

For a sensor connected through an existing gateway, start with the [Zigbee2MQTT skill](/company/meta/skill/gadget-zigbee2mqtt-gateways). It resolves the broker, base topic and current exposed capabilities before querying or setting anything. For a temperature reading, distinguish a new measurement from a cached value carried in a new message. For a blind, check whether its expose actually permits intermediate positioning; an optimistic position update does not prove physical movement.

For hardware already running ESPHome, use the [ESPHome device skill](/company/meta/skill/gadget-esphome-devices). It works with entities exposed by the installed firmware and resolves their live identifiers and capabilities. That is a different task from compiling or flashing a new ESP32 Muse gadget: the recipe does not make firmware installation a prerequisite for reading an existing entity.

For television navigation, the [Apple TV 4K skill](/company/meta/skill/gadget-apple-tv-4k) uses a compatible local client and normal on-screen pairing for supported remote keys, installed apps and text input. Choose it when the requested result is a remote-control action, rather than loading media through Cast. Confirm supported features on the actual device, and distinguish an accepted key command from the screen result you can observe.

In each case, give the agent the complete package: SKILL.md and the supporting repository-context reference. The source notices identify the upstream recipe and our additions. Inspect current device identity and credentials, perform the requested supported operation, and retain the evidence that answers the user's question. Our directory review checks the instructions and their sources; it does not claim these devices were tested on a physical bench.

For a first build, pair the device, verify one observable read, and make one recurring question easier to answer. Once that works, the repository's command handlers and community recipes give you specific places to extend it.