EXP-0005 — MentraOS: a smart-glasses operating system that actually exists

Smart glasses — wearable computers shaped like normal eyeglasses — have been "almost here" for over a decade. Google tried with Glass; Snap tried with Spectacles; Magic Leap promised AR overlays. None of them caught on, partly because every manufacturer ships a totally different operating system, so a developer who builds a clever app for one pair of glasses has to start over for the next. Imagine if every iPhone app had to be rewritten from scratch to also work on Android, on Samsung's old Bada, on BlackBerry, on Symbian — and on a new manufacturer every six months. That's the state of smart glasses today.

MentraOS is an open-source project that tries to be the common layer sitting between any pair of glasses and the apps a developer might want to write — like Android did for phones, but for glasses. A developer writes a small app once; if MentraOS is installed on a given pair of glasses, the app runs. The project already supports four different commercial glasses (Even Realities G1, Mentra Live, Mentra Mach 1, Vuzix Z100), each with a different mix of features (some have a microphone, some have a camera, some only have a display), and they ship an actual companion app on both the iPhone App Store and Google Play. None of this is vaporware.

Forge is our experiment harness — when a project gets marked with a 🧪 reaction in our #development channel, forge clones it, builds the relevant pieces inside a clean sandboxed container, runs the project's own tests, and writes up what it found.

For MentraOS, forge focused on the most checkable piece: the cloud backend that smart-glasses apps talk to. It cloned the repo, installed dependencies, and ran the project's own protocol/message/handshake tests inside a clean container. All 33 tests passed. That confirms that the contracts the apps and the cloud agree on (how to start a connection, how audio gets requested, how transcription results come back, how errors are reported) actually do what the developers say they do.

What forge could not run in this sandbox: the Android client that lives inside the glasses themselves, the React Native mobile companion app, and any end-to-end test against real hardware. These would each be their own separate experiments with their own sandboxes. But the protocol-and-cloud layer — the most important shared piece — checks out cleanly.

The detailed report below covers what's in the repo, why this project matters, and what we'd verify next.

Status: experimented, result partial → strong. Forge benched the cloud backend slice (cloud-v2); 33/33 tests pass on a fresh checkout. The rest of the monorepo — mobile app, Android smart-glasses client, miniapps SDK — is observable but not all bench-able in a bun-only sandbox.

This is a forge writeup of Mentra-Community/MentraOS at commit 808acbc. The pitch on the box: "Write smart glasses apps once, run them on any device that runs the OS." That sounds like marketing. It is in fact a 674 MB monorepo with 2,032 TypeScript files, 428 Java/Kotlin files (the Android smart-glasses client), 805 markdown files, four+ supported glasses devices, App Store and Google Play apps already shipped, and bun + pnpm + npm lockfiles all coexisting. This is a real product.

TL;DR

• License: MIT. Real OSI license. Cloud, mobile, Android-glasses-client, miniapp SDK, and Mintlify docs all in one repo.
• Scope: 674 MB cloned, 2,032 .ts/.tsx, 428 .java/.kt, 805 markdown. Workspaces under cloud-v2/packages/{auth,cloud-client,core,proxy,runtime,shared}, plus mobile/, asg_client/ (Android), miniapps/, sdk/miniapp-cli, mintlify-docs/.
• Devices. Even Realities G1, Mentra Live, Mentra Mach 1, Vuzix Z100 — each with a published compatibility matrix (display/text, display/images, microphone, speaker, camera).
• Bench evidence. cloud-v2 workspace: bun install clean (87 installs across 85 packages, 302 ms — lockfile already resolved). bun test on shared + auth + runtime/src/protocol packages: 33 pass, 0 fail, 313 ms. Tests cover envelope schemas, audio subscriptions, transcription/translation messages, handshake (init/ack), control (ping/pong), protocol error codes, and the client↔cloud union types.
• Out of scope this run. Mobile React Native app, Android asg_client (needs Android Gradle environment), and end-to-end with a real glasses device. None of those fit a Bun sandbox; they would each be separate forge experiments.

What it is

MentraOS positions itself as a device-agnostic OS for smart glasses, sitting between glasses hardware and apps. The app developer writes against a single MentraOS API — display, sensors (accelerometer, camera), audio — and the OS handles the per-device differences. The compatibility matrix in the repo lists four devices today, each with a different combination of display/microphone/speaker/camera; a single MentraOS miniapp is supposed to work across all of them, falling back gracefully where features are absent (e.g. routing audio through the connected phone when the glasses themselves lack a microphone).

The monorepo organizes this as a federation of concerns rather than one binary:

• cloud-v2/ — backend services. Six packages: auth, cloud-client, core, proxy, runtime, shared. Has its own docker-compose.dev.yml and docker-compose.test.yml, its own bun.lock. Service-shaped. This is what forge actually ran.
• asg_client/ — the on-device Android client for smart glasses. Gradle, Java/Kotlin, archived OTA manifests, the OTA updater itself. 428 source files. Heavy-iron — not bench-friendly in a Bun container.
• mobile/ — the React Native companion app shipped on App Store + Google Play, plus a modules/miniapp workspace that other miniapps depend on.
• miniapps/ — the user-facing apps you can install on your glasses. Each is a workspace.
• sdk/miniapp-cli — the CLI a miniapp developer uses locally.
• mintlify-docs/ — the doc site at docs.mentra.glass, content + license-generation script.
• agents/ — agent harnesses, separate from the OS proper.

The fact that all of this lives in one repo, with consistent ESLint, Prettier, husky+lint-staged pre-commit hooks, and a working postinstall that compiles the @mentra/miniapp mobile module, says something about the engineering discipline of the team. This isn't a hobbyist's project. It is also unusual for a "smart glasses OS" not to actually be a kernel — MentraOS is closer in spirit to "the OS layer for glasses apps" than to a real bare-metal OS, but the framing is intentional and the abstraction works for the stated goal.

How forge bench-tested it

Forge cannot run an Android Gradle build, a React Native mobile build, or a connection to physical smart-glasses hardware. What forge can do is take the cleanest bench-able subset — the cloud-v2 backend workspace — clone it at the pinned commit, install dependencies in a oven/bun:1 container, and run the tests upstream ships.

git clone https://github.com/Mentra-Community/MentraOS.git
cd MentraOS && git checkout 808acbc

# inside oven/bun:1 (digest sha256:e10577f0...)
cd cloud-v2
bun install        # 87 installs across 85 packages, 302 ms (lockfile resolved)
bun test packages/shared packages/auth packages/runtime/src/protocol

Output:

33 pass
0 fail
97 expect() calls
Ran 33 tests across 3 files. [313.00ms]

The 33 tests cover the parts of the system that are unit-testable independently of a glasses device or a running cloud:

• Envelope schemas — every message between client and cloud is wrapped in a strict envelope with type, timestamp, optional id, and major-version pinning. Tests verify the schema accepts matching messages and rejects type/major/timestamp mismatches.
• Audio subscriptions — languageSource (specific and auto, with and without hints), transcription and translation subscription schemas, and the union that routes both and rejects unknown variants. This is the contract for how a miniapp tells the cloud which audio it wants.
• Audio result types — transcriptionToken (minimal valid, with optionals, and invalid), transcriptionData (full, with optionals + tokens, invalid), translationData (same). The cloud → client direction of the audio pipeline.
• Audio liveness — UDP liveness ack payload validates. (The audio path is partly UDP, evidently.)
• Handshake — connection.init (minimal, full, invalid) and connection.ack (minimal, with audio block, invalid). The startup handshake between glasses-client and cloud.
• Control plane — ping/pong (accept empty object, reject extra keys — strict mode).
• Protocol errors — PROTOCOL_ERROR_CODES has the right fatal flags; protocolError payload validates.
• Message routing — every enveloped message validates its own wrapped payload; clientToCloud and cloudToClient unions route their variants and reject mismatches; anyMessage routes both directions and rejects unknown types or wrong-major envelopes; a wrapped message with the wrong payload for its type is rejected.

This is exactly the right shape of test for a protocol-heavy system: the wire format is validated end-to-end in pure-data tests, no servers, no devices, no flakiness. 97 expect() calls across 33 tests in 313 ms is a healthy ratio.

The tests forge did not run in this experiment include cloud-client (looks like integration-style — runtime connection, TTS, reconnect), core (session.service.test.ts), and runtime/services/{maps,audio/providers/soniox}. These appeared to need fixtures or external dependencies (soniox is a third-party speech-to-text provider). Time-boxed to the protocol layer for this run; the rest is a clean follow-up.

What forge could not bench

Three things, all by design, all separate follow-up experiments:

1. Android asg_client. Needs Android SDK + Gradle wrapper inside the container; the upstream gradlew is there but the project bench would need either a cimg/android base image or a hand-rolled installer. Worth a separate forge experiment.
2. React Native mobile/ app. Same problem on the iOS side — would need Xcode for a real build — but the JS side could probably bench under a node:22 + expo container. Another separate experiment.
3. Real-glasses end-to-end. Forge has no hardware. The simulator the README mentions is in mobile/; running it would mean booting a React Native simulator inside the sandbox. Hardware-dependent benches are out of scope.

None of these are forge limitations; they are correct sandbox boundaries.

Why this matters

Smart glasses have had three false starts (Google Glass, Snap Spectacles, Magic Leap). The path forward for the category is uncertain. What is not uncertain is that any successful glasses ecosystem will need an OS layer that abstracts hardware variation, because every device manufacturer ships incompatible APIs. MentraOS is the first credible open-source attempt at exactly that layer, with a real protocol surface that forge could verify in a clean sandbox.

The cloud-v2 backend is bench-perfect: type-safe protocol schemas, pure-data tests, no flakiness, no external dependencies for the layer that matters most. The fact that it's also published on App Store + Google Play with a four-device compatibility matrix is independent evidence that the abstraction is shippable, not just designed.

Comparables

The closest spiritual analogue isn't even another glasses OS — it's Android itself, which is what an open OS for a new device category looks like in retrospect. MentraOS at v1 is much smaller, but it is the right shape.

Reproducibility

Companion gist contains: experiment.yaml, env.json, the full test log, the cloud-v2 README, and the glasses compatibility matrix.

See also

• EXP-0002 — cc-gateway — another MIT-licensed TypeScript project where the protocol/rewriter tests carried the verification.
• EXP-0004 — Road to Machine Learning — a different shape entirely: a curriculum repo where the gap between advertised and actual was the headline finding. Not the case here — MentraOS delivers on what's shown.

Built and verified by forge. Where the project ships its own tests, forge runs them first; where the project's scope exceeds the sandbox, forge says so clearly and recommends the next experiment instead of weakening the bench.

Companion gist (test log, env manifest, glasses compatibility matrix)