Aura Build Manual
A structured reference for sourcing the pendant, flashing firmware, running the backend, and pairing the app.
Build Path
- Source the bill of materials and confirm the pendant case parts.
- Assemble the XIAO ESP32-S3 Sense hardware with camera, microphone, battery, and enclosure.
- Flash the firmware and verify serial, camera, BLE, and Wi-Fi.
- Deploy the backend with transcription, model routing, storage, and retrieval configured.
- Pair the companion app, test capture, and review privacy controls.
What Is Aura
Aura is an open-source AI wearable you wear around your neck. It listens, sees, and transcribes so you do not have to pause your day to take notes.
Audio and images are captured and routed to a self-hosted or cloud backend. Summaries and transcripts are accessible from your phone.
Built on the open Omi ecosystem. Parts cost ~$50.
How It Works
The pipeline is straightforward:
- The microphone captures audio continuously.
- The camera captures images at set intervals.
- Both are sent to the backend over Wi-Fi.
- The backend transcribes audio via Deepgram or Whisper.
- Images are analyzed via GPT-4o Vision or Moondream.
- Everything is summarized and stored.
- You see it all in the Omi app on your phone.
The ESP32-S3 handles capture and transmission. The backend handles all AI processing. Your phone is the interface.
Bill of Materials
Parts list with estimated costs:
| Item | Where to Buy | Approx Cost |
|---|---|---|
| XIAO ESP32-S3 Sense | Seeed Studio / Amazon | $15–24 |
| 150mAh LiPo × 6 | Amazon | $12 |
| Wires | Amazon | $5 |
| 3D printed case | Print yourself or order online | $5–10 |
| USB-C cable | Anywhere | Free |
| Total | ~$50 |
Case STL files are in the Aura hardware folder on GitHub.
Safety: Use only LiPo cells with protection circuits. Never charge unattended. Follow the XIAO ESP32-S3 Sense charging current limits (500 mA default).
Hardware Overview
The XIAO ESP32-S3 Sense is the core. Camera and microphone are integrated on the board, so no extra modules are needed.
| Component | Details |
|---|---|
| Microcontroller | XIAO ESP32-S3 Sense |
| Camera | OV2640 (built into board) |
| Microphone | PDM (built into board) |
| Battery | 6× 150mAh LiPo cells |
| Enclosure | Custom 3D printed case |
| Connectivity | Wi-Fi 2.4 GHz + Bluetooth LE |
| Dimensions | 50 × 68 × 18 mm |
| Weight | 80 g |
| Battery Life | 4 h active / 45 min charge |
Mount the board, route the battery wires, and snap the case shut. The pendant loop is built into the design.
Flashing Firmware
Clone the firmware repo and flash using PlatformIO:
git clone https://github.com/thesohamdatta/aura.git cd aura/firmware pio run --target upload
Connect the board via USB-C, select the correct port, and upload. Verify with the serial monitor:
pio device monitor --port COM3
You should see the boot log with Wi-Fi, camera, and BLE initialization.
Backend Setup
The backend handles transcription, image analysis, and memory storage. Deploy with Docker:
git clone https://github.com/thesohamdatta/aura.git cd aura/backend cp .env.example .env # Edit .env with your API keys docker compose up -d
Required services:
- Deepgram or Whisper for transcription
- Groq or OpenAI for LLM inference
- Pinecone for vector memory storage
Companion App
The Omi app connects to Aura over Bluetooth LE. It shows your conversation history, memory timeline, and device controls.
Download the app, create an account, and pair your device. The setup wizard will guide you through the process.
Once paired, you can browse transcripts, search memories, and adjust capture intervals.
AI Providers
Aura supports multiple AI backends. Configure your providers in
the .env file:
| Service | Purpose | API Key Required |
|---|---|---|
| Deepgram Nova-2 | Speech-to-text | Yes |
| Groq LPU | Fast LLM inference | Yes |
| GPT-4o Vision | Image analysis | Yes |
| Pinecone | Vector memory | Yes |
| Whisper (local) | Self-hosted transcription | No |
Memory & RAG
Aura uses retrieval-augmented generation to make conversations searchable:
- Audio is transcribed, chunked, and embedded into Pinecone.
- When you ask a question, the query is matched against stored vectors.
- Matched context is fed to the LLM along with your query.
All memory is private and stored on your own Pinecone index.
Troubleshooting
Device not connecting to Wi-Fi
Verify SSID and password in the firmware config. Confirm 2.4 GHz band is enabled; ESP32-S3 does not support 5 GHz.
Camera not capturing images
Ensure the camera ribbon cable is fully seated. Try re-flashing the firmware. The OV2640 may need a power cycle.
Audio transcription failing
Check your Deepgram API key in .env. Verify network
connectivity from the backend to Deepgram's API endpoint.
Battery not charging
The ESP32-S3 charges at 500 mA max. Use a quality USB-C cable and power supply. Check voltage with a multimeter.
FAQ
How much does it cost to build?
The bill of materials runs roughly $50 for all parts, including the XIAO ESP32-S3 Sense, six LiPo cells, wires, and a printed case.
Do I need to buy a case?
The case is 3D printed. Print it yourself from the STL files in the hardware folder, or order the print from an online service.
Can I run the backend without cloud services?
Yes. Use the local Whisper model for transcription and any OpenAI-compatible server for inference. Pinecone is optional for vector memory.
Is my data private?
You own your data and your device. Run the backend on your own hardware. Audio and images never leave your network, or use the cloud backend you deploy and control.
Is the Aura software open source?
Yes. Aura is MIT licensed. Firmware, backend, and app code live in the public aura repository on GitHub.