Frogmote is a Windows desktop application that transforms a smartphone into a gesture‑based remote control. It streams real‑time sensor (gyroscope) data over WebSockets, recognizes swipe gestures, and maps them to system or media actions on the desktop.
- Features
- Prerequisites
- Installation & Build
- Running the App
- Configuration & Presets
- Code Structure
- Core Components
- Extending Frogmote
- Known Issues
- 🎛️ Gesture‑driven Control: Detects four swipes (left/right/up/down) via gyroscope data.
- 🔌 WebSocket Communication: Phone runs the Sensor Server app and streams data to the desktop client.
- 🗂️ Dynamic Presets & Cards: Organize gesture–action mappings into up to 4 preset panels.
- 💾 Persistent Storage: Saves and loads configurations automatically from
presets.dat. - 💡 Immediate Feedback: System tray notifications confirm connection status and actions taken.
- 📸 Screenshot Support: Capture screen via gesture, saved with timestamped filename.
- Qt 6 (Core, Widgets, WebSockets modules)
- C++17‑compatible compiler (MinGW, MSVC, Clang)
- Sensor Server app installed on your smartphone (Android or iOS)
- CMake ≥ 3.16 or qmake (Qt) for build system
- Clone the repository:
git clone https://github.com/yourname/frogmote.git cd frogmote - Generate Makefile and build:
qmake Frogmote.pro make # or mingw32-make on Windows
mkdir build && cd build
cmake .. -G "MinGW Makefiles" # or your preferred generator
cmake --build .- Start the Sensor Server app on your phone, note its IP:port (e.g.,
192.168.1.50:8000). - Launch Frogmote:
./Frogmote # or Frogmote.exe on Windows - In the URL bar, enter
ws://<IP>:<port>and press Enter. - Select one of the 4 presets and configure gesture cards by clicking the + button.
- Perform swipes on your phone; mapped actions will execute on the desktop.
-
presets.dat: Located in the application working directory. First line lists card counts for each preset, followed by lines ofgesture actionpairs per card:2 3 0 1 ← preset counts (e.g. 2 cards in preset 1, 3 in 2, etc.) 1 4 ← card1: Gesture=1 (LEFTWARD_SWIPE), Action=4 (FORWARD) 2 1 ← card2: Gesture=2, Action=1 ... -
The file is loaded at startup and saved on exit via
LoadPresets.
│
└─ Remote/
├─ mainwindow.* # Builds main window, layouts, preset-switch logic
├─ preset.* # Manages one preset panel: Card widgets + Add button
├─ card.* # Individual gesture→action UI element
├─ loadpresets.* # Loads/saves `presets.dat` data model
├─ client.* # WebSocket client, parses incoming JSON
├─ gesturehandler.* # Gesture detection state machine
├─ actioncontroller.* # Abstract interface + Windows implementation
├─ urlbar.*, urltextedit.*, urlcheckmark.* # Validated URL input widget
├─ notifiable.* # System tray notifications
├─ res/ # PNG/SVG assets: icons, frames, backgrounds
└─ build/...
└─ presets.dat # Generated on first run or on exit
- Sets up top URL/settings bar and bottom split: preset selector and scrollable card area.
- Instantiates 4
Presetobjects, each with its ownstd::vector<Card::data>. - Calls
enable()/disable()on presets when the user switches tabs.
- Derives from
QPushButtonfor the preset tab. Contains:- A QGridLayout displaying
Cardwidgets. - A reusable Add Card button that spans both columns.
- A QGridLayout displaying
enable(): populates cards from data vector and displays the “+” at the bottom center.disable(): hides “+” and deletes card widgets, preservingaddCardButtonitself.- Slot
onAddCardClicked(): appends default entry to data vector, creates newCard, repositions “+”, and callsupdateData().
- A custom
QWidgetshowing:- Two
QComboBoxdropdowns for gesture and action selection. - Icons previewing the selected gesture/action.
- Painted card frame via
paintEvent().
- Two
- Signals trigger updates to internal
Card::datapointer.
- Reads/writes
presets.datin text format. - Maintains a
std::vector<std::vector<Card::data>>for up to 4 presets.
- Wraps
QWebSocketto connect to the phone’s sensor server. - Parses incoming JSON, forwards numeric arrays to
GestureHandler.
- State machine with phases:
STANDBY,GESTURING, etc. - Computes vector deltas over time to detect swipe direction.
- Emits
gestureDetected(Gesture)when a motion completes.
- Abstract interface defines methods:
play(),mute(),next(), etc. - Windows implementation uses
keybd_event()and Qt notifications. updateData()points the controller at the current preset’s vector, used byactionDispatch(Gesture).
- Composite widget for WebSocket URL input.
UrlTextEdit: masked input & customQValidatorto restrict invalid IP/port.UrlCheckMark: shows connection status via a toggled checkmark icon.
- New Gestures:
- In
GestureHandler.cpp, analyze the diff vector to detect new patterns. - Add enum in
gestures.h. - Update dropdown in
Cardto include new gesture names.
- In
- New Actions:
- Add enum in
actions.h. - Implement behavior in
ActionControllerWindows(or create other platform subclass). - Update dropdown in
Cardto include new action names.
- Add enum in
- Gestures can be sensitive to small jitters. Tuning thresholds may be required.
- macOS/Linux action implementations are not yet provided (Windows only).
- It is currently impossible to delete a card without modifying the
presets.datfile manually. - The settings button does not work yet.