HeartLen is an web-based application for real-time heart rate and heart rate variability (HRV) monitoring using camera-based photoplethysmography (PPG). It analyzes subtle color changes in facial skin that correspond to blood flow, enabling contactless cardiovascular monitoring. Note that this is a student assignment for BIOF3003, do not use this for any medical purpose :)
You can play with the online demo of HeartLen
- A smart phone/tablet or other devices with built-in camera with a flash light
- Please refer to Development and Deployment Setup if you are willing to self-host HeartLen
- Navigate to the HeartLen (e.g, https://heartlen-gamma.vercel.app or https://localhost:3000 if you self-host it locally) in your web browser
- When prompted, enter your Subject ID (this is required for data tracking)
- When prompted, allow camera access permissions
- Position your finger on the camera and flash light
The HeartLen interface is divided into three main sections:
Login Page:
- Enter your Subject ID to start a new session or retrieve your historical data
Left Panel:
- Camera Feed displaying real-time video from your camera
- Current Heart Rate, HRV, and Signal Quality
- Last access time
- Con 10000 figuration Options for PPG signal channel combination
Right Panel:
- Real-time PPG Chart to visualize the PPG signal
- Historical Averages that displays your average heart rate and HRV from previous sessions
- Historical Trends Chart to demonstrate the heart rate and HRV changed over time
- Start Recording: Click the "START RECORDING" button in the top-right corner.
- Monitor Metrics: Watch as your heart rate, HRV, and signal quality metrics update in real-time.
- Adjust Position: If signal quality is poor, adjust your position or lighting conditions.
- Save Data: After recording for at least 30 seconds, click "Save Data to MongoDB" to store your session.
- Stop Recording: Click "STOP RECORDING" when finished.
- Click "Show Config" to reveal configuration options.
- Select a signal combination method:
- Default: Balanced approach using a combination of RGB channels (2R-G-B)
- Red Only: Uses only the red channel (may work better in certain lighting)
- Green Only: Uses only the green channel (typically most sensitive to blood volume changes)
- Blue Only: Uses only the blue channel
- Custom: 3R-G-B channel combination
- BPM (Beats Per Minute): Normal resting heart rate for adults ranges from 60-100 BPM.
- Confidence: Indicates the reliability of the current heart rate measurement.
- High confidence (>70%): Measurement is likely accurate
- Low confidence (<70%): Measurement may be unreliable
- HRV Measured in milliseconds, represents the variation in time between heartbeats.
- Higher values (>50ms): Generally associated with better cardiovascular health and stress resilience
- Lower values (<30ms): May indicate higher stress levels or potential cardiovascular issues
- Confidence: Indicates the reliability of the current HRV measurement.
- Excellent: Clean signal with minimal noise
- Acceptable: Mostly clean signal with some minor noise
- Bad: Bad signal with high noise or artifacts
- Ensure your finger covers the entire camera and the flash light
- Sometimes the camera switches randomly, please make sure your fingers are always on the camera being recorded
- Minimize finger movement during recording
- Try a different signal combination in the configuration settings
- Record for at least 30 seconds to allow the algorithm to stabilize
- Check signal quality indicator - only trust readings with "Excellent" or "Acceptable" quality
- Try adjusting the signal combination in the configuration settings
- Check that camera permissions are granted
- Check if you are aceesing through https in case of self-hosting
- Try refreshing the page
- Change another browser
- Node.js (v14+)
- npm or yarn
- MongoDB (local or Atlas)
- Git
-
Clone the repository:
git clone https://github.com/yourusername/heartlen.git cd heartlen -
Install dependency:
npm install
-
Set up environment variables :Create a .env.local file in the project root with the
MONGODB_URI:MONGODB_URI=mongodb+srv://username:password@cluster.mongodb.net/heartlen?retryWrites=true&w=majorityYou may create a MongoDB Atlas account for free-tier cluster and get the from Clusters > Connect > MongoDB For VS Code Ensure the database has a collection named
recordsto store PPG data -
Start the development server:
# Must use https to use the camera on mobile devices npm run dev --experimental-httpsOR
Build and start the production server:
npm run build npm run start
-
Open the application: Open your browser and navigate to
https://localhost:3000
-
Frontend:
- Next.js 13+
- React 18+
- TypeScript
- Tailwind CSS
- Chart.js
-
Backend:
- Next.js API Routes
- MongoDB
-
PPG Quality Assessment Model Backend:
- Tensorflow.js
├── app/
│ ├── components/
│ │ ├── CameraFeed.tsx # Handles video capture and display
│ │ ├── MetricsCard.tsx # Displays health metrics with confidence indicators
│ │ ├── SignalCombinationSelector.tsx # UI for selecting signal processing methods
│ │ ├── ChartComponent.tsx # Visualizes the PPG signal and detected valleys
│ │ ├── HistoricalChart.tsx # Displays historical heart rate and HRV data
│ │ └── Layout/
│ │ ├── Header.tsx # Application header with navigation
│ │ └── Footer.tsx # Application footer
│ ├── hooks/
│ │ ├── usePPGProcessing.ts # Handles camera initialization and PPG signal extraction
│ │ └── useSignalQuality.ts # Evaluates the quality of the PPG signal
│ ├── utils/
│ │ ├── signalProcessing.ts # Signal processing algorithms
│ │ ├── peakDetection.ts # Peak/valley detection algorithms
│ │ └── metrics.ts # Heart rate and HRV calculation functions
│ ├── api/
│ │ └── handle-record/
│ │ └── route.ts # API endpoint for saving and retrieving PPG records
│ ├── models/
│ │ └── Record.ts # Mongoose model for PPG records
│ ├── lib/
│ │ └── mongodb.ts # MongoDB connection utilities
│ ├── layout.tsx # Root layout component
│ └── page.tsx # Main application page
├── types/
│ └── index.ts # TypeScript type definitions
├── styles/
│ └── globals.css # Global styles with Tailwind CSS
├── .env.local # Environment variables (MongoDB connection string)
├── next.config.js # Next.js configuration
├── tailwind.config.js # Tailwind CSS configuration
├── tsconfig.json # TypeScript configuration
├── package.json # Project dependencies and scripts
└── README.md # Project documentation
-
CameraFeed (
/app/components/CameraFeed.tsx)- Purpose: Handles video capture and display
- Props:
videoRef: React ref for the video elementcanvasRef: React ref for the canvas element used in processing
- Functionality:
- Renders video element with appropriate settings
- Positions canvas for frame processing
- Handles camera initialization and cleanup
-
MetricsCard (
/app/components/MetricsCard.tsx)- Purpose: Displays health metrics with confidence indicators
- Props:
title: Name of the metric (e.g., "HEART RATE")value: Object containing metric values (e.g.,{bpm: 75}or{sdnn: 45})confidence: Number between 0-100 indicating measurement reliabilityclassName: Optional CSS classes for styling
- Functionality:
- Renders metric title and value
- Displays confidence indicator with appropriate color coding
- Adapts display based on metric type
-
SignalCombinationSelector (
/app/components/SignalCombinationSelector.tsx)- Purpose: Allows users to select different signal processing methods
- Props:
signalCombination: Current selected methodsetSignalCombination: Function to update the selection
- Functionality:
- Renders radio buttons for different signal combinations
- Provides descriptions of each method
- Updates parent component when selection changes
-
ChartComponent (
/app/components/ChartComponent.tsx)- Purpose: Visualizes the PPG signal and detected valleys
- Props:
ppgData: Array of PPG signal valuesvalleys: Array of indices where valleys are detected
- Functionality:
- Renders a line chart of the PPG waveform
- Highlights detected valleys with markers
- Updates in real-time as new data arrives
-
HistoricalChart (
/app/components/HistoricalChart.tsx)- Purpose: Displays historical heart rate and HRV data
- Props:
historicalData: Array of previously saved records
- Functionality:
- Renders a time-series chart of heart rate and HRV
- Formats timestamps for readability
- Provides tooltips with detailed information
The application uses React's built-in state management with useState and useEffect hooks. Here's a detailed breakdown of the state variables in the main application component:
-
isRecording (Boolean)
- Purpose: Controls whether the camera is active and processing frames
- Interactions:
- Toggled by the START/STOP RECORDING button
- Triggers camera initialization/shutdown via
useEffect - Controls the animation frame loop for processing
-
isSampling (Boolean)
- Purpose: Enables automatic periodic data sampling
- Interactions:
- When true, automatically calls
pushDataToMongoat regular intervals - Used for continuous monitoring scenarios
- When true, automatically calls
-
isUploading (Boolean)
- Purpose: Prevents multiple simultaneous API calls
- Interactions:
- Set to true before API calls begin
- Set to false when API calls complete
- Used to disable the "Save Data" button during uploads
-
signalCombination (String)
- Purpose: Determines which color channels to use for PPG extraction
- Options: "default", "red", "green", "blue", "custom"
- Interactions:
- Updated by SignalCombinationSelector component
- Passed to usePPGProcessing hook to adjust signal processing
-
showConfig (Boolean)
- Purpose: Toggles the configuration panel visibility
- Interactions:
- Toggled by the "Show/Hide Config" button
- Controls rendering of SignalCombinationSelector component
-
lastRecordTime (String | null)
- Purpose: Stores the timestamp of the last saved record
- Interactions:
- Updated by fetchLastRecordTime function
- Displayed in the UI to show when data was last saved
-
historicalAvgs (Object)
- Purpose: Stores average heart rate and HRV from historical data
- Structure:
{ heartRate: number, hrv: number } - Interactions:
- Updated by fetchLastRecordTime function
- Displayed in MetricsCard components
-
ppgData (Array)
- Purpose: Stores the processed PPG signal values
- Interactions:
- Updated by usePPGProcessing hook
- Used for real-time chart visualization
- Sent to backend when saving data
-
valleys (Array)
- Purpose: Stores indices where valleys in the PPG signal are detected
- Interactions:
- Updated by usePPGProcessing hook
- Used for heart rate calculation
- Visualized on the PPG chart
-
heartRate (Object)
- Purpose: Stores the calculated heart rate and confidence
- Structure:
{ bpm: number, confidence: number } - Interactions:
- Updated by usePPGProcessing hook
- Displayed in MetricsCard component
- Sent to backend when saving data
-
hrv (Object)
- Purpose: Stores the calculated HRV (SDNN) and confidence
- Structure:
{ sdnn: number, confidence: number } - Interactions:
- Updated by usePPGProcessing hook
- Displayed in MetricsCard component
- Sent to backend when saving data
-
signalQuality (String)
- Purpose: Indicates the quality of the current PPG signal
- Options: "Excellent", "Acceptable", "Bad"
- Interactions:
- Updated by useSignalQuality hook
- Displayed in MetricsCard component
-
qualityConfidence (Number)
- Purpose: Represents confidence in the signal quality assessment
- Range: 0-100
- Interactions:
- Updated by useSignalQuality hook
- Used for confidence visualization in MetricsCard
-
historicalData (Array)
- Purpose: Stores previously saved records for trend visualization
- Interactions:
- Updated by fetchLastRecordTime function
- Used by HistoricalChart component
File: /app/hooks/usePPGProcessing.ts
Purpose: Handles camera initialization, frame processing, and PPG signal extraction
Parameters:
isRecording: Boolean indicating whether recording is activesignalCombination: String indicating which color channels to usevideoRef: React ref for the video elementcanvasRef: React ref for the canvas element
Returns:
ppgData: Array of processed PPG signal valuesvalleys: Array of indices where valleys are detectedheartRate: Object containing BPM and confidencehrv: Object containing SDNN and confidenceprocessFrame: Function to process a single video framestartCamera: Function to initialize the camerastopCamera: Function to shut down the camera
Please refer to Signal Processing if you are interested in how it works.
File: /app/hooks/useSignalQuality.ts
Purpose: Evaluates the quality of the PPG signal by infering a classifier with Tensorflow.js
Parameters:
ppgData: Array of PPG signal values
Returns:
signalQuality: String indicating quality levelqualityConfidence: Number representing confidence in the assessment
Please refer to the the Jupyter notebook ppg-classifier-auto-tuning.ipynb if you are interested in the details of the model.
- Data Capture and Processing: Camera → videoRef → Canvas → RGB Extraction → Signal Processing → PPG Data
- Camera captures frames at ~30fps
- Frames are rendered to the video element (videoRef)
- processFrame function extracts RGB values from regions of interest
- Signal processing algorithms filter and analyze the data
- Processed data is stored in ppgData state variable
- Metric Calculation:
- Valley detection algorithms identify dips in the PPG signal
- Inter-beat intervals (IBIs) are calculated from the time between consecutive valleys
- Heart rate is computed as 60 / (average IBI in seconds)
- HRV (SDNN) is calculated as the standard deviation of IBIs
- Confidence metrics are derived from signal stability and consistency
- Data Storage: User Action → pushDataToMongo → API Request → MongoDB Storage
- User clicks "Save Data to MongoDB"
- pushDataToMongo function prepares data object with metrics and timestamp
- API request is sent to
/api/handle-recordendpoint - Backend processes and stores data in MongoDB
- Success/failure response is returned to frontend
- Historical Data Retrieval: Page Load → fetchLastRecordTime → API Request → Data Processing → UI Update
- After entering Subject ID, fetchLastRecordTime function is called
- GET request is sent to
/api/handle-recordendpoint - Backend retrieves recent records and calculates averages
- Response data updates lastRecordTime, historicalAvgs, and historicalData states
- UI components re-render with the updated data
- Record Handling: API Request → Request Validation → Database Operation → Response Formation
- API route handler receives GET or POST request
- Request method and body are validated
- For POST: New record is created and saved to database
- For GET: Recent 50 records are retrieved and processed
- Response object is formed with appropriate status and data
- Response is sent back to client
- Data Aggregation: Database Query → Record Filtering → Metric Calculation → Response Preparation
- Database is queried for 50 most recent records that are not 0
- Average heart rate and HRV are calculated and display in metric cards
- Data is structured for frontend visualization
Methods: GET, POST
GET Request:
- Purpose: Retrieve historical data and statistics for a specific subject
- Parameters:
subjectId: The identifier for the subject whose data is being requested
- Response:
{ "success": true, "_id": null, "avgHeartRate": 58.6, "avgHRV": 292, "lastRecordTime": "2025-03-03T17:38:51.113Z", "historicalData": [ { "heartRate": { "bpm": 43 }, "hrv": { "sdnn": 60 }, "_id": "67c5d6e847c20a0430fde691", "timestamp": "2025-03-03T16:20:56.950Z" }, { "heartRate": { "bpm": 50 }, "hrv": { "sdnn": 282 }, "_id": "67c5d6dd47c20a0430fde68c", "timestamp": "2025-03-03T16:20:45.607Z" }, { "heartRate": { "bpm": 70 }, "hrv": { "sdnn": 415 }, "_id": "67c5d6d847c20a0430fde687", "timestamp": "2025-03-03T16:20:40.099Z" }, {...} ] }
POST Request:
- Purpose: Save new PPG record to database
- Body:
{ "subjectId": "S001", "heartRate": { "bpm": 75, "confidence": 85 }, "hrv": { "sdnn": 48, "confidence": 80 }, "ppgData": [0.5, 0.52, 0.54, ...], "timestamp": "2025-03-01T14:30:45.123Z" } - Response:
{ "success": true, "data": { "subjectId": "S001", "heartRate": { "bpm": 37, "confidence": 0 }, "hrv": { "sdnn": 0, "confidence": 0 }, "ppgData": [ 0, 255, 255, ... ], "timestamp": "2025-03-03T17:38:51.113Z", "_id": "67c5e92bb73962f68c0297e5", "__v": 0 } }
Collection: ppgrecords
Schema:
{
subjectId: { type: String, required: true, index: true },
heartRate: {
bpm: { type: Number, required: true },
confidence: { type: Number, default: 0 }
},
hrv: {
sdnn: { type: Number, required: true },
confidence: { type: Number, default: 0 }
},
ppgData: { type: [Number], required: true },
signalQuality: { type: String, enum: ['Excellent', 'Acceptable', 'Bad'] },
timestamp: { type: Date, default: Date.now },
metadata: {
signalCombination: { type: String, default: 'default' },
deviceInfo: { type: String },
sessionDuration: { type: Number }
}
}The application extracts PPG signals from video frames using the following process:
- Color Channel Extraction
- Uses 5 sampling points on the video frame: top-left, top-right, center, bottom-left, bottom-right
- For each sampling point:
- Extracts RGB values
- Accumulates R, G, B values separately (rSum, gSum, bSum)
- Validates coordinates to ensure they are within canvas bounds
- Signal Filtering
- Maintains a sliding window of 300 most recent PPG signal values
- Scale all singal values to a 0-1 range
- Valid sample counting ensures that only frames with sufficient data points are processed
- Valley Detection:
- A window size of 0.5 seconds (based on the current FPS) is used to determine the context for each potential valley point
- Set a minimum distance of 0.4 seconds between detected valleys to prevent false positives
- Each detected valley is recorded with its timestamp, signal value, and index in the data array
- The valley detection algorithm works on normalized signal values to improve consistency across different lighting conditions
- Heart Rate Calculation
- The heart rate calculation requires at least two detected valleys to establish a meaningful interval
- Time intervals between consecutive valleys are calculated and converted to seconds for processing
- Filters for valid intervals between 0.4 and 2.0 seconds, corresponding to heart rates between 30 and 150 BPM
- The median interval is used to calculate the final BPM value
- A confidence score is calculated based on the Coefficient of Variation, where lower variation results in higher confidence
- HRV
- The Standard Deviation of NN intervals is calculated as the primary HRV metric
- It requires at least two valleys and filters for valid RR intervals between 250 and 2000 milliseconds
- Finds the mean of all valid intervals and then computing the standard deviation
- The confidence score is calculate by adding up both the quantity of valid intervals and their consistency
The application integrates a 1D CNN-based auto tuned machine learning model for signal quality assessment. The model is trained on a dataset of PPG signals and their corresponding quality labels. It's then used to predict the quality of a given PPG signal with a length of 300 data points.
For detailed dataset collection, labeling, and model training, please refer to the the Jupyter notebook ppg-classifier-auto-tuning.ipynb in the root directory, there are detailed instructions on each step.
As long as you are not modifying the format of input data, which is a 300-length array of floating point numbers, and not modifying the output labels, you should be able to train and integrate the model as following:
-
Run auto-tune to obtain the optimal model:
# Tuner initialization tuner = kt.Hyperband( build_tunable_model, objective='val_accuracy', max_epochs=200, factor=3, directory='auto_tuning', project_name='ppg_cnn' ) # Early stop early_stop = tf.keras.callbacks.EarlyStopping( monitor='val_loss', patience=20, restore_best_weights=True ) # Tuning tuner.search( X_train, y_train, epochs=200, validation_split=0.2, callbacks=[early_stop], batch_size=32, verbose=1 ) best_model = tuner.get_best_models(num_models=1)[0] best_hp = tuner.get_best_hyperparameters()[0] print("Best Hyperparameters:") for hp, value in best_hp.values.items(): print(f"{hp}: {value}") # Evaluation of the best model test_loss, test_acc = best_model.evaluate(X_test, y_test, verbose=0) print(f"\nOptimized Test Accuracy: {test_acc:.4f}") print(f"Optimized Test Loss: {test_loss:.4f}") # Save the optimal model best_model.save('optimized_cnn_model.h5') tfjs.converters.save_keras_model(best_model, 'tfjs_model')
Then you should able to find a
tfjs_modelfolder in the root directory, which contains the optimized model. -
Replace
public\tfjs_modelwith the model folder generated by the auto-tuning process.