A Wearable Thumb Device for Fruit Firmness Estimation with Vision-Based Tactile Sensing

Mashood M. Mohsan *,1
Basma Hasanen *,1
Taimur Hassan 2
Lakmal Seneviratne 1
Irfan Hussain 1

*Equal Contribution, 1Khalifa University Center for Autonomous Robotic Systems (KUCARS), Khalifa University, UAE, 2Department of Electrical, Computer and Biomedical Engineering, Abu Dhabi University, UAE

Paper DOI Code Dataset Youtube

Abstract

Recent advancements in non-destructive technologies have enabled precise firmness measurement for various fruits, including kiwifruit. However, existing methods remain limited by high costs, environmental sensitivity, and field application impracticality. This work introduces a novel wearable device for estimating non-destructive fruit firmness, combining human tactile interaction with vision-based tactile sensing and edge computing. Worn on the thumb, the device leverages embodied intelligence, merging intuitive human touch with the precision of a vision-based tactile sensor. A single-board computer processes tactile images locally, enabling reliable operation even in remote environments. The device employs our proposed deep learning model for real-time firmness predictions from a single palpation, minimizing repetitive handling and reducing fruit bruising. Its ergonomic, symmetrical design supports comfortable use on either hand, enhancing usability. Compact and portable, the device integrates essential components within a housing measuring 40 mm × 25 mm × 72 mm and weighing only 135 g. Validated through non-destructive ripeness assessments on ‘Hayward’ Kiwifruit, the device demonstrated a strong correlation between tactile images and firmness values when paired with our proposed model, achieving a coefficient of determination (R2) of 0.89. This study created a dedicated dataset on Kiwi firmness to support model development and validation.

Diagram of the transformer deep learning architecture.
The proposed wearable device for real-time, and non-destructive fruit firmness estimation. The user palpates a Kiwi, and the proposed model processes the VBTS palpation recording to predict firmness in a non-destructive approach.

Comparison with Conventional Firmness Testing Methods

For fruits that do not exhibit significant color changes during ripening, firmness serves as a key indicator of ripeness. Existing methods for firmness assessment include mechanical (rupture and durometer), optical, and vibrational techniques. However, these approaches are often invasive or influenced by environmental factors. Our device integrates human dexterity with AI-driven tactile sensing, offering non-destructive, real-time firmness estimation with improved adaptability and accuracy.
Diagram of the transformer deep learning architecture.

Development of Our Device

The device comprises two modules: a thumb unit with a vision-based tactile sensor (VBTS) and a wrist unit housing the control board and user interface in a 3D-printed casing. Adjustable elastic straps ensure comfort and stability. The hardware integrates a Radxa Zero 2 Pro, OLED display, single button, DIGIT sensor, and is powered by a 15W external power bank. A CAD exploded view highlights its modular internal design.
Diagram of the transformer deep learning architecture.
Diagram of the transformer deep learning architecture.

Dataset Collection

A dataset of 530 paired samples was collected from 106 randomly selected kiwifruits using our device. Each fruit was palpated at five points (P1–P5) to capture regional firmness variations, while ground truth was obtained at two equatorial points (P1–P2 or P3–P4) using a penetrometer. Although soft (0.5 kg/cm²) and firm (3.15 kg/cm²) fruits appeared visually similar, their VBTS signatures differed significantly.
Diagram of the transformer deep learning architecture.

Proposed Architecture

VBTS palpation recordings over n frames are processed by a CNN-based video encoder to extract spatial features. These are linearly projected into 1D representations at each time step and passed to an LSTM to capture temporal dependencies. The LSTM output is fed into a fully connected layer to predict fruit firmness.
Diagram of the transformer deep learning architecture.

Results

The proposed model demonstrated a strong correlation between predicted and reference firmness values, achieving an R² score of 0.89 (Figure, left). The residual analysis (Figure, right) shows a random scatter around zero, indicating no systematic bias and suggesting reliable estimation performance across the entire firmness range.
Diagram of the transformer deep learning architecture.
Diagram of the transformer deep learning architecture.

This project builds on our prior research in tactile sensing and fruit firmness estimation. For further exploration of our methods and insights, refer to:

We encourage readers to explore these works for deeper technical context and complementary advancements.

Acknowledgements

This publication is based upon work supported by the Khalifa University of Science and Technology under Award No. RC1-2018-KUCARS. Some elements of this project’s README design were adapted from YCB-Slide. The website was built using Roman Hauksson’s academic project page template.

BibTeX citation

    @article{MOHSAN2025110593,
title = {A wearable thumb device for fruit firmness estimation with vision-based tactile sensing},
journal = {Computers and Electronics in Agriculture},
volume = {237},
pages = {110593},
year = {2025},
issn = {0168-1699},
doi = {https://doi.org/10.1016/j.compag.2025.110593},
url = {https://www.sciencedirect.com/science/article/pii/S0168169925006994},
author = {Mashood M. Mohsan and Basma B. Hasanen and Taimur Hassan and Lakmal Seneviratne and Irfan Hussain}
}