Two Experimental Tests
					Note: The tasks described below can be seen in this MOVIE!! (45 MB)

Telemanipulation Task Descriptions

A diverse set of ten subjects were asked to perform two prototypical manipulation tasks in order to evaluate the performance of the system. Each subject had the CyberGlove® calibrated to their hand, and the mapping to the robot hand (object-based) was customized to best allow the robot fingers to follow the motions of the subject'sfingers.

Subjects performed each task twelve times, six with force feedback from the CyberGrasp® and six without it in a pattern of AAAA-BBBB-AA-BB, where A is one force mode (on/off) and B is the other. The pattern was selected in an effort to minimize the effect of learning. Each subject has a trial with each force mode near the end of the task set, and time to completion analysis was performed on each subject's best run under each condition. In order to eliminate order-sensitive issues, a random determination was used to select whether a subject began with force feedback on or off. The time to completion of each trial was measured, as well as the subject's success in completing the task.

Block Stacking

The subjects were asked to construct a tower of four blocks upon a target location, as seen in the pictures below. This test examines whether the presence of the internal forces displayed by CyberGrasp affects performance of a primarily pick-and-place.

Qualitatively, most of the subjects preferred the force feedback mode. During the run, comments such as "That feels nice," and "The forces tell you where you are and what is going on," were common. Upon completion of all the tasks, all subjects said they felt more comfortable with the forces on in this task. Some subjects pointed out that the forces could still mislead you, however. One subject mentioned that touching the table felt exactly the same as touching the object, since the CyberGrasp can only pull in one direction.

Quantitatively, however, there was no significant time difference between a subjects best performance with force feedback and their best performance without force feedback. A likely reason for the overall lack of time difference between the two modes, besides human muscle memory and proprioception, is that a significant portion of the task involved moving the larger and slower industrial robot arm. Conditions for moving the arm were identical in the two force modes and likely contributed significantly to the overall time.

Object Rolling

The subjects were asked to use the robot fingers to roll a cylinder through a full 360° rotation, as seen in the figure to the right. The purpose of this task was to examine the effect of the CyberGrasp force feedback on performing two fingered dexterous manipulation.

The task began with the 400 mm long by 50 mm diameter cylinder resting in two V-shaped notched supports near the ends of the cylinder. The robot hand was located above the cylinder. The subject was instructed to lower the hand and rotate the object through a full clockwise rotation. If the subject knocked the cylinder off of the stand, the attempt was considered to be unsuccessful.

This task demonstrated the varying levels of mapping quality. One subject rotated the object with nearly flawless two fingered manipulation on his first attempt, with a time lower than many subjects' best performance. Other subjects struggled to coordinate their finger motions even after several trials.

It was fairly clear from watching the subjects that the addition of force feedback actually impeded their ability to roll the object quickly. The chart to the left shows the ratio of best run without force feedback over best run with force feedback for each subject. The average ratio across subjects was 0.72 (a ratio of 1.0 would indicate no difference in performance due to force feedback, while a ratio less than one indicates faster completion time without force feedback). Thus, on average, the subjects' best run without force feedback was 28% faster than their best run with force feedback. Using a bootstrap t-test to resample our data, we can state with 95% confidence that the mean population ratio would be 0.72±0.16, illustrating that force feedback has a negative effect on subject performance time with respect to this task.

On the other hand, the chart to the right displays that our subjects were three times more likely to have a trial failure without force feedback as compared to with force feedback (a 10% failure rate compared to a 3.3% failure rate). Analyzing the data with a bootstrap t-test to examine whether the population is more likely to have a failure without force feedback, we get an alpha significance level of 20%. This is not significant enough to state strongly, but it is a promising indication and should be examined further.

These results are confirmed by the qualitative comments made by the subjects. One subject pointed out that the system does not provide a sensation of curvature or rolling. Another subject said, "The forces don't match what I see, so I don't know how to adjust my hand." The difficulty of the task may be due in part to the uncommon motion. One subject pointed out that she really wanted to grasp the cylinder and turn it with her wrist.

These results indicate that the single axis of force representation provided by the CyberGrasp does not sufficiently represent the expected forces for rolling an object, and the potentially misleading information slows the user more than having no force information. Rolling of an object uses regulation of the ratio of normal to tangential forces, which cannot be accurately displayed by this system. However, the presence of contact force information does seem to improve a user's ability to maintain a stable grasp and aid the task learning process.

Conclusions

The results indicate that the CyberGrasp force feedback system does not increase speed of performance for simple telemanipulation tasks such as block stacking and object turning. This is due in part to human skills in learning and muscle memory, as well as the imperfect force transparency due to the single degree of actuation.

Conversely, there is some evidence that force feedback improves manipulation stability and may benefit the task learning process. Subject responses indicate that force feedback?s benefits may be more apparent in tasks which require delicacy or precision. One pointed out ?It seemed a little more difficult with the forces, because you are taking care where your fingers are.... Otherwise, you just pinch as hard as possible.? and another said, ?Without forces you aren?t worried about damaging the machine or forcing things.?