Interview with G.J. Vermeij

On Friday November 3, 1995, M. Costa and M. Cutkosky met with Professor G.J. Vermeij at the U.C. Davis Geophysics Dept.

Professor Vermeij can be reached at: 296 Physics/Geology, University of California at Davis, Davis, CA 95616.
e-mail: vermeij@geology.ucdavics.edu
phone: 916-752-2234

The following are notes from the meeting. The main issues discussed included:

Some questions prepared for the meeting:

  • How do you detect materials. What tools do you use? Any special techniques?
  • Do you find it useful to scrape, chip or otherwise modify parts of materials to assess them?
  • Are you aware of standard "exploratory procedures" that you use?
  • Imagine that you were doing field exploration through a remote interface. What would you expect to find most valuable. (What kinds of transmitted sensations, for example)?
  • If you work in cold environments and your fingers get numb, what are the most noticeable effects? How do you compensate?
  • How important is it to feel immediately in control of the motions of your fingers? If communications delays precluded direct control, how might we compensate? How useful could training be?

Issues discussed

Multiple scales of space and time

Haptic sensing is characterized by simultaneous exploration on different scales of space and time. In the discussion, the following main scale divisions were identified:
Space scales:
  1. Hands with fingers outstretched > 20 cm.
  2. Handling within the hand(s) 5-10 cm
  3. Fingertips 0.25-2 cm
  4. Nails/stylus < 0.25cm
Time Scales:
  1. < 0.5 sec for vital object recognition
  2. > 10 sec for some kinds of gross geometry integration
  3. many seconds or even minutes, with repeated explorations, for characterizing and classifying in the lab.
Because of the temporal aspect of integrating haptic information, the time scale is partially related to the spatial scale. Vermeij felt that a critical aspect of haptic field work is being able to quickly identify objects and conditions. This basically precludes large objects from his field work. If the object takes longer than ten seconds to scan Vermeij finds it difficult to integrate the information.

"Large" scale geometry and shape sensing (scale 1)

Vermeij pointed out that in the field what he mostly misses is the large-scale integrated picture that vision provides. He focuses on exploration on scales of less than one cubic foot. For example, he pointed out that he does not enjoy sculpture greater than about one foot cubed. In fact, he prefers sculpture at considerably smaller scales.

Integration of geometry over scales of several inches is difficult. It is done best with all ten fingers moving in an imprecise meandering, nonlinear fashion over a surface, lingering and back-tracking wherever necessary. Vermeij felt that it was more difficult to integrate information from a linear exploration than a nonlinear one.

General macroscopic object properties such as temperature and weight are also perceived with the whole hand as a "large" scale phenomenon.

Vermeij recalled a paper in Nature by Kennedy et al (see Ideas about stereotypical exploratory actions below) in which blindfolded subjects reportedly did better at recreating raised line drawings when their index was guided around the drawings in a linear fashion. He pointed out that this is in contrast to the approach that he and other blind individuals find most effective for rapidly integrating the geometry of a complex raised-line image such as a map.

Fine features (scale 4)

At the finest scale (from one or two millimeters to sub-millimeter scales) Vermeij prefers to use his fingernails and/or a stylus such as a hypodermic needle. The stylus is a good complement to the fingernails, with two dimensional versus one dimensional motion over the surface and less local integration of shape in the direction perpendicular to motion.

Hypodermic needles are especially useful because they are light and stiff (hollow cross section). If a stylus bends or flexes this makes it more difficult to use. Needles also have the advantage of being able to probe places inaccessible by finger nail. This includes enclosed regions whose interiors are not visible, such as the insides of sea shells.

A stylus can get stuck. Therefore, a robot with styli might need either disposable ones or a mechanism for freeing them. It is also best if the tip of the stylus is not too sharp. This reduces the incidence of snagging and catching on things.

Vermeij demonstrated how he drags a stylus over shell surfaces to detect fine striations and crenelations and to count dentate features. This practice has been so successful that a number of his sighted colleagues have adopted the practice. In this way they obtain morphological details without being influenced by color or shading.

Fingernail scrapes across a surface are also used in a destructive testing mode to asses hardness.

Gloves are not desired - even thin surgical gloves. Fingerless gloves are used even in the coldest water to preserve fine tactile sensibility while reducing numbness (see Effects of cold).

Object recognition

Sometimes one needs to identify objects in the environment very quickly. It is not surprising that people are good at this.

Vermeij notes that we use a combination of texture, thermal conductivity, and aural (except underwater) cues as well as shape and vibration information. Even underwater one can identify danger very quickly. (For example he recalled an instance when he needed to identify a stone fish by touch before it stung him.)

Integration of information

Vermeij emphasized that he uses both hands to explore different size scales simultaneously. The fingers, fingertips, nails and his hand as a whole each work most effectively at a different physical scale.

Redundancy seems to be important. Vermeij can perform best when integrating information at multiple scales (texture, fine features, gross shape) as well as aural cues. He thus obtains repeated and overlapping sources of this information. He noted that Braille is read better and faster with two hands than one.

Vermeij prefers to handle an object and feel it from many orientations. (a problem with museum specimens glued to cardboard stock). Even quite small shells (1-2 cm on a side) are held and felt by the fingers of both hands and traced with a combination of fingers and nails. Vermeij did point out that haptic exploration of an immovable object can still take place, it just takes longer to integrate the information.

Rubbing the surface provides particular information about fine textures. For example, living coral feels very different from dead. It's rough and has traces of mucus. The fine roughness is one of the first things to wear away when the coral dies. The best way to detect this is by rubbing the surface lightly.

In the field Vermeij also uses smell and aural cues (unless underwater); for example, the sounds and feeling of different shells or gravel crunching beneath the feet. Scallop shells sound like bone china, the carbonate rings. Any latency degrades the ability to integrate information. More then 10 seconds and it is very difficult. Aural cues might help. Consequently, for perceiving large-scale (scale 1) geometry in a planetary sensing application it would probably be best to scan and reproduce a virtual environment for human exploration on earth. (Perhaps ultrasonic scanning, compression, and recreation of the geometry?)

Detailed object characterization

While basic object recognition can happen very quickly, detailed investigation takes time, and repeated explorations. Vermeij will generally take samples back to the lab for detailed exploration.

Object characterization and taxonomic classification involves repeated, meticulous haptic explorations. Vermeij uses features such as the homology of growth spirals, grooves, ridges, and dentate features to classify shells.

The possible role of features in supervised robotic exploration was discussed. One reason for defining and refining features might be to help create a convincing virtual environment without directly transmitting every nuance of geometry back to the user. Features are a good shorthand. For example, a texture feature could be identified and its name transmitted in just a few bytes, and then recreated on earth using fractals or other techniques.

Effects of cold

When doing field work in cold water (like off the Aleutian islands) the fingers get numb. We might expect that the superficial type I mechanoreceptors would be the first to deteriorate. These receptors have small fields and are most important for perceiving fine features. As anticipated, Vermeij noted that the ability to perceive fine surface features deteriorated fastest in cold water. He commented that while he could always quickly distinguish between, say, seaweed and something harder, his ability to distinguish between two subspecies of mollusks might disappear.

Despite the problems caused by cold, Vermeij refuses to wear gloves. Their effect on fine-feature sensing is unacceptable, even with surgical gloves. Instead, he prefers fingerless gloves that insulate the fingers while keeping the tips exposed for fine texture sensing.

Ideas about stereotypical (remote) exploratory actions

Vermeij guessed that having a robot take a person on a guided exploration would be less effective than allowing the person to explore at will -- mainly because it would be harder to effectively integrate the information at different scales (several inches, one inch or less, one millimeter or less). As mentioned in the notes on Integration, aural cues could help.

He commented that people prefer to move somewhat randomly, lingering over spots, backing up, retracing, etc. Vermeij showed us two short papers he had written in which he disputed the findings of some psychophysics researchers who claimed that blindfolded subjects reportedly did better at recreating raised line drawings when their index finger was guided around the drawings in a linear fashion [G.J. Vermeij, "Observations on raised-line Drawings," in Education of the Visually Handicapped, May 1969; G. J. Vermeij "Exploring pictures by hand," a letter written to the editor of Nature, vol. 285, No. 5766, p. 594 in response to an article by Magee, L.A. and Kennedy, J.M., Nature, Vol. 283, p. 287, (1980)]. Again, this appears to be relate to the problem of integration of haptic cues at different scales. Vermeij's letter is followed by a reply from Magee and Kennedy.

Practice would help. It could also help if the robot learned the human operator's common exploratory behaviors through training and then emulated them. It would help even more if the robot would announce what it was doing. "I think this might be sand here. Let's dig into it a bit to see how it feels. Here's an interesting rock. Let's rub it."

Separate versus integrated perceptual scales (how important is an anthropomorphic approach?)

How detrimental would it be to separate the different size scales? For example, suppose one could provide local shape sensing akin to wearing gloves + a stylus for fine perception?

Vermeij had no strong opinion about this question, but felt that there would clearly be some degradation if the different size scales were separated. Training could help. Also, allowing people to revisit the objects repeatedly could help.

We speculated that a robot might also employ a variety of sensors not found on the human hand. These could be mapped to human aural and haptic sensations. People would learn to utilize the information. Some discussion ensued about insect and arthropod approaches to haptic sensing. Many arthropods rely heavily on haptic cues -- they are at a size scale for which haptic feedback (e.g., stylus) is very relevant. Crabs use pointy feet like a stylus. Insects often have sensory hairs on their legs, as well as antennae on their heads. Scorpions even locate prey using vibratory cues (phase relationships in Raleigh surface waves traveling over the sand). Catfish use barbules for locating prey in mucky water.

For an unmanned submersible working in mucky water, haptic and contact chemical and electrical sensors, as used by arthropods and deep sea fish, could be important. (Vermeij observed that deep sea fish are also virtually blind.)

Michael A. Costa
{sub_heading}{sub_heading}