Support: NSF grant CMS-0600658 and UIC College of Dentistry
Recently, haptic simulators have shown great potential in teaching
sensorimotor skills. This is especially true for areas where the
traditional training technique is expensive, such as medical and
dental training. The goal of this work is to address several key areas
where improvement is needed to increase the realism of haptic
interaction for teaching of sensorimotor skills. These key areas are:
1. Haptic rendering algorithms. Modern penalty-based haptic rendering
algorithms do not produce realistic forces in certain situations. A
new method for haptic rendering needs to be implemented based on
physics rather than on heuristics. At the same time the new method has
to be fast enough to be executed in the allowed time frame.
2. Recording haptically augmented training video aids. This would
provide a way to convey the information necessary to master a new
skill from a teacher to a trainee. Several haptic playback schemes
have been proposed. Evaluation of the effectiveness of different
schemes for learning the task is required.
3. Collaborative haptic environments. In these interactive
environments the information to master a sensorimotor skill could be
conveyed from a teacher to several trainees simultaneously. For this
idea to work, methods for running haptically augmented online
demonstrations have to be developed. Other issues, such as switching
between user behaviors and arbitration mechanisms need to also be
addressed.
Work in the first two areas is described below.
Existing penalty-based 6-DOF haptic rendering algorithms have a built-in deficiency: they do not properly account for the orientation. Consider two different scenarios:
Whether a pure force or torque is applied to a cube resting on a
plane, existing penalty-based methods would yield roughly the same
contact force – the force perpendicular to the surface of the plane
and a function of the penetration depth defined in strictly
translational sense. In reality, pure instantaneous torque applied to
the object should correspond to the contact wrench that has only
torque components.
Instead of computing translation-based penetration depth, our proposed
haptic rendering algorithm takes the full rigid-body configuration
(translation and rotation) into account. We look at the set of all
possible configurations that correspond to the situation when there is
no collision, and find the closest-point projection of the initial
inadmissible configuration onto a set of admissible configurations.
For this approach to work a suitable distance metric function in
configuration space SE(3) has to be found. This function is
required to be invariant to the choice of reference frames, to allow
to obtain scale-invariant solutions and to be based on physics rather
than heuristics and be theoretically justified. We propose to use the
kinetic energy function as the distance metric function.
Once the optimal admissible configuration is found, generalized
penetration depth is computed and feedback wrench is determined using
a standard penalty-based approach.
If the objects are convex, then in the planar 2D case the optimal closest-point configuration can be found analytically as a solution of a constrained optimization problem. In the 3D case we can use bisecting planes to find cross-sections of the object, and then use the exact solution method to find the optimal configuration for each 2D cross-section. Then these solutions can be combined to get the solution in 3D.
Haptic applications can be used in two phases of teaching of a motor
task: in the initial expert-influenced phase and in the component
strengthening phase which is a sole responsibility of the trainee. It
is known that the haptic applications are very useful during the
second phase, but their use is limited in the first phase. The
challenge is to find a suitable way to convey the information
necessary to master a new skill from an expert to a trainee.
Haptically augmented video training aids are a solution. First, such
video is recorded by the teacher on a simulator. Then it can be
played back either with haptic augmentation or as a regular video.
Each sensorimotor skill has two components: position and force. To
acquire the skill is to be able to reproduce both components with a
certain degree of accuracy. Both components can be recorded by using
a force sensor and tracking devices. But force and position cannot be
both displayed simultaneously. Any form of haptic augmentation should
pursue two goals: convey both the position and force data to the
possible extent and achieve educational objectives, i.e. it should be
an aid in sensorimotor skill acquisition, not a distraction.
One possible method of haptic augmentation is haptic playback. Visual
display is used to produce visual cues (Corno, 2005). As long as the
user follows the target with a reasonable degree of accuracy, he will
feel the same force that was recorded and will reproduce the same
spatial trajectory. For this approach to work, the user is assumed to
be cooperative and active participant. It should be noted that the
trajectory of the visual target and the recorded trajectory may differ
considerably. An important question that we intend to investigate is
whether this has any significant impact on the achievement of
educational objectives.
A simpler method of haptically augmenting training videos is
penalty-based haptic pseudo-playback (haptic virtual fixtures). In
this case the priority is given to following the correct trajectory.
Position and orientation of the user controlled object is recorded
along with some auxilliary data. When played back, the recorded
trajectory is displayed and haptic device renders a guiding force to
drag the tool towards the desired recorded position. Here the user
may not be cooperative. A force sensor is not needed in this case.
For the purpose of testing various haptic augmentation schemes we use
the PerioSim which is a fully functional haptic application intended
to teach dental students to learn the correct use of dental tools in
various dental procedures. It has been developed jointly by the UIC
College of Dentistry and the UIC College of Engineering.
Several issues need to be addressed in future. For the energy based
haptic rendering algorithm the feasibility of obtaining a closed form
solution directly for general three-dimensional case without resorting
to plane bisection technique needs to be investigated. If found, such
a solution would allow for even faster and computationally less
complex haptic rendering. Also, the performance of the algorithm
needs to be evaluated from the perceptual point of view. It is
necessary to study how humans respond to various haptic rendering
schemes and to learn more about the users’ psycho-haptic experiences.
For producing haptically augmented training video aids it needs to be
investigated whether the haptic playback algorithm is effective in
achieving educational objectives. The main question here is how
closely the target tracking performance correlates with learning
performance. Human studies will be performed and quantitative metrics
will be used.
An extensive area that is also planned to be addressed is the
extension of results in haptic playback to collaborative haptic
environments. This is a potentially promising area. Having several
users interact with a haptic environment at the same time would be
useful in situations when an instructor demonstrates a sensorimotor
skill to students. Results in haptic playback and/or penalty-based
haptic pseudo-playback can be applied with the main difference being
the ability to convey haptic information at the same time
instance. However, other issues need also be addressed. They include
switching between user behaviors (active and passive), dealing with a
wide range of dynamic behaviors that users can display when they
interact, mechanisms for access control and arbitration between the
intensions of the users.
Please, have a look at the slides of a more
technical presentation of this project.