購(gòu)買設(shè)計(jì)請(qǐng)充值后下載，，資源目錄下的文件所見即所得，都可以點(diǎn)開預(yù)覽，，資料完整，充值下載可得到資源目錄里的所有文件。。?！咀ⅰ浚篸wg后綴為CAD圖紙，doc,docx為WORD文檔，原稿無(wú)水印，可編輯。。。具體請(qǐng)見文件預(yù)覽，有不明白之處，可咨詢QQ:12401814
1496 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012
Using Machine Vision and Hand-Motion Control
to Improve Crane Operator Performance
Kelvin Chen Chih Peng, William Singhose, and Purnajyoti Bhaumik
Abstract—The payload oscillation inherent to all cranes makes
it challenging for human operators to manipulate payloads
quickly, accurately, and safely. Manipulation difficulty is also
increased by nonintuitive crane-control interfaces. This paper
describes a new interface that allows operators to drive a crane
by moving a hand-held device (wand or glove) freely in space.
A crane-mounted camera tracks the movement of the hand-held
device, the position of which is used to drive the crane. Two control
architectures were investigated. The first uses a simple feedback
controller, and the second uses feedback and an input shaper.
Two operator studies demonstrate that hand-motion crane control
is faster and safer than using a standard push-button pendent
control.
Index Terms—Control interface, cranes, input shaping,
machine vision, oscillation.
I. INTRODUCTION
RANES PLAY a key role in maintaining the economic
Fig. 1. Standard push-button pendent crane control.
In addition to facing the challenges of controlling large-
amplitude lightly-damped payload swing, operators must also
C
vitality of modern-day industry. Their importance can be
master nonintuitive control interfaces. Fig. 1 shows the pendent
seen at shipyards, construction sites, and warehouses and in
a wide variety of material-handling applications. The effec-
tiveness of crane manipulation is an important contributor to
industrial productivity, low production costs, and worker safety.
One inherent property of cranes that is detrimental to efficient
operation is the natural tendency for the payload to oscillate like
a pendulum, a double pendulum [1], or with even more complex
oscillatory dynamics [2]. Significant effort has been made to
develop control schemes to reduce the oscillatory response
from both issued commands and external disturbances [3]-[9].
There has also been research in controlling cranes that contain
rotational joints, which adds an extra level of complexity due
to their nonlinear dynamics [10]-[13]. Operators who manip-
ulate a crane using traditional interfaces such as push-button
pendents benefit from oscillation-suppression technology. They
generate safer (less collisions with obstacles) and more efficient
crane motions (faster task completion times and less operator
button pushes) than operators without such compensation [10],
[14]-[16].
Manuscript received September 26, 2010; revised April 7, 2011, June 10,
2011, and February 9, 2012; accepted April 6, 2012. Date of publication
June 8, 2012; date of current version October 12, 2012. This work was
supported in part by Siemens Industrial Automation, by the Manufacturing
Research Center, Georgia Institute of Technology, and by Boeing Research and
Technology. This paper was recommended by Associate Editor E. J. Bass.
The authors are with the Woodruff School of Mechanical Engineering,
Georgia Institute of Technology, Atlanta, GA 30332 USA (e-mail: kccpeng@
gatech.edu; Singhose@gatech.edu; pjbhaumik@live.com).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSMCA.2012.2199301
control of a typical overhead crane. The operator must be adept
in the cognitive process of transferring the desired manipulation
path into a sequence of button presses that will produce the
desired crane motion. For example, if the operator wants to
drive the crane through a cluttered workspace, then the desired
path must be mapped into a sequence of events where the
“forward (F),” “backward (B),” “l(fā)eft (L),” and “right (R)”
buttons are pushed at the correct time and in the correct se-
quence. Furthermore, as operators move through the workspace
to drive the crane and monitor its progress, they may rotate
their bodies and change the direction they are facing. In such
cases, the “forward” button causes motion to the left, right,
or even backward. As an additional challenge, the operator
can only directly drive the overhead trolley, not the payload.
Therefore, the operator must account for the time lag between
the commanded motion of the trolley, which can be many
meters overhead, and the delayed oscillatory response of the
payload.
While significant strides have been made to improve the
operational efficiency of cranes by controlling the dynamic
response to issued commands, relatively little consideration has
been given to the way in which operators issue those commands
[17]. It has been proven that interfaces that are tailored to the
cognitive processes associated with specific control systems
have beneficial effects [18]-[20]. For example, in the field
of laparoscopic surgery, medical robots such as the da Vinci
improve on the traditional procedure by allowing surgeons to
operate in a more ergonomic manner and with less cognitive
load [21], [22]. The controls move in the same direction as
the end effectors for da Vinci, unlike traditional laparoscopic
1083-4427/$31.00 ? 2012 IEEE
PENG et al.: USING MACHINE VISION AND MOTION CONTROL TO IMPROVE CRANE OPERATOR PERFORMANCE 1497
procedures where surgeons have to reverse map the controls
due to the instruments’ pivot point at the point of insertion.
This paper presents a novel control interface that allows an
operator to drive a crane by moving a hand-held device in
space. Machine vision is used to track the position of the device
(a wand or a glove), which is then used to generate the
command signal to drive the crane. The hand-motion control
interface is well tailored to the task of driving a crane through
a cluttered workspace because it eliminates the cognitive map-
ping process that is necessary with traditional control interfaces.
As a result, operators no longer need to account for the direction
in which they are facing. The manual dexterity required for
safe and efficient operation is also reduced. Additionally, the
control algorithm minimizes payload swing without signifi-
cantly slowing the system response. Therefore, the burden of
manually reducing payload oscillation is removed. This allows
the operator to concentrate solely on the path planning and final
positioning of the payload.
Hand-motion control offers other cognitive advantages over
traditional interfaces. There are two primary divisions of cogni-
tive control: analytic problem solving and perceptual process-
ing [23]. Perceptual processing tends to be faster and can be
performed in parallel, while analytic processing takes longer
and typically progresses serially. Analytic problem solving also
tends to be more prone to error [23], [24]. The results of many
studies also suggest that people prefer, and adopt, perceptual
processing when possible [16], [23], [25], [26]. From this
perspective, hand-motion control helps operators by lowering
the cognition level required to drive the crane. Operators no
longer need to think analytically about the sequence of buttons
to push or to account for the swinging payload; they only need
to move the hand-held device to the desired position or along
a desired path. This allows the operators to perform simpler
perceptual processing.
The major contribution of this paper is the novel hand-motion control interface. The benefits of this interface are validated by human operator studies. Section II describes the novel inter-
faces (the wand and glove). The control algorithms that are used in conjunction with the interfaces are discussed in Section III. This is followed by the operator studies in Section IV and conclusions in Section V.
II. INTERFACES FOR HAND-MOTION CRANE CONTROL
The application investigated in this paper is for a single-
pendulum point-mass payload that is suspended from a motor-
ized overhead crane. The 10-ton industrial bridge crane shown
in Fig. 2 was used for experimental verifications. A bridge crane
consists of a fixed overhead runway, a bridge that travels along
the runway, and a trolley that runs along the bridge. Laser
range sensors measure the trolley position along the runway
and the bridge. The hook, which represents the payload, is
suspended from the trolley by cables. A Siemens programmable
logic controller is used to control the motor drives and acts
as the central control unit. Commands to the crane can be
issued with a push-button control pendent, the wand or glove
for hand-motion control, or other devices [17]. A downward-
pointing Siemens Simatic VS723-2 camera mounted on the
Fig. 2. Typical bridge crane.
Fig. 3. Driving a crane by moving a reflective wand.
trolley measures the position of the hook. Reflectors mounted
on the topside of the hook aid vision-detection algorithms [27].
There are two hand-held devices for hand-motion crane
control: 1) The wand, shown in Fig. 3, is a reflective ball
mounted to the end of a hand-held pole, and 2) the glove,
shown in Fig. 4, has a circular reflector attached to the backside.
Fig. 5 shows a schematic diagram of hand-motion control
using machine vision. The crane-mounted camera is used to concurrently track the positions of the wand/glove and the hook. Because all reflectors appear as bright blobs in the camera image, a K -means clustering algorithm is used to distinguish the wand/glove reflectors from the hook reflectors [28]. The camera refresh rate is approximately 140 ms. The position of the wand/glove relative to the crane is used to generate an error signal to drive the overhead trolley.
1498 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012
Fig. 6. Standard pendent controller.
Fig. 7. Standard push-button pendent control response.
Fig. 4. Driving a crane by moving a reflective glove.
Fig. 8. PD hand-motion controller.
A. Standard Push-Button Pendent Control
The block diagram for standard pendent control is shown
in Fig. 6. The operator analyzes the workspace, considers the
required manipulation goal, and then decides on a course of
action. This plan is then implemented by pushing buttons on
the control pendent. These buttons send energy to the motors
and move the overhead crane trolley. The suspended payload is
moved indirectly by the motion of the trolley.
Fig. 5. Schematic of hand-motion crane control.
III. HAND-MOTION CRANE CONTROLLERS
Three control architectures were investigated. First, the stan-
dard push-button pendent controller was used as the baseline for performance comparisons. Then, a proportional-derivative (PD) feedback controller was investigated for its suitability in hand-motion crane control. Finally, an input shaper was added to the PD controller in order to reduce payload swing.
Note that, from the perspective of the control architecture,
the wand and the glove are identical. Both devices are used to
communicate the operator’s desired position to the controller.
For this reason, there is no distinction between the wand and
glove in the simulation and experimental verification results
that are presented in this section. However, in terms of er-
gonomics during operation, the wand has a greater reach and
can drive the crane toward tight spaces, such as corners. On the
other hand, the glove sacrifices range of reach for a smaller size
and ease of use.
Computer-simulated responses for point-to-point movements
of approximately 2 and 3 m using the pendent controller are shown in Fig. 7. Pressing a pendent button for a certain amount of time issues a trapezoidal velocity command to the crane motors. Due to the pendulum-like nature of the payload, this type of trolley movement will, in general, induce significant payload oscillations.
B. PD Hand-Motion Control
The well-known and popular PD controller represents one of
the simplest forms of feedback control. It is the most commonly
used feedback method in industry and has been applied to the
control of cranes [29], [30]. It provides a realistic choice for
hand-motion crane controllers. The PD hand-motion control
block diagram is shown in Fig. 8. The position of the wand
or glove is compared to the position of the overhead crane
(neglecting the vertical height difference) to generate the error
signal e. The command generator converts the error signal
(a positional measurement) into a velocity command that can
be sent to the motor drives. If e is within the designed range
specified by e0 and e100 , then the command generator linearly
PENG et al.: USING MACHINE VISION AND MOTION CONTROL TO IMPROVE CRANE OPERATOR PERFORMANCE
Fig. 9. Simulated PD controller with low gains. Fig. 10. Simulated PD controller with high gains.
scales e. Otherwise, the command generator outputs either 0%
or 100%. The values for e0 and e100 were 0.25 and 1.0 m.
These were selected based on comfortable distances at which
the crane followed the operator. The command generator is
described as
{ 0% : e ≤ e0
Command = 100% × e e??0 : e0 < e < e100 (1)
1499
100%
100 e0
: e ≥ e100 .
Fig. 11.
Starting and stopping with hand-motion control.
A PD control law is then applied, and the result is passed
through a saturator to ensure that crane velocity and accel-
eration limits are not exceeded. Note that the position of the
crane trolley, rather than of the payload, is used for feedback.
This is because, in practice, sensing the position of the trolley
(using laser range sensors) is much more reliable than sensing
of the payload (using machine vision). Furthermore, the single-
pendulum payload is an inherently stable plant: The payload
will always come to rest directly beneath a stationary crane.
Therefore, correct final positioning of the crane trolley ensures
correct final positioning of the payload.
1) Simulation Verification: A crucial design challenge is the
selection of PD gains. Computer simulations were constructed
to aid the gain-selection process. Hand-motion trajectories were
specified as ramps in position with gradients equivalent to the
maximum velocity of the 10-ton industrial crane (0.3577 m/s). This is approximately the speed of a slow walk and mimics the typical hand-motion trajectories from a human operator.
Figs. 9 and 10 show the simulation results for PD hand-
motion controllers with low and high feedback gains, respec-
tively. These two figures show the inherent tradeoff in using the
PD controller: With low gains, the crane was slow to respond,
but the payload oscillation was small; with high gains, the crane
moved quickly but at the expense of large payload oscillations.
2) Experimental Verification: The hand-motion control sys-
tem was implemented on the 10-ton bridge crane. The
wand/glove trajectories produced by human operators were similar to those used in the simulations. The ramp gradient was approximately equivalent to the maximum velocity of the crane, and the move distance was approximately 2 m for the tests reported here.
Fig. 11 shows an operator using hand-motion control to start
and stop the crane. To start moving, the operator can expose
Fig. 12. Experimental PD controller with low gains.
the wand/glove to the camera at some distance away from the
crane. When the crane approaches the desired position, the
operator lowers the wand/glove, which becomes undetectable
by the camera. When the camera is unable to locate the position
of the wand/glove, e is set to zero. Because the position of the
wand/glove may be unknown at certain times, there are breaks
in the curves that are labeled “Wand/Glove” in the experimental
response plots.
Figs. 12 and 13 show the experimental results for PD hand-
motion controllers with low and high feedback gains using
the glove interface, respectively. The experimental data re-
inforce the results that were established by the simulations:
Low gains produced slow crane movements and small payload
oscillations, while high gains yielded fast crane movements
but large payload oscillations. For safety reasons, minimizing
payload oscillation is normally a higher priority than fast crane
n
n
1500 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 42, NO. 6, NOVEMBER 2012
Fig. 13. Experimental PD controller with high gains.
Fig. 14. PD with input shaper hand-motion controller.
movements. Therefore, practical implementations of PD hand-
motion controllers should only use low gains.
C. PD With Input Shaper Hand-Motion Control
Section III-B demonstrated the inherent weakness in using PD hand-motion controllers (the tradeoff between low and high gains). However, performance can be improved with the addi-
tion of an input shaper that modifies the shape of the command signal to reduce oscillation. Fig. 14 shows the new control block diagram that shows an input shaper inserted between the saturator and the crane blocks.
1) Input Shaping: Input shaping is a technique that reduces
the residual vibration of flexible systems by properly shaping
the commands. This is accomplished by convolving the base-
line input command with a series of impulses, known as an
input shaper. The result is a shaped command that will reduce
Fig. 15. Simulated PD with input shaper controller.
Fig. 16. Experimental PD with input shaper controller.
ω is the natural frequency of the system, ζ is the damping ratio, and Ai and ti are the ith impulse amplitude and time, respectively.
Equation (2) gives the ratio of vibration with input shaping to that without input shaping. A constraint on residual vibration amplitude can be formed by setting (2) less than or equal to a tolerable level of residual vibration at the modeled natural fre-
quency and damping ratio [32]. For the simplest zero vibration (ZV) shaper, the tolerable amount of vibration is set to zero. This results in a shaper of the form [31], [33]
[ ] [ ]
residual vibration.
In order to determine the impulse amplitudes and time loca-
tions of an input shaper, certain design constraints must be sat-
isfied. The primary design constraint is a limit on the amplitude
of vibration caused by the shaper. The normalized percentage
ZV =
where
Ai
ti
1 K
1+K 1+K
= 0 √ π (5)
ω 1?ζ 2
√ ?ζπ
residual vibration (PRV) amplitude of an underdamped second-
order system from a sequence of n impulses is given by [31]
√
P RV = V (ω, ζ ) = e?ζωtn [C (ω, ζ )]2 + [S(ω, ζ )]2 (2)
where
∑ √
C (ω, ζ ) = Aieζωti cos(ωti 1 ? ζ 2 ) (3)
i=1
∑ √
S(ω, ζ ) = Aieζωti sin(ωti 1 ? ζ 2 ) (4)
i=1
K = e 1?ζ2 . (6)
2) Simulation and Experimental Verifications: The goal of combining high-gain PD feedback with a ZV input shaper is to obtain fast crane response and low-amplitude payload oscillations. The hand-motion controller in this section uses high PD gains (identical to the ones from Section III-B) to move the crane trolley quickly, combined with a ZV input shaper to cancel payloa