• Always present in useful frames
• Usually the largest and only near-vertical bone (except head, also near-vertical)
• All limbs, including head, join to it via an "intersection" POI

• Always with one intersection and one endpoint
• Near-vertical, almost a continuation of the torso

• Rejoin torso/head and become invisible at extreme angles; near-vertical only detectable if head-segment and arm(s) disappear and the torso's top endpoint jumps toward the top of the frame
• May consist of more than one segment if elbow is detectable
• Intersects torso near "top"
• Focus of hold-up detection

• Frequently consist of 2 or 3 segments (leg & 2 for feet)
• May also have large slope; range still not as high as torso/head
• Often legs join together
• Often vestigial--much like the feet, not very distinguishable in blob
• Bottom of torso

A

B

C

1. head, endpoint & intersection at top, near-vertical; arm, endpoint & intersection with torso, non-vertical; torso, 2 intersections, long, high slope; legs, intersection & endpoint, high slope
2. torso and vestigial legs only; torso includes endpoint
3. torso only (head can be created as part of segment)

• Ends of blob: 2 body segments (such as legs) mistakenly joined at one end
• Otherwise: hole in blob that shouldn't be there

Solved by eliminating such holes before skeleton calculation.

• Eliminate it because it is an error caused by the irregular blob boundaries
• Absorb it into the bone to which it really belongs (can still eliminate it because it offers no additional information)

Method Overview

First determine the POIs and segments of the skeleton.

Find the head based on the silhouette.

Highest unchecked pixel Connected components downward Centermost above threshhold is head Dimensions may offer shoulder guess

Find the end of the head using three different calculations:

Width-Based
Gradient-Based
Circle-Fit

Sample Image and Video:

Head-based identification.

• Head: reliable starting point (relatively stable, easily found)
• Torso: top attached to head
• Legs: tops attached to torso
• Arms: highly mobile, therefore highly variable

Arms must be addressed further.

• Two levels of certainty to keep arms during appearance and rotation.

• Broken arms adjusted to simplify skeleton and improve accuracy.

Also, the height:width ratio is calculated, and if it is

greater than 3.9, legs are possible.
greater than 3.0, legs may be possible, based on history.
less than 3.0, legs are not possible unless history strongly contradicts.

This helps avoid marking arms in a partial silhouette as legs and helps narrow the search for arms.

Arm positions are analyzed based on slope. The first and lowest alarm is the horizontal arm position. This is the lowest level because when an arm first becomes distinguishable from the torso, it almost always falls into this category. It is identified by an orange bar across the top (for left arm) or bottom (for right arm) of the image.

The second alarm level is the vertical arm position. An arm will move from the first alarm level to no alarm before moving into this position, although this is not yet checked by the program. A red bar down the left or right side of the image indicates a raised arm.

Globally, the highest alarm is reached when there are multiple silhouettes and both level and raised arm alarms.

August 2002

Sequence	# Frames	Successful ID
walaim1	68	94.12%
walaim2	115	93.04%
walup1	104	87.5%
walup2	97	93.81%
mikeaim	512	92.77%
mikemulti	548	91.24%
TOTAL	1444	91.96%

These sequences are of a single person fully visible outdoors. The "wal" sequences were of a quick mover late on an overcast afternoon in front of a wall and moving branches. The "mike" sequences were of a slower mover who rotated after walking to the center of the picture; the setting was a bright, early afternoon on grass in front of a pale wall.

Sample Videos