FreeTrack

From Wikipedia, the free encyclopedia




Latest release	v2.1 / October 5, 2007
OS	Microsoft Windows
Genre	Optical motion tracking
License	GNU General Public License
Website	www.free-track.org

FreeTrack is a free, general purpose optical motion tracking application for Microsoft Windows, released under the GNU General Public License. Its main function is inexpensive head tracking in computer games and simulations but can also be used for virtual reality, 3d modeling, computer aided design and general hands-free computing to improve computer accessibility. Tracking is sensitive enough that only small head movements are required so that the user's eyes never leave the screen.

Head motions are tracked with six degrees of freedom (6DOF), specifically; yaw, pitch, roll, left/right, up/down and forward/back. This is done by means of a video capture device, typically a webcam, which is placed in front of the user and tracks a rigid point model headpiece. This point model usually consists of infrared LEDs but can also be normal LEDs and even retroreflective material illuminated by a source of infrared light. The only requirement is that the points can be isolated from background light by means of physical and/or software filters.

FreeTrack can output head tracking data to programs directly using TrackIR, SimConnect and FSUIPC interfaces, programs that support these interfaces are regarded as having FreeTrack support. FreeTrack can also emulate mouse, keyboard, and joystick (via PPJoy) if a program does not support a direct interface.

FreeTrack is coded in Delphi 7 and uses DirectShow and DirectX. Head tracking is achieved using implementations of DeMenthon's four point iterative pose estimation algorithm (POSIT) ^[1] and Alter's three point geometric algorithm. ^[2]

1 Software
2 Webcam
3 Point model
4 Comparisons with TrackIR
5 Controversy
6 References
7 See also
8 External links

[edit] Software

FreeTrack runs in the background, processing image frames from a webcam into a list of point coordinates, these coordinates are used to create a head pose estimate before profile-specific adjustments are applied and lastly, the modified pose estimate is sent to user defined outputs.

Each degree of freedom (axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. Typically this is used to create a central deadzone, a point of stability where the user’s head can relax (while increasing sensitivity in the non-deadzone region) and can be especially useful for games that require steady aim, like combat flight simulators.

A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first person. This is useful for assessing tracking quality, notably the smoothness (stability) and responsiveness of motion.

Linear interpolation is used to convert the relatively low webcam frame rate into roughly 120 frames per second to improve panning smoothness.

Control input uses DirectInput, allowing keyboard (mouse and joystick support to come) buttons to be used in-game to toggle tracking parameters and individually toggle axes and outputs.

[edit] Webcam

A webcam is the key component of a FreeTrack setup but doesn’t need to be anything special, most common, low cost webcams are sufficient. It needs to operate at its maximum frame rate (typically 30frame/s) which requires disabling automatic exposure and manually reducing exposure. If the webcam settings reset themselves, FreeTrack can be configured to force them on start.

Some webcams can achieve frame rates higher than 30frame/s through video quality reduction, providing a marginal improvement in tracking responsiveness. Webcams that do this without using quality reduction can consume excessive CPU resources for minimal improvement in responsiveness.

FreeTrack requires the tracking points to be isolated from all other light (for reasons of efficiency), this is done using a low webcam exposure, a low filter threshold in FreeTrack and placing an infrared-pass filter over the camera lens. Photographic film or the magnetic storage medium inside floppy disks can be used as inexpensive infrared-pass filters. Webcams have a built-in infrared blocking filter, removing this is not required for LED tracking but can be done to improve the robustness of tracking and also allow retroreflective tracking.

Webcam CPU usage can be considerable, depending on the specific webcam hardware and driver in use. This can potentially result in a noticeable reduction in game frame rates, depending on the game’s CPU dependency, however modern multi-core CPUs are making this less of an issue. Webcam resolution can be reduced to minimize CPU usage.

Most good webcams work with FreeTrack, however an ideal webcam would have (from most important to least important):

Auto-exposure can be disabled - mandatory for maximum frame rate and exposure control.
30 frames per second capture rate – for good responsiveness and panning smoothness.
An easily removed IR filter - filter removal recommeded for best results.
Low CPU usage – (measured in System process)
An ‘Outside mode’ or equivalently, the ability to disable anti-flicker – providing a more consistent frame rate for better panning smoothness.
I420 compression – more efficient and reliable than RGB compression.
Wide field of view (46 degrees or more) – allowing more head movement (alternatively the webcam can be positioned further away from the user).
Low image noise – less parasitic tracking jitter.
352x288 resolution - a little better than the more common 320x240 while not as demanding as the next higher resolution of 640x480.
Quality adjustment - provides more than 30frame/s at reduced quality.

[edit] Point model

FreeTrack is able to track head movements by comparing webcam observations of a rigid point model headpiece with its known shape and size. The model can have a variety of shapes and mountings depending on the chosen tracking method and personal preference. The model can be made in a DIY fashion at minimal expense using readily available components from electronics retailers. Component packs and fully constructed models are also available for purchase from some members of the FreeTrack community.

An active point model uses LEDs (preferably 5 mm size) to represent the tracking points, powered by battery, transformer (plug pack) or USB. The electric circuit is very basic and can be made by someone with little or no experience with electronics. However, using USB as a power source should be done with extreme caution as it can potentially damage a connected motherboard.

Common infrared LEDs, like those found in remote controls, have a narrow, highly focused beam which is not suitable for head tracking. Suitable wide angle LEDs, like the Siemens/Osram SFH485P, with a half-angle of 40 degrees, can be purchased from more specialised electronics retailers. A normal infrared LED can be turned into a wide angle LED by filing down the lens tip.

Alternatively, a reflective point model can be built using retroreflective material to represent the tracking points and illuminated by an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases the webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen by the webcam. This is normally an easy and reversible procedure for which there are online guides that can assist.

Model configurations:

Single point: a single tracking point attached on a baseball cap or wire that extends forward of the head (for example). Unlike other tracking methods, this is limited to only two degrees of freedom (yaw and pitch) and can be more sensitive to translation than rotation.
Three point clip: asymmetrical triangular arrangement of points, clipped to a headphone and positioned to the side of the head, opposite the ear and parallel with it.
Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap.
Four point cap (for backward compatibility): symmetrical non-coplanar arrangement of points typically attached to the rim of a baseball cap.

[edit] Comparisons with TrackIR

TrackIR camera specifications: ^[3]

TrackIR2: 60k pixels
TrackIR3: 101k pixels, 358X290, 120frame/s, 33 degrees fov
TrackIR4: 101k pixels, 358X290 (accurate to 716 x 290), 120frame/s, 46 degrees fov, operating temperature 40-50°C.

Most webcams have resolutions greater than or equal to 320x240 and around 42 degrees field of view. Like TrackIR4, FreeTrack uses pixel doubling, effectively doubling tracking accuracy.

The main difference between TrackIR4 and FreeTrack’s webcam-based tracking is that TrackIR4’s image sensor reportedly operates at 120frame/s and image capture and processing is done in a dedicated on-board FPGA chip. This frees the computer from having to do these tasks, which would have been essential in previous years to achieve acceptable in-game frame rates. Modern CPUs and GPUs are making this less of an issue so that FreeTrack can use CPU dependent webcam drivers for image capture with little to no impact on frame rate (for example, FreeTrack plus Saitek W300 webcam use only 1-3% of an Opteron 175 CPU).

TrackIR4’s 120frame/s capture rate makes it capable of excellent panning smoothness and responsiveness. FreeTrack tackles the issue of panning smoothness by using interpolation to artificially increase the webcam’s frame rate to 120frame/s. However, responsiveness is fundamentally limited to the capture rate; a 30frame/s webcam has a maximum response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120frame/s camera.

Arguments supporting suitability of 30frame/s for head tracking:

A ping of 33ms is regarded as being competitive in an online multiplayer first person shooter.
The virtual head typically maps small real head movements to 360 degrees and so, from a realism perspective, it should not be able to move as quickly as the user’s real head. Some games enforce this by limiting maximum virtual head movement speed and applying head movement latency.
A human’s reaction time to visual stimulus (finger reflex) is typically around 200ms.
High responsiveness can create an unpleasantly unstable view, unless the user tries to hold their head very still, which can become uncomfortable. Applying software smoothing to increase stability effectively reduces responsiveness which is similar to using a lower camera capture rate.

[edit] Controversy

NaturalPoint, Inc. alleges that a FreeTrack library violates its copyright by using the TrackIR Enhanced interface without authorization to gain view control in games which support TrackIR hardware. These TrackIR Enhanced games constitute the majority of games which FreeTrack works with, and without them FreeTrack would be significantly less useful to game and simulation users. Other software also appears to be using the TrackIR Enhanced interface without authorization, including GlovePIE, NewView and TIR4FUN .

Text strings bearing explicit notice of "EyeControl Technologies" copyright (former name of NaturalPoint, Inc.) can be found within previous versions of the library in question. However, the strings have been encrypted in recent versions of the library to hide them. The continued presence of the strings can be verified by connecting to the interface and reading them.

The creators of FreeTrack have not denied copyright infringement at this time.

During early releases of FreeTrack the source for their TrackIR Enhanced DLL was publicly viewable. The copyrighted strings were clearly visible in the file NPClient_h.pas. Since that time, the developers have removed public access to the TrackIR Enhanced DLL in their source control system, while the rest of their source remains open to the public.