Measuring Finger / Mouse Velocity at Release Time

Any interface involving touch will inevitably need to know how fast the user is moving their finger. They touch the screen, drag their finger, release and expect motion to continue at that velocity. It sounds simple but has been causing me a few difficulties. I think I’ve finally solved it with a low-pass filter and pointer acceleration.

This article is based on my experience writing gesture code control for the Fuse product. It’s interesting how these seemingly simple problems turn out to be relatively complex tasks.

The Wrong Approach

There are two approaches that seem obvious but are both wrong. They technically measure a “velocity” but it certainly won’t be the one we want.

The first approach is to use only the last sample. Simply take the difference in time and position between the last two pointer events and calculate a velocity from that. The problem with this approach is that pointer events can be jittery at times, both due to how the hardware works and how the user actually moves their finger. We could end up with zero speed, infinite speed, or everything in between.

The second approach is to measure total offset and total time. This is just the difference in position and time of the pointer down event and the pointer release event. This removes the jitters of the previous one. It actually works to some degree if our user cooperates and makes unidirectional fluid motions. It does not work though if they ponder their actions or move their finger in multiple directions. Consider the worst case they move up 100 pixels, then down 100 pixels, resulting in an offset of 0!

A Moving Average/Low-pass Filter

To remove jitter but retain a current value sounds like the role of a moving average: the mean velocity of several samples is calculated to determine the current velocity. Nobody really likes keeping multiple samples around though, so instead we’ll use an exponential moving average, also known as a low-pass filter. This has the advantage of preferring recent values and also having a simple mathematical relation:

This says the new average velocity is equal a combination of the new sample and the previous averaged velocity. We just need to pick a value for α that makes sense.

Unfortunately, we have a problem here. The above formula, with a constant α only works if the sampling interval is constant. That is, each sample has the same time difference to the previous one. This won’t be the situation we have with our user; events come at rather jittery intervals.

There is an approach here, which I tried, where we can tie the sampling to an application’s frame rate (assuming it has a frame rate). The sampling period is then constant. This kind of works, but fails for two reasons: the frame rate actually fluctuates; and within one frame we may have several, or even zero, messages.

The formula must be converted into a continuous form, rather than a quantized one. Actually, the formula can stay the same, but we need a way to calculate a continuous alpha. I needed this on a previous finance project as well (I think for live charting) and found a helpful answer.

Given the time period over which we want the average, τ, and the actual elapsed time Δt from the previous sample, we calculate α to be:

In practice you don’t really have to understand what τ truly denotes, since you’ll just be adjusted it up or down based on user testing. As of this writing, I’m using a value of 0.1.

Pointer Acceleration

The above gives us a decent velocity measurement at the time the user releases their finger. Playing around though it doesn’t seem entirely satisfactory. I swing my finger quickly and it has quite a range of speeds. This is natural since my finger doesn’t always move the same speed. This issue is made worse by the user not moving their finger in a constant direction, so while the speed may be similar, the velocity up or down is certainly not.

To help here, I took a cue from desktop mouse movement: pointer acceleration. Once the mouse reaches a particular speed, that speed is multiplied by an acceleration factor. This makes faster pointer movement even faster than it really is.

It also leaves slow movement matching the users actual speed. This ensures that fine movements behave as the user expects. The acceleration is applied only at high speeds where the user is expecting fast motion.

This is some rough pseudo-code that applies acceleration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apply_accel = (speed) -> {
  var accel_threshold = 1000
  var accel_limit = 2000
  var accel_factor = 1.5
  var speed_limit = 2000
			
  speed = min( speed, speed_limit )
			
  var accel_range = clamp( speed, accel_threshold, accel_limit ) / (accel_limit - accel_threshold)
  var accel = accel_range * accel_factor
  speed = speed * accel

		    return speed
}

The acceleration curve starts at speeds above accel_threshold with a linear increase to accel_limit. Note I also have an absolute speed_limit on the input here. This prevents the occasional super fast swipe from going superman speeds.

Again here, I’m uncertain of what the best values are. I’m sure it’ll take quite a bit of tweaking. I saw a study about mouse acceleration that showed the acceleration curve also plays a role. For now, I’m just doing linear since it seems okay.

Perhaps something to note about this function is that it is working with speed as input rather than velocity. Our moving average of course is working with velocity so we have to convert back and forth: a very common conversion in physics and graphics code.

1
2
3
4
5
6

var unit_velocity = normalize( velocity )
var speed = length( velocity )

speed = apply_accel( speed )

velocity = unit_velocity * speed

Special Conditions

The approach I’ve outlined seems to work well, though I still have to tweak several of the constants. This is of course only part of writing gesture based controls. I’ve gone through a variety of elastic end snapping models, as well as several pseudo-physics handlers.

I need to handle very short input durations. If the user taps the screen very quickly, it may lead to a relatively high velocity. This is due to a small pixel movement over a very short time period. A threshold is needed to prevent this. Related to this is the initial value to the moving average, . At first I used 0, but this had a kind of slow start for quick actions. I switched it to using the first velocity sample and got better results.

I now have to go adjust this all to work with angular velocities on my virtual trackball. But more on that later.