Swipe Gestures Using a TFT Display

• Download source - 1.6 KB

Introduction

Gestures allow for more complex and expressive inputs than simple touch allows for. With gestures, a user can indicate a variety of actions by "drawing" on the screen in a particular way. On little IoT devices whose touch display may be the only way of communicating with them directly, gestures become extremely useful.

Unfortunately, there's simply not a lot of information for implementing gestures for these little gadgets.

In this article, I aim to show you how to implement simple swipe gestures using a TFT touch display attached to an Arduino compatible device.

Prerequisites

• An Arduino compatible device. I am using an ESP32 with this demonstration, but other devices can be used with some wiring modifications
• A TFT display device. The code is written for an RA8875 but can be used with other displays like the ILI9341 with minimal code modifications
• The Arduino IDE and appropriate board manager
• The usual assortment of jumper wires and prototype boards

Conceptualizing this Mess

We have a number of problems to solve with respect to coding this. Reading touch events can be a little tricky with these devices - oh the joys of coding without all encompassing frameworks handling the gritty details! We also are going to need timing capabilities for various purposes. Furthermore, we need to actually implement the gesture computations themselves.

Timing the Code

Let's start with a simple pattern - the delay()-free timer. delay() can be problematic to use inside loop() because it blocks, so it can cause other activity to halt while waiting. Well, sometimes we need to run something at timed intervals while allowing the rest of loop() to continue. We can accomplish that with a single global variable for a timestamp and the millis() function:

#define INTERVAL 100 // <sup>1</sup>/<sub>10</sub> of a second
// set to 0 if you want it to fire 
// immediately the first time
uint32_t _timeoutTS = millis(); 
...
void loop() {
    if(millis()-_timeoutTS>INTERVAL) {
        _timeoutTS = millis();
        // do work
        ...
    }
    ...
}

What we're doing is making it such so the section under "do work" only runs 10 times a second instead of every time loop() is called. This is preferable to using delay(100); which would slow down the entire loop() routine, not just that one section of code. We'll be using this technique later, so keep it in mind since this is how we solve the problem of timing.

Reading Touch Events

Note that this might be different from device to device. Please look in your library's example code for how to read touch events. One problem that is more or less universal is the device cannot process touch events as quickly as loop() is called. The SPI bus is simply not as fast as our CPU. To deal with this, we only process our touch events at timed intervals using the technique outlined prior. What we're going to do however, is provide a single routine that hides the details. It is called tryGetTouchEvent(), and it will fill a tsPoint_t structure and return true if a touch event occurred. How your device does this is your business, but I'll be showing you the code for the RA8875.

Note that most of these screens need calibration. The points they return from touch events do not match the points on the screen and must be offset. Luckily for us, we don't need accurate physical coordinates. We only need to know the coordinates in relation to each other, not the physical device. We won't be covering calibration here.

Computing Gestures

For any gesture computation, we're going to need to know when the device was touched and when the touch was released. To do this, we poll for touch events at an interval the device can deal with - here ¹/₁₀ of a second. We keep track of whether the device was touched and whether it is currently touched. This is important. We need these edge conditions to determine when someone puts their finger on the screen and then continues until they release it. Basically, what we do is this:

...
_touched = tryGetTouchEvent(&pt);
if(_touched!=_touchedOld) { 
    if(_touched) {
        // touched event handling code
    } else {
        // released event handling code
    }
}
_touchedOld=_touched;

This should be fairly straightforward. We're just seeing if the touched state changed and if so we determine if it was a touch or a release.

For complicated gestures, we would need to keep an array of points that were "drawn" while touching the display and process them when the touch is released. However, since we're doing basic swipes, we can take a significant shortcut and drastically simplify the code.

All we need is to keep track of the point where they first touched the display, and the point where they released it. We then compare the differences in the x and y values. Whichever is largest decides whether we swiped vertically where the y difference is largest, or horizontally where the x difference is largest.

We also need to make sure they actually swiped far enough to register but this is simple. All we do is make sure the difference in the x or y coordinates is long enough. We have different thresholds vertically and horizontally since the screen isn't square.

We'll see all this in action when we get to the code.

Building this Mess

Wiring this Mess

Note that this is hardware specific. The idea is to wire your display to your device's primary SPI bus and then wire any additional pins like RST. On the ESP32 with the RA8875 that looks like this:

RA8875 VIN ➜ ESP32 VIN (+5vdc)
RA8875 GND ➜ ESP32 GND
RA8875 3Vo ➜ N/C
RA8875 LITE ➜ N/C
RA8875 SCK ➜ ESP32 GPIO18
RA8875 MISO ➜ ESP32 GPIO19
RA8875 MOSI ➜ ESP32 GPIO23
RA8875 CS ➜ ESP32 GPIO5
RA8875 RST ➜ ESP32 GPIO15
RA8875 WAIT ➜ N/C
RA8875 INT ➜ N/C
RA8875 Y+ ➜ N/C
RA8875 Y- ➜ N/C
RA8875 X- ➜ N/C
RA8875 X+ ➜ N/C

Coding this Mess

Finally we get to the code. Since we've already explored the techniques, we'll visit them all in action by going over the entire tft_swipe.ino file below:

// touch configuration
#define TOUCH_INTERVAL 100
#define TOUCH_THRESHOLD_X 200
#define TOUCH_THRESHOLD_Y 120

// time before text indicator disappears
#define TEXT_TIMEOUT 2000

// hardware configuration
#define RA8875_CS 5
#define RA8875_RST 15

#include <math.h>
#include <Adafruit_RA8875.h>
#include <Adafruit_GFX.h>

void drawCentered(char* sz);
bool tryGetTouchEvent(tsPoint_t * point);

Adafruit_RA8875 tft = Adafruit_RA8875(RA8875_CS, RA8875_RST);

// for handling touched/release events
bool _touchedOld = false;
bool _touched = _touchedOld;
uint32_t _touchTS = 0;

// first and last points touched
tsPoint_t _touchFirst;
tsPoint_t _touchLast;

// the timestamp for making text go away
uint32_t _textTimeoutTS = 0;

void setup() {
  Serial.begin(115200);

  // init the display
  if (!tft.begin(RA8875_800x480)) {
    Serial.println(F("RA8875 Not Found!"));
    while (1);
  }
  tft.displayOn(true);
  tft.GPIOX(true);      // Enable TFT - display enable tied to GPIOX
  tft.PWM1config(true, RA8875_PWM_CLK_DIV1024); // PWM output for backlight
  tft.PWM1out(255);
  tft.touchEnable(true);
  tft.fillScreen(RA8875_WHITE);
}

void loop() {
  
  if (millis() - _touchTS > TOUCH_INTERVAL) {
    _touchTS = millis();
    // process our touch events
    tsPoint_t pt;
    _touched = tryGetTouchEvent(&pt);
    if(_touched)
      _touchLast = pt;
    if (_touched != _touchedOld) {
      if (_touched) {
        _touchFirst = pt;
      } else {
        pt = _touchLast;
        // compute differences
        int32_t dx = pt.x-_touchFirst.x;
        int32_t dy = pt.y-_touchFirst.y;
        uint32_t adx=abs(dx);
        uint32_t ady=abs(dy);
        // swipe horizontal
        if(adx>ady && adx> TOUCH_THRESHOLD_X) { 
          if(0>dx) { // swipe right to left
            drawCentered("Right to left");
            _textTimeoutTS=millis();
          } else { // swipe left to right
            drawCentered("Left to right");
            _textTimeoutTS=millis();
          }
        // swipe vertical
        } else if(ady>adx && ady>TOUCH_THRESHOLD_Y) { 
          if(0>dy) { // swipe bottom to top
            drawCentered("Bottom to top");
            _textTimeoutTS=millis();
          } else { // swipe top to bottom
            drawCentered("Top to bottom");
            _textTimeoutTS=millis();
          }
        }
      }
    }
    _touchedOld = _touched;
  }
  // make the text we displayed disappear, if necessary
  if (_textTimeoutTS && millis() - _textTimeoutTS > TEXT_TIMEOUT) {
    _textTimeoutTS = 0;
    tft.fillScreen(RA8875_WHITE);
  }
}
// read the TFT touch
bool tryGetTouchEvent(tsPoint_t * point) {
  uint16_t x, y;
  tft.touchRead(&x, &y);
  delay(1);

  if (tft.touched()) {
    tft.touchRead(&x, &y);
    point->x = x;
    point->y = y;
    return true;
  }
  return false;
}
// draws text centered on the display
// BUG: getTextBounds() doesn't seem
// to work right for computing width
void drawCentered(const char *sz) {
  int16_t x, y;
  uint16_t w, h;
  tft.textMode();
  tft.setTextWrap(false);
  tft.setTextSize(1);
  tft.textEnlarge(1);
  tft.getTextBounds(sz, 0, 0, &x, &y, &w, &h); 
  tft.textTransparent(RA8875_BLACK);
  tft.textSetCursor((tft.width() - w) / 2, (tft.height() - h) / 2);
  tft.textWrite(sz);
}

Given what we covered in the earlier section, most of this should be pretty clear. We're running two "timers", one for the touch events and one for clearing the text. We're processing our touch events on release by computing differences as noted earlier. When we swipe, we write the swipe command to the display and then start the "timer" to clear the text.

Bugs

The text does not center properly. I'm either using getTextBounds() incorrectly or it doesn't actually work with my setup.

History

• 4^th December, 2020 - Initial submission