Using SSE in Updating Clients Changes in the Server

Introduction

It's been many days of Googling on how to use SSE in posting real-time updates to clients every time a change is made in the server like for example when a data is written to the database. Unfortunately, I didn't find any article that directly answers the problems I was facing when developing my little web application.

Background

When I wrote a database application in C#, updating clients was very easy using windows sockets. But PHP is a different world and I am very much new in this world—technically because this is the first time I created a web app and almost everything was a first. The main problem I came face to face was: how will I notify the web pages connected to my server when the server enters a new record or modify a record in the database? This could be happening directly in the server or another client sending data to the server.

What I came up with?

Let's get started with code.

The Trigger

<?php
// pushdata.php

class PushData
{
    private $folder;
    function __construct($t) {
        $this->folder = $t;
    }
    function trigger($msg) {
        file_put_contents($this->folder . '/' . date('YmdHis'), $msg);
    }
}

The Puller

<?php
// datasource.php

require __dir__ . '/../sse/pushdata.php';

function push($t, $m) {
    static $path = __dir__ . '/../push';
    $pref = "sse-$t";
    foreach(scandir($path) as $v) {
        $t = "$path/$v";
        if (is_dir($t) || stripos($v, $pref)!==0)
            continue;
        $push = unserialize(file_get_contents($t));
        $push->trigger($m);
    }
}

The Pusher

<?php
// sse.php

header("X-Accel-Buffering: no");
header("Content-Type: text/event-stream");
header("Cache-Control: no-cache");
//header('Connection: keep-alive');

require_once __dir__ . '/../../.ek/filter.php';
require_once __dir__ . '/../includes/utils.php';
require 'pushdata.php';

$table = ek\Filter::get('t') or die;
$lastId = ek\Filter::server('HTTP_LAST_EVENT_ID');
ignore_user_abort(true);

$mainpath = __dir__ . '/data';
if (!is_dir($mainpath)) {
    if (file_exists($mainpath))
        unlink($mainpath);
    mkdir($mainpath);
}

function send(...$msgs) {
    foreach($msgs as $m) {
        echo $m . PHP_EOL;
    }
    echo PHP_EOL;
    ob_flush();
    flush();
}

if (!$lastId) {

    $lastId = time();
    send("id: $lastId", "data: connection id");
}

$storage = $mainpath . '/' . $table . '.' . $lastId;
$trigger = __dir__ . "/../push/sse-$table.$lastId";

class Push
{
    private $connected = true;
    private $trigger;
    private $storage;
    function __construct($s, $t) {
        if (file_exists($s)) {
            if (!is_dir($s)) {
                unlink($s);
                mkdir($s);
            }
        }
        else
            mkdir($s);
        $this->trigger = $t;
        $this->storage = $s;
        $this->start();
    }
    private function start() {
        while(1) {
            send('event: ping');
            while (ob_get_level() > 0) {
                ob_end_flush();
            }
            flush();
            if (!$this->checkIfConnected())
                break;
            if(file_exists($this->storage))
            {
                $files = array_diff(scandir($this->storage), ['.', '..']);
                $count = count($files);
                if ($count != 0) {
                    foreach($files as $file) {
                        $t = $this->storage . '/' . $file;
                        $ar = file_get_contents($t);
                        unlink($t);
                        send('data: ' . $ar);
                        sleep(1);
                    }
                    continue;
                }
            }

            sleep(1);
        }
    }
    private function checkIfConnected() {
        if (!$this->connected)
            return false;
        if (connection_aborted()) {
            $this->connected = false;
            Utils::removeDir($this->storage, true);
            unlink($this->trigger);
            return false;
        }
        return true;
    }
}

if (!file_exists($trigger)) {
    file_put_contents($trigger, serialize(new PushData($storage)));
}
new Push($storage, $trigger);

This is the trinity that gets one portion of my web app working. What are their roles? Pardon the names, btw.

First, the Pusher (sse.php)

When a client initiates a communication through JavaScript’s EventSource with the pusher, the latter IDs the connection utilizing one feature of SSE which is the server’s HTTP_LAST_EVENT_ID. Additionally, in my case, the client must also send with the url a parameter (in my code I called t) pointing to a table that will be the subject of this communication.

In the articles I've read, the id was used as the event id. But here I used it as the ID of the communication between SSE server and client. And to make sure the ID is unique, I used PHP’s time() function.

Upon initiation of the communication, HTTP_LAST_EVENT_ID is checked which is null when establishing the communication the first time. After checking the ID, a call to PHP’s ignore_user_abort was made supplying true as argument. What is the significance of this? I surmised that when you set it false which is the default setting, your script will not get notified when the connection is lost which in effect will not give you a clue when to do some clean ups.

Then I created the main path stored in the $mainpath variable. That is the path where all data that need to be sent to the client will be stored. Then a little function called send follows which takes care of sending events to the client.

Upon checking that the last event id is null, the ID is generated (stored in the $lastId variable) and sent to the client with the data 'connection id' which is intentionally ignored in the client's script.

Following are two variables namely $storage and $trigger. Take not that both of them uses that $lastId. The is to make sure that even if more than one SSE communication have the same subject table, they will not touch what's not theirs. The $storage is the path where the data the SSE server must be sent to its client is stored. And the $trigger is the full path name of the data storing the serialized info the PushData class instance.

And there is a class I called Push. This class in charge of collecting all the data to be sent to the client and the cleaning up. It has three properties: $connected, $storage and $trigger. $storage and $trigger are actually copies of global variables $storage and $trigger, respectively. I am not sure if it is necessary for the Push class to have its own copy or just use those global variables. The $connected variable seems redundant but it's not.

When Push class is instantiated, it does some setups then calls its private function called start which has an infinite loop. I have read somewhere that a loop is not necessary in the SSE server but I will explain it later in observations why I stuck to it. Inside this loop are some codes I copied from stuff I stumbled upon when searching the internet. Honestly, I really don't understand why they are there in the first place. Then there is the consistent call to check whether or not the script is still connected. Remember what I said earlier, when calling ignore_user_abort with a false argument, calling connection_aborted will not have any effect because the script will never get there. Anyway, Inside this loop takes place the actual collecting and sending of data.

            if(file_exists($this->storage))
            {
                $files = array_diff(scandir($this->storage), ['.', '..']);
                $count = count($files);
                if ($count != 0) {
                    foreach($files as $file) {
                        $t = $this->storage . '/' . $file;
                        $ar = file_get_contents($t);
                        unlink($t);
                        send('data: ' . $ar);
                        sleep(1);
                    }
                    continue;
                }
            }

The Push class in its loop constantly checks the contents of the storage folder and send anything inside it. Meaning, while sending the data and the data is large and sending takes longer and somebody in your office make fun of throwing some files in that storage folder, the SSE client will break.

Below the class is the creation of the trigger file which is a serialized PushData class and the instantiation of the Push class.

Second, the Trigger (pushdata.php)

PushData works like a pipe. When a change is made in the server, this saved PushData object is resurrected by calling unserialize then its trigger function is called supplied with the data that is to be sent to client. The trigger function then simply saves this data to the storage folder supplied to it when it was constructed. This storage folder is constantly monitored by the Push class.

Lastly, the Puller

The puller is actually a function I accidentally named push and is a part of larger script (datasource.php) that takes charge of everything concerning data retrieval and storage. When changes is made to the database, this script, after making it happen, calls the push function supplying as arguments the name of the database table where modification was made, and the modified data. Then push function then unserialize the saved PushData info and calls its trigger function supplying the said modified data as argument. Take not that the file containing the serialized PushData object has a prefix sse.

The filter.php file are just functions that calls filter_input while inside the utils.php file is a helper function to delete the contents a folder.

That's all I've got. I'm not sure if I was just not able to find the answers that suits my case or... Anyway, I posted this after thoroughly testing it but it's not a guarantee that will for anybody else, too. Anyone can try, tough. Your PC will not get fried, anyway, and that I guarantee.

Points of Interest

What I observed when developing this, when a connection between client and SSE server is connected, connection seems to bereestablished continuously as manifested by the many trigger files being created. This is one of the reasons I've used the HTTP_LAST_EVENT_ID as the identity of the communication rather than the event id. This fixes the creation of many trigger files.

When the connection is reestablished, everything is like overwritten: the loop inside the start function stops. Actually the entire instance of the Push class is lost which in effect breaks that loop. So the last line of the sse.php file is always a fresh instance of the Push class every time a connection is made or reestablished. I think I am confident that I am not making a lot of unnecessary instances of this class although, aside from confirming (which, I may guess also partial) that the instance of the Push class is lost, I still don't have a way of checking if I am correct.

About the $connected property of the Push class, I just wondered at the end of this writing, is it really needed? But, for the mean time, I will just leave it there.

History

While this stuff may help some people, the main purpose of submitting this is to learn from comments of other people. I am a newbie in the PHP world. All I've got were general ideas, the right questions to ask and a very helpful internet connection. I came from C#/C++ and this is my first time going full-stack with a very minimal knowledge of front-end and back-end technologies. Lastly, English is not my native language so there may be confusing statements due to difficulty in constructing it well and I beg your pardon.