The article discusses implementing non-blocking GUI in a Python application using a timer event callback and the 'select' API to handle blocking external I/O, with a practical example involving Blender and socket communication between a client and server for executing Python scripts.
1. Introduction
GUI application works in callback or asynchronous call flow which is event driven in architecture. An event loop is maintained and all registered/scheduled callback functions get executed one by one. An event, needs to get processed, must register with event loop manager. A callback function is associated with an event. Events in event loop get executed through execution of associated callback function.
GUI callbacks are called against GUI events. Making a blocking I/O from an event callback would delay the processing of GUI events leading to freezing GUI. This article discusses about serving an blocking external I/O in a GUI framework in a manner where non freezing GUI is possible.
2. Problem
How to making blocking external I/O in GUI related application?
3. Scenario
GUI application works in event callback program flow rather than sequential program flow, i.e., ‘C programming’. An event, with associated callback function, needs to get registered with event loop manager. Once event triggers, event loop dispatcher/manager calls the associated callback function. Making a blocking I/O call from any of the event callback functions would do freeze of the GUI, as execution of program would not return to the event loop. Barring event loop manager to dispatch any GUI event from the event loop.
4. Solution
Two things in this arena:
- I/O happens through a file descriptor, i.e., socket.
- Along with all other event types, there is a timer event which gets triggered upon timeout expiry.
Solution is proposed to have I/O operation in ‘Timer
’ event callback where ‘select
’, I/O descriptor multiplexer, api is used to check for read or write activity on set/list of file descriptors in non-blocking manner. I/O would happen once ‘select
’ api returns without a timeout. Timeout would be zero making timer event callback completely non blocking.
5. External Reference
‘select
’’ api is blocking when timeout argument is not provided, otherwise non blocking with timeout. ‘select
’ api device driver implementation can be found in book Linux Device Drivers 3rd edition. Published under Oreilly publication.
Open Book - Linux Device Drivers, 3rd Edition
Under subsection ‘Poll and Select’’ in chapter ‘Advanced Char driver Operation’. Whereas Python implementation documentation can be found at docs.python.org.
select - Waiting for I/O completion
‘GUI programming’ event mechanism can be found in Xlib Programming Manual under chapter ‘Events’.
XLIB Programming Manual, Rel. 5, Third Edition
In Python, ‘Event Loop’ is a subsection in topic ‘Async i/o’.
asyncio - Asynchronous I/O
6. Design
Design is made to have a timer event registered with Event Loop manager. Timer event callback function would do ‘select
’ on I/O file descriptor in non blocking timeout mode. I/O would happen if descriptor is set for I/O. Timer callback function would return to event loop processing or zero timeout.
7. Experimental Output
We have a scenario when Blender, a 3D GUI modeling tool, needs to run Python program that is external to the GUI tool and executes as standalone process. By default, Blender supports “in application” Python interpreter. Follow the image below:
Blender In Process/Application Python IDE
We need Blender Python support as out process/application Python application. Something like:
Python client connects to Blender running Python server code through socket connection.
Python server code registers Timer
event with event loop manager. In timer event callback, Python server checks for I/O through ‘select
’ api call. If timeout happens or descriptor is set (data arrives), timer event callback would return after processing the request. Client and Server Python code.
$cat blender_client2.py
import sys,time,select
PORT = 8081
HOST = "localhost"
def main():
import sys
import socket
if len(sys.argv)<2:
print(f'---usage---\npython3 blender_client.py blenderpythonscript.py blenderpythonscript2.py ..')
sys.exit(-1)
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
clientsocket.connect((HOST, PORT))
except Exception as exc:
print(f'E Error in connection with server, probably try after sometime/minutes... Exception type -
> {type(exc)=}')
sys.exit(-1)
else:
print(f'I blender_client connection to server successful')
filestosend=' '.join(sys.argv[1:])
print(f'sending file(s) -> {filestosend} to the server')
clientsocket.sendall(filestosend.encode("utf-8") + b'\x00')
print(f'I blender_client message sent successfully, waiting for response..')
while True:
messagerecved=clientsocket.recv(4096)
if not messagerecved:
print(f'Empty message received, sleeping for 10 secs...')
time.sleep(10)
else:
print(f'Message received {messagerecved=}, exiting...')
clientsocket.close()
break
if __name__ == "__main__":
main()
$cat blender_server2.py
import socket,time,select,re,datetime
import bpy
PORT = 8081
HOST = "localhost"
PATH_MAX = 4096
def execfile(filepath):
import os
global_namespace = {
"__file__": filepath,
"__name__": "__main__",
}
with open(filepath, 'rb') as file:
exec(compile(file.read(), filepath, 'exec'), global_namespace)
def main():
global serversocket,read_list,file_list,connection,result_list
file_list=[]
result_list=[]
connection=None
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((HOST, PORT))
serversocket.listen(5)
print("Listening on %s:%s" % (HOST, PORT))
read_list=[serversocket]
def handle_data():
global file_list,connection,result_list
timeout=20
def send_data():
nonlocal timeout
print(f'I blender_server executing file {file_list[0]} full {file_list=} ')
try:
execfile(file_list[0])
print(f'executed successfully {file_list[0]=}')
result_list.append(f'{file_list[0]} - success')
except Exception as exc:
print(f'Error while executing {file_list[0]=} {exc=}')
result_list.append(f'{file_list[0]} - failed exception {exc}')
file_list[0:1]=[]
timeout=2
if file_list:
send_data()
else:
if connection:
connection.sendall('\n'.join(result_list).encode('utf-8') + b'\x00')
print("response ",'\n'.join(result_list)," sent to client")
connection.close()
connection=None
result_list=[]
readable,writable,errored=select.select(read_list,[],[],0.0)
print(f'E handle_data() {(readable,writable,errored)=} {read_list=} at time ->
{datetime.datetime.now():%H:%M:%S}')
for s in readable:
if s in read_list:
connection, address = serversocket.accept()
print(f'I blender_server connection accepted {connection=} {address=}')
file_list = re.split(r'\s+',re.split(b'\x00',connection.recv(PATH_MAX))[0].decode())
print(f'I blender_server data received {file_list=}')
send_data()
print(f'handle_data, returning {timeout} second timeout..')
return timeout
if __name__ == "__main__":
main()
bpy.app.timers.register(handle_data)
Script can be executed in two terminals as:
- First terminal
blender — python blender_server.py
- Second terminal
python3 blender_client.py <pythonprogram1> <pythonprogram2> <pythonprogram3>
Following image for opening two terminals:
Dual terminal each for client and server
7.1 Program Description
Client program accepts multiple Python programs as command line argument. Program names are joined in a string
and sent to the server. Server parses the request and processes each Python file one by one. Each time it returns from the Timer Event Loop so that Event loop manager would get a chance to process other events.
Python program sent to Python server as concatenated string
. Server process one by one. Making return to event loop after each file processing.
7.2 Video
Video on forking separate terminal for client and server.
Video - Firing python scripts from client to server.
7.3 GIF Animation of Cube Addition and Deletion
GIF animation of cube addition and deletion through scripts fired by client application
7.4 Cube Addition and Deletion Code
$ cat cubeadd_y.py
import bpy
import bmesh
import mathutils
bm = bmesh.new()
bmesh.ops.create_cube(bm, size=4)
mesh = bpy.data.meshes.new('Basic_Cube')
bm.to_mesh(mesh)
mesh.update()
bm.free()
basic_cube = bpy.data.objects.new("Basic_Cube", mesh)
basic_cube.matrix_world.translation += basic_cube.matrix_world.to_3x3() @
mathutils.Vector((0.0,6.0,0.0))
bpy.context.collection.objects.link(basic_cube)
$ cat cubeadd_x.py
import bpy
import bmesh
import mathutils
bm = bmesh.new()
bmesh.ops.create_cube(bm, size=4)
mesh = bpy.data.meshes.new('Basic_Cube')
bm.to_mesh(mesh)
mesh.update()
bm.free()
basic_cube = bpy.data.objects.new("Basic_Cube", mesh)
basic_cube.matrix_world.translation += basic_cube.matrix_world.to_3x3() @ mathutils.Vector((-
6.0,0.0,0.9))
bpy.context.collection.objects.link(basic_cube)
$ cat cubedelete.py
import bpy
import mathutils
try:
cube = bpy.data.objects['Cube']
bpy.data.objects.remove(cube, do_unlink=True)
except:
print("Object bpy.data.objects['Cube'] not found")
bpy.ops.outliner.orphans_purge()
8. How It Works ?
Two Python programs client and server interacts with each other through ‘socket’ inter process communication. Socket can be used among inter computer communications also. IP address needs to be real IP address of the server. Blender starts in scripting mode with ‘ — python’ as command line argument. Blender starts python program in main thread. Python program rather than doing any task, it registers a ‘Timer Event’ to event loop through code.
Activity diagram for interaction flow
‘bpy.app.timers.register(handle_data)
’ passing ‘handle_event
’ as callback function. Event callback function ‘handle_data
’ is called in main loop and it uses ‘select
’ as I/O multiplexer to handle the I/O in non-blocking mode. Once connection arrives ‘read descriptor is set’, connection request is read and processed. In case of multiple files, Timer callback returns (to event main loop) with timeout of 0 seconds. Here, 2 seconds is used to make explanation more visual. Returning to event loop between each Python script file processing, gives event loop manager chance to execute other GUI event, making GUI appear interactive.
9. Further Enhancements
Client Python scripts can be edited through IDE editor. Where editors will have GUI button option and context menu options to execute the script.
Proposed IDE editor with separate Tool Bar Button and context menu to execute the script
Data/string communications with server would be displayed in ‘docket’ window.
10. Further Studies
- Advanced Programming in the UNIX Environment, W. Richard Stevens. Addison-Wesley Professional Computing Series
- TCP/IP Illustrated, W. Richard Stevens : The Protocols Volume 1, The implementation Volume 2, TCP for Transactions, HTTP, NNTP and the UNIX Domain Protocols Volume 3