Introduction
VNC and Remote Desktop software were around for a long while now. Undoubtedly, they work great for most purposes but, I wanted to bring the old VNC / Remote Desktop concept over the web, to allow full web-application integration and full cross-browser cross-platform support out-of-the-box.
Despite its name, ThinVNC is not a traditional VNC, as it does not implement the AT&T RFB protocol. Instead, it is built on top of today's web standards: AJAX, JSON and HTML5.
The Code (Part I)
In this first part, we'll see the code to do the screen capture. The standard approach is to capture the whole desktop. However, in this case, we'll capture every window individually, applying clipping regions and saving the individual bitmap for later comparison and difference extracting.
Firstly, we need to enumerate all visible top windows:
TWin = class(TObject)
private
Wnd : Hwnd;
Rect : TRect;
Pid : Cardinal;
public
constructor Create(AWnd:HWND;ARect:TRect;APid:Cardinal);
end;
function EnumWindowsProc(Wnd: HWnd; const obj:TList<TWin>): Bool; export; stdcall;
var ProcessId : Cardinal;
R,R1 : TRect;
Win : TWin;
begin
Result:=True;
GetWindowThreadProcessId(Wnd,ProcessId);
if IsWindowVisible(Wnd) and not IsIconic(wnd)then begin
GetWindowRect(Wnd,R);
IntersectRect(R1,R,Screen.DesktopRect);
if not IsRectEmpty(R1) then begin
win := TWin.Create(Wnd,R,ProcessId);
obj.Add(win);
end;
end;
end;
procedure GetProcessWindowList(WinList:TList<TWin>);
begin
WinList.Clear;
EnumWindows(@EnumWindowsProc, Longint(WinList));
end;
We want to keep a list of windows, with their basic attributes and their bitmaps, so we can compare with the new ones and send the differences to the client. Here we merge the window list into a list of TWindowMirror
:
TWindowMirror = class
private
FIndex : Integer;
FRgn : HRGN;
FHandle : THandle;
FBoundsRect : TRect;
FProcessId : Integer;
FImage : TBitmap;
FDiffStreamList : TList<TImagePart>;
...
...
end;
procedure TMirrorManager.RefreshMirrorList(out OneMoved:Boolean);
procedure GetProcessWindowList(WinList:TList<TWin>);
begin
WinList.Clear;
EnumWindows(@EnumWindowsProc, Longint(WinList));
end;
var
wl : TList<TWin>;
n : Integer;
wm : TWindowMirror;
begin
OneMoved:=False;
wl := TList<TWin>.Create;
try
// Enumerates top windows
GetProcessWindowList(wl);
try
for n := wl.Count - 1 downto 0 do begin
// Looks for a cached window
wm:=GetWindowMirror(FMirrorList,wl[n].Wnd);
if assigned(wm) then begin
if IsIconic(wl[n].Wnd) then
wm.SetBoundsRect(Rect(0,0,0,0))
else wm.SetBoundsRect(wl[n].Rect);
// Returns true when at least one window moved
OneMoved:=OneMoved or (DateTimeToTimeStamp(Now-wm.FMoved).time<MOVE_TIME);
end else begin
// Do not create a TWindowMirror for invisible windows
if IsIconic(wl[n].Wnd) then Continue;
wm:=TWindowMirror.Create(Self,wl[n].Wnd,wl[n].Rect, wl[n].pid);
FMirrorList.Add(wm);
end;
// Saves the zIndex
wm.FIndex:=wl.Count-n;
// Generates clipping regions
wm.GenRegions(wl,n);
end;
finally
ClearList(wl);
end;
// Sorts the mirror list by zIndex
FMirrorList.Sort;
finally
wl.free;
end;
end;
Now, we do the capture:
function TWindowMirror.Capture(ANewImage:TBitmap): Boolean;
function BitBlt(DestDC: HDC; X, Y, Width, Height: Integer; SrcDC: HDC;
XSrc, YSrc: Integer; Rop: DWORD): BOOL;
begin
// Capture only visible regions
SelectClipRgn(DestDC,FRgn);
result:=Windows.BitBlt(DestDC, X, Y, Width, Height, SrcDC,
XSrc, YSrc, Rop);
SelectClipRgn(DestDC,0);
end;
var
DC : HDC;
RasterOp,ExStyle: DWORD;
begin
RasterOp := SRCCOPY;
ExStyle:=GetWindowLong(FHandle, GWL_EXSTYLE);
if (ExStyle and WS_EX_LAYERED) = WS_EX_LAYERED then
RasterOp := SRCCOPY or CAPTUREBLT;
DC := GetDCEx(FHandle,0,DCX_WINDOW or DCX_NORESETATTRS or DCX_CACHE);
try
Result:=BitBlt(ANewImage.Canvas.Handle,0,0,
Width(FBoundsRect),Height(FBoundsRect),DC,0,0, RasterOp)
finally
ReleaseDC(FHandle,DC);
end;
end;
Now that we have captured all visible regions, we need to get the bitmap differences against the previous capture. We do this by looping through the windows, then their visible regions and finally calculating the regions where we find bitmap differences:
function TWindowMirror.CaptureDifferences(reset:boolean=false): Boolean;
....
begin
...
result:=Capture(TmpImage);
if result then begin
...
ra:=ExtractClippingRegions(Rect(0,0,TmpImage.Width,TmpImage.Height));
for n := 0 to Length(ra) - 1 do begin
ra2:=GetDiffRects(FImage,TmpImage,ra[n]);
for m := 0 to Length(ra2) - 1 do begin
Jpg := TJpegImage.Create;
...
CopyBmpToJpg(Jpg,TmpImage,ra2[m]);
FDiffStreamList.Add(TImagePart.Create(rbmp,'jpeg'));
Jpg.SaveToStream(FDiffStreamList[FDiffStreamList.Count-1].FStream);
...
Bitblt(FImage.Canvas.Handle,
ra2[m].Left,ra2[m].Top,Width(ra2[m]),Height(ra2[m]),
TmpImage.Canvas.handle, rbmp.Left,ra2[m].Top,SRCCOPY);
end;
end;
...
end;
In the next post, we'll focus on the protocol with the client and the client code. A post with updated source code can be found here.
History
ThinVNC is currently in alpha stage and this version must be taken as a technology preview. Remote mouse input is partially implemented, remote keyboard input is still missing.