The purpose of this article is how to supply a Super Web Content Ecosystem to existing or under developing 64-bit desktop applications, especially those applications developed using .NET, C++/MFC, and Win32 SDK, in order to make desktop applications surpass Current Web Browsers and make the native desktop window object become an Extended Web DOM Element comparable to html div.
What Problem Do We Need to Solve?
If Web DOM Description Technology can directly and deeply modify the UI Structure of Native Window Objects of Windows Operating System, then the Desktop Application World will undergo tremendous changes.
The problem we will solve is how to provide a "Super Web Content Ecosystem" for existing or under developing 64-bit desktop applications, especially those applications developed using .NET, C++/MFC, and Win32 SDK, in order to enable Ordinary Desktop Applications to have web content processing capabilities beyond Current Web Browsers, and make the Native Desktop Window Object become an Extended Web DOM Element comparable to "html div element", even if the developer does not have the source code of these applications, specifically:
- Allowing developers to directly treat Ordinary Desktop Applications as a "Super Web Browser" that surpasses Standard Web Browsers, developers can write any number of Webpages directly for Ordinary Desktop Applications.
- Provide a Dynamic Description Driven Mechanism based on Web DOM to control "the UI Structure of Native Desktop Window Objects", developers can assemble "Composite Native Window Objects" in a way like "organizing HTML div elements within a webpage to express rich webpage content".
- All Large Language Model Techniques provide the friendliest support for Web Browsers, once desktop applications can be viewed as "Super Web Browser", this means that developers can deeply integrate LLM technology into the content ecosystem of ordinary desktop applications, even if they do not have the source code of the corresponding desktop application.
The Background and Vision of This Article
So far, developers have compiled countless 64-bit Desktop Applications based on .NET, C++/MFC and Win32 SDK, meanwhile there are countless Desktop Applications currently under development. When a desktop application completes native code compilation, its rigidity will dominate its runtime behavior, specifically, the structure of the native desktop window is basically in a relatively fixed state at runtime , making it difficult for developers to reasonably adjust and extend it at runtime. When we face a compiled MFC Desktop Application or a compiled WinForm Desktop Application:
If we wish to add a "Web Browser Tabs Group" to their Main Window Objects, or if we want to use Large Language Model Techniques such as ChatGPT to modify their Main Window Structures, as shown in the following figures:
When we face the above problems, we may powerless, even if we have the source code of corresponding application, our existing technology is difficult to solve such problems.
If we have the possibility to discover the "dynamic description driven mechanism" that control the structure of native window objects, then for those desktop applications that already compiled or under development, whether it is based on Web technology or ChatGPT technology to adjust or modify the UI structure of native window objects, the "dynamic description driven mechanism" will become the driving force for innovation in "dynamic application content", and we will enter a brand-new world of desktop applications.
The vision of this article is to reveal that every 64-bit desktop application has webpage parsing capabilities beyond standard web browsers, thus having its own web content ecosystem , allowing developers to directly write any number of web pages for 64 bit desktop applications. All modern Large Language Model Techniques provide the friendliest support for Web Browsers, which undoubtedly lays an objective foundation for the deep integration of LLM technology into desktop applications.
AIGC Browser and Dynamically Integrate X Objects
AIGC Browser is an Open-Source Chromium-Based Web Browser that views native window objects as "Description Driven Super HTML Elements", just like organizing sentences by using a group of tokens in NLP, AIGC Browser allows developers to assemble powerful "Composite Windows" based on Web DOM Technology starting from a "existing child window", this gives "Web DOM Technology" strong control force for the structure of native window objects, allowing Web Technology and Large Language Model Technology to be fully integrated into modern desktop software.
If the executable file of your application is "exeName.exe", and the Initialize Webpage named "exeName.app.html" exists in the folder containing this executable file, then:
- For Desktop application that has been compiled, start "AIGC Browser" after this application is launched or start this application after "AIGC Browser" is launched.
- For Desktop application under development, add or modify no more than 5 lines of Native Code, allowing applications to directly activate their built-in AIGC Browser.
Regardless of which of the above two scenarios your application is in, AIGC browser will generate corresponding "Web DOM" elements for the native window objects of your application, and web technology will fully integrate into the content ecosystem of your application. your application will accommodate all the features of modern web browsers and will generate more and more web pages, as shown in the following figure, with the help of hyperlinks in "exeName.app.html":
your desktop application will directly open any number of web pages, generating various browser windows:
This means that developers can directly write any number Webpages for 64-bit desktop applications, as if they were a modern browser themselves.
When the "Whole Web Browser" directly becomes a part of the Desktop Application Functional Structure, UI components that comply with industry standards, such as .NET UI components, ActiveX Control, etc., will naturally become some kind of “Extended Web DOM elements”. Meanwhile, as all Large Language Model Technologies support Web Browsers, developers can directly use technologies such as ChatGPT in existing desktop applications. For example, ChatGPT's "multimodal input" technology can be used to directly manipulate native desktop windows and may add new UI elements to them.
The "Composite Window" in desktop applications can treat the "Web Browser Window" as its "Dynamic Child Window", and further make this "Composite Window" a super web content portal. For example, an MFC Frame window as shown below
It can be used as a Super Browser Window in a practical scenario:
AIGC Browser provide a Highly Universal, Least Dependent and Source Code Independent Solution for Dynamically Integrating X objects into Ordinary 64-bit Desktop Applications, where "X objects" can be any one or a group of the following:
- Full-Function Chromium-Based Web Browser Subsystem.
- .NET Core/.NET Framework Engine.
- Any type of window object you are interested in, such as web pages, WinForms/User Control, MFC windows, or more general desktop window object types.
Native Desktop Window Object as Web Content Entry Point
If the runtime screen position of a native window object is uniquely dependent on its parent window, then the window is called the "Window Nucleus" of its parent window, and the parent window is called a "Nucleated Window".
As shown in the following figure, we can see a ordinary MFC Frame window, and the "FormView" object presented in its center position is a "window nucleus":
At runtime, we assign different Web DOM Elements to the "Window Nucleus(FormView)", and the periphery of the "FormView" presents completely different structural changes.
If we concatenate the following Web DOM elements into the "FormView" object in MFC Frame:
<someWindow>
<nucleus>
<xobj rows="3" cols="3"
width="150,100,100"
height="200,180,100">
<xobj></xobj>
<xobj></xobj>
<xobj></xobj>
<xobj></xobj>
<xobj objid="nucleus">
</xobj>
<xobj></xobj>
<xobj></xobj>
<xobj></xobj>
<xobj></xobj>
</xobj>
</nucleus>
</someWindow>
We will see a 'layout structure' presented around the periphery of 'FormView':
If we concatenate the following Web DOM elements into the "FormView" object in MFC Frame:
<someWindow>
<nucleus>
<xobj rows="1" cols="2" width="350," id="xxx">
<xobj id=""></xobj>
<xobj rows="3" cols="1" heigh="150,150" id="xxx">
<xobj id=""></xobj>
<xobj style="23">
<xobj objid="nucleus"
caption="Using Tabbed Window">
</xobj>
<xobj objid="SunnyCtrl.UserControl3,SunnyCtrl"
caption="Second Page">
</xobj>
<xobj objid="SunnyCtrl.UserControl2,SunnyCtrl"
caption="Third Page">
</xobj>
</xobj>
<xobj id=""></xobj>
</xobj>
</xobj>
</nucleus>
</someWindow>
We will see another 'layout structure' presented around the periphery of 'FormView':
Through the above two scenarios, we can see that "Window Nucleus" concatenate different Web DOM elements to produce completely different peripheral layout structures. Since there are an infinite number of Web DOM elements, this means that at runtime, there will be endless structural changes on the periphery of "Window Nucleus", which in fact reveals the existence of a "new type of web content entrance" associated with "Window Nucleus".
Install and Build AIGC Browser
Developers can visit AIGCBrowser to obtain installation packages of AIGC Browser. After installation AIGC Browser, open "AIGCBrowser.sln" to obtain the Source Code of AIGC Browser and various examples:
After AIGC Browser is installed, developer will get a Visual Studio Solution "AIGCBrowser.sln", which includes the Source Code of AIGC Browser and various examples.
If developers are familiar with C++, they can compile the "core components" of AIGC Browser directly. If developers need to have a deep understanding of the specific integration details of X-objects, they can refer to the "UniversePro" project. For the ".Net Part", they need to refer to the "CosmosPro" project. If you are a C# developer, you can ignore all the associated projects and use them directly.
If developers are familiar with the Chromium Project and correctly installed the AIGC Browser installation package, they will obtain the "ChromiumSrcPatch" folder. Developers first need to obtain the Chromium Project Code Branch that matches AIGC Browser Version and ensure that this Branch is compiled properly. Then, they need to copy all subfolders contained in the ChromiumSrcPatch folder to this Code Branch and recompile the updated Chromium Code Branch to finish the compilation of the AIGC Browser Chromium subsystem. Due to the complexity of the code and the fact that compiling Chromium-Based C++ code typically takes 6-8 hours, most developers should not need to do this step and can directly focus on desktop application development.
About manifest configuration
The Webpage Rendering Technology of modern browsers requires Win10 Compatibility support for the executable files of the browser, therefore, developers need to provide a manifest file that meets the requirements during the compilation process. For specific information on manifest configuration, please refer to the manifest configuration related documentation provided by Microsoft. The basic structure of a typical manifest file "aigc.manifest" is as follows:
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}" />
<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}" />
<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}" />
<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}" />
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />
</application>
</compatibility>
</assembly>
In the compilation process, developers need to configure the manifest files required for the compilation process as shown in the following figure:
For desktop applications that do not provide manifest configuration, AIGC Browser may adjust the manifest configuration of the Corresponding Desktop Applications to make them compatible with Win10, developers will see the following Messagebox :
The reason why this Messagebox appears is that there is no manifest configuration for "Win10 compatibility" in the corresponding desktop application source code. AIGC Browser will "fix" these missing configurations at the specific application startup stage.
Using Visual Studio Wizard:
Every desktop application generated by Visual Studio Wizard has a web development mechanism after compilation, although developers are completely unaware of its existence or have not deliberately prepared for its existence. For Desktop Applications generated by Visual Studio Wizards (WinForms, MFC Applications, etc.), developers can first provide a "Blank Initialization Page Without Any Content", and at runtime the AIGC Browser will generate an initialization page based on this page that matches the application type for developers to continue with subsequent work.
Due to length limitations in the article, we will only provide a brief demonstration of the application process of AIGC Browser for .Net Framework WinForm type applications. We suggest that readers of this article experience the application process of AIGC Browser one by one based on various application types generated by Visual Studio Wizard such as .Net Core and MFC etc.
We first use the "Visual Studio Wizard" to create a simplest "WinForm application WindowsFormsApp1", you need add a panel control with Dock property value "DockFill" on the WinForm object:
Select 64-bit compilation, and all options are default. After compilation, we will obtain the executable file “WindowsFormsApp1.exe”. In the folder containing this executable file, provide a "blank" web page named "WindowsFormsApp1.app.html". Start AIGC Browser after starting this application, we will see the following runtime WinForm object, and a "Web Browser Window":
We have noticed that the WinForm application here is just an ordinary desktop application, and its corresponding project does not contain any controls or code related to Web browser. However, in its runtime process, it does indeed create "Web Browser Window Objects". We see that "Web Browser Window Objects" can be a "component" of WinForm objects. If we replace the "C# Project" with the "MFC Project", we will obtain completely consistent experimental conclusions.
Using Examples
All examples in "AIGCBrowser.sln" can directly activate the AIGC Browser based on native code, which means that after compiling these examples, once the developer provides the corresponding application's initialization web page, the relevant web development mechanism is also activated, and there is no need to launch the AIGC Browser separately.
For .NET developers, they need to pay attention to 'AIGCBrowserSharedProject' Project Item in "AIGCBrowser.sln", it is a "Shared Project", Once other .Net Projects reference this project, developers only need to replace
Application.Run
with
AIGC.AIGCApp.InitCosmos
to complete activating its web content ecosystem directly at runtime.
For C++developers, we provide a pair of C++source files:
aigc.h and aigc.cpp
these two files contain derived classes of the CWinApp(Ex) class for MFC applications and the CAtlExeModuleT class for ATL applications as well as necessary C++classes for Win32 SDK application.
MFC developers only need to replace the base class of the Application class
from "CWinApp(Ex)" to "CAIGCApp(Ex)"
in their MFC applications, after recompiling the web content ecosystem of the application itself can be directly activate at runtime. For applications such as ATL and Win32 SDK, this solution provides a processing mechanism like MFC applications. For specific details, please refer to the specific examples in the solution.
How to use AIGCSDK
If you need to directly activate its web content ecosystem within the desktop application, you need Download AIGC SDK package, and after decompression, you can obtain the SDK package. Developers should note that all desktop applications related to AIGC SDK must be compiled based on 64 bit .
How to use AIGCSDK in C# WinForm application
The AIGC SDK includes a C# Shared Project 'AIGCBrowserSharedProject.shproj', which contains a C# static class with the following structure:
using System;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Text;
namespace AIGC
{
static public class AIGCApp
{
static IntPtr initDll = IntPtr.Zero;
[DllImport("kernel32.dll")]
public static extern IntPtr LoadLibrary(string dllToLoad);
[DllImport("kernel32.dll")]
public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void InitWebRT(IntPtr IUnkPtr, IntPtr IUnkWndPtr);
private static string BuildConfigDataFile(string strExeName, string strProductName, string strCompanyPathName)
{
string _strProductName = strProductName.ToLower();
string _strCompanyPathName = strCompanyPathName.ToLower();
StringBuilder sb = new StringBuilder();
sb.Append(Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData)).Append("\\TangramData\\").Append(strExeName).Append("\\");
string _strConfigDataFile = sb.ToString().ToLower();
if (Directory.Exists(_strConfigDataFile) == false)
Directory.CreateDirectory(_strConfigDataFile);
using (var md5 = MD5.Create())
{
StringBuilder sb2 = new StringBuilder();
_ = sb2.Append(_strConfigDataFile).Append("@").Append(_strCompanyPathName).Append("@").Append(_strProductName);
var result = md5.ComputeHash(Encoding.UTF8.GetBytes(sb2.ToString()));
_strConfigDataFile += BitConverter.ToString(result).Replace("-", "");
}
_strConfigDataFile += "\\";
if (Directory.Exists(_strConfigDataFile) == false)
Directory.CreateDirectory(_strConfigDataFile);
_strConfigDataFile += strExeName;
_strConfigDataFile += ".tangram";
return _strConfigDataFile;
}
public static bool InitCosmos(object StartObj, object MainWndObj = null)
{
initDll = LoadLibrary(@"universe.DLL");
if (initDll == IntPtr.Zero)
{
String strCfgFile = BuildConfigDataFile("aigcbrowser", "aigcbrowser", "Tangram Team");
if (File.Exists(strCfgFile))
{
string strData = File.ReadAllText(strCfgFile);
String strTemp = strData.Substring(strData.IndexOf("Universe") + 8);
if (String.IsNullOrEmpty(strTemp) == false)
{
strTemp = strTemp.Substring(0, strTemp.IndexOf(".dll") + 4);
initDll = LoadLibrary(strTemp.Substring(strTemp.IndexOf(":") - 1));
}
}
}
if (initDll == IntPtr.Zero)
{
StringBuilder sb = new StringBuilder();
sb.Append(Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles)).Append("\\Tangram\\AIGCBrowser\\universe.dll");
string strLib = sb.ToString();
if (File.Exists(strLib))
{
initDll = LoadLibrary(strLib);
}
}
if (initDll != IntPtr.Zero)
{
IntPtr fnInitWebRT = GetProcAddress(initDll, "InitWebRT");
if (fnInitWebRT != IntPtr.Zero)
{
InitWebRT InitWebRT = (InitWebRT)Marshal.GetDelegateForFunctionPointer(fnInitWebRT, typeof(InitWebRT));
InitWebRT((StartObj != null) ? Marshal.GetIUnknownForObject(StartObj) : IntPtr.Zero, (MainWndObj != null) ? Marshal.GetIUnknownForObject(MainWndObj) : IntPtr.Zero);
return true;
}
}
return false;
}
}
}
The developer's C# WinForm Project needs to reference sharedproject 'AIGCBrowserSharedProject.shproj':
after referencing AIGCBrowserSharedProject, open the 'program.cs' file of this C# project and rewrite the main function as follows:
static class Program
{
[STAThread]
static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Form startForm = new Form1();
if (AIGC.AIGCApp.InitCosmos(startForm) == true) return;
Application.Run(startForm);
}
}
How to use AIGCSDK in C++ Application
How C++/ATL/MFC developers handle precompiled header files
Copy "aigc.h" and "aigc.cpp" from AIGC SDK to C++ desktop software project. Normally, C++desktop application projects will have a pair of "precompiled" header files, "stdafx.h" and "stdafx.cpp", or "pch.h" and "pch.cpp", depending on the version of Visual Studio.
In the C++ Desktop Application Project, open the precompiled file "stdafx.h" or "pch.h", and add the following code:
#include "AIGC.h"
at the bottom of "stdafx.h" or "pch.h"
#pragma once
#ifndef STRICT
#define STRICT
#endif
#include "targetver.h"
#define _ATL_APARTMENT_THREADED
#define _ATL_NO_AUTOMATIC_NAMESPACE
#define _AFX_NO_MFC_CONTROLS_IN_DIALOGS
#define _ATL_CSTRING_EXPLICIT_CONSTRUCTORS // some CString constructors will be explicit
#include "framework.h"
#include "AIGC.h"
Next, open "stdafx.cpp" or "pch.cpp", and add the following code：
#include "AIGC.cpp"
at the bottom of "stdafx.cpp" or "pch.cpp":
#include "stdafx.h"
#include "AIGC.cpp"
How to use AIGCSDK in MFC application
In the MFC project, open the "header file" where the Application Class is located:
class CMFCSDIApp : public CWinApp
{
public:
CMFCSDIApp() noexcept;
public:
virtual BOOL InitInstance();
virtual int ExitInstance();
afx_msg void OnAppAbout();
DECLARE_MESSAGE_MAP()
};
Replace the base class 'CWinApp(Ex)' of the Application Class with 'CAIGCApp(Ex)'
class CMFCSDIApp : public CAIGCWinApp
{
public:
CMFCSDIApp() noexcept;
public:
virtual BOOL InitInstance();
virtual int ExitInstance();
afx_msg void OnAppAbout();
DECLARE_MESSAGE_MAP()
};
How to use AIGCSDK in ATL application
In the ATL project, open the "header file" where the AtlExeModule Class is located:
#include "pch.h"
#include "framework.h"
#include "resource.h"
#include "ATLApp_i.h"
using namespace ATL;
class CATLAppModule : public CAtlExeModuleT< CATLAppModule >
{
public:
DECLARE_LIBID(LIBID_ATLAppLib)
DECLARE_REGISTRY_APPID_RESOURCEID(IDR_ATLAPP, "{5a319c27-7415-4304-80ee-4bce0f6759da}")
};
CATLAppModule _AtlModule;
extern "C" int WINAPI _tWinMain(HINSTANCE , HINSTANCE ,
LPTSTR , int nShowCmd)
{
return _AtlModule.WinMain(nShowCmd);
}
Replace the base class 'CAtlExeModuleT' of the AtlExeModule Class with 'CAIGCModuleT'
#include "pch.h"
#include "framework.h"
#include "resource.h"
#include "ATLApp_i.h"
using namespace ATL;
class CATLAppModule : public CAIGCModuleT< CATLAppModule >
{
public:
DECLARE_LIBID(LIBID_ATLAppLib)
DECLARE_REGISTRY_APPID_RESOURCEID(IDR_ATLAPP, "{5a319c27-7415-4304-80ee-4bce0f6759da}")
};
Conclusion and Points of Interest
Every Windows 64 bit desktop application has a 'Super Web Content Ecosystem', and even if the developer does not have the source code for this application, they can still use AIGC Browser to open this 'Web Ecosystem Mode', which is crucial for the productivity of desktop applications. If developers plan to integrate AI technologies such as ChatGPT into ordinary desktop applications, then after opening this "super web content ecosystem", integrating "ChatGPT" or Large Language Model Techniques technologies into desktop applications will become a web technology problem, which is also the simplest way for web developers to fully participate in desktop application development.
For technical details about AIGC Browser, you can visit our GitHub site
