Lately I've been playing around with automation testing based on image processing.
The problem I am facing - I am trying to make a program that screenshots a webpage loaded into WebBrowser Control to an image, finds some images based on image processing and simulates user clicks on them, all this should be working while minimised.
The image recognition part is working perfectly.
The screenshot part was taken from
. It bugs when the page has flash object. If I click on it manually, the screenshot is all white, and if I click outside of the flash object manually it's ok.
The clicking part rarely works, I couldn't find what causes it to glitch. I'm posting code, couldn't remember where I got it from.
class Interactor
{
public static WebBrowser WebBrowser;
[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = false)]
static extern IntPtr SendMessage(IntPtr hWnd, uint Msg, IntPtr wParam, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true)]
static extern IntPtr GetWindow(IntPtr hWnd, uint uCmd);
[DllImport("user32.dll", CharSet = CharSet.Auto)]
static extern int GetClassName(IntPtr hWnd, StringBuilder lpClassName, int nMaxCount);
public static void ClickAt(int x, int y)
{
x += 15;
y += 232;
IntPtr handle = WebBrowser.Handle;
StringBuilder className = new StringBuilder(100);
while (className.ToString() != "Internet Explorer_Server")
{
handle = GetWindow(handle, 5);
GetClassName(handle, className, className.Capacity);
}
IntPtr lParam = (IntPtr)((y << 16) | x);
IntPtr wParam = IntPtr.Zero;
const uint downCode = 0x201;
const uint upCode = 0x202;
SendMessage(handle, downCode, wParam, lParam);
SendMessage(handle, upCode, wParam, lParam);
}
}
I'd prefer solutions using C# and WinForms, but other solutions are wellcome too.