Introduction
The reason behind writing this article comes from a project I was working on recently for a client. I was tasked with writing .NET code that essentially gets run by another application as a plugin. This plugin interface allowed us very limited scripting access to an API which allowed very little control over the application itself or the UI.
The Application
Without revealing too much (it was proprietary software), the application basically loaded and showed a list of images which the user was then meant to go through via clicking each individual image and perform certain work using information on the selected image.
The Request
Every task the user had to perform could be done from the keyboard and never required use of the mouse except for when they needed to go to another image, which had to be clicked. This was apparently causing a real issue with the users by breaking their flow of work. They wanted to be able to automatically skip the image based on whether or not any work needed to be done for that particular image, something that couldn't be known until that particular image had been selected and loaded by the application.
The Problem
While I could know whether or not the selected image required work from the user, via info exposed through the scripting API, I had know way to select another image from the code. There were no exposed methods that allowed such control or manipulation.
The Realization
I had limited access to a few structures representing work done by the users for each image. While stepping through the code in Visual Studio I noticed that a particular type of object had a private field of type FrameworkElement, which made me realize that this application was a WPF application, which obviously meant (duh) XAML.
Background
To really understand what I'm actually doing in this next part, you're going to have know a bit about DotPeek, mainly how to
However all this is just part of researching the problem, something that could also be done several other ways with programs such as ILSpy (has an awesome BAML decompiler), .NET Reflector, or basically anything other decompiler.
It would also help to understand a bit about RoutedEvents in WPF, but you don't need any extremely deep knowledge to follow what is going on.
Researching the "Image Click"
Okay, so, I knew that I would be able to manipulate certain things in the UI through some methods exposed to us through the PresentationFramework assembly used by WPF applications, but to know what exactly might and might not work, I needed a better idea of what the application was actually doing when a user clicked the next image, or any image for that matter.
So I opened DotPeek and followed the basic steps listed above to get the assembly for the running application in the process explorer, export it to a project, and set up DotPeek as my symbol server so I could step through the code and see what it is doing.
After stepping through for a while I noticed that when a user clicks an image, the actual type of the element they were clicking was a type that derived from the base Button class, called ImageButton. All of the buttons of this type were stored in a ListView named "ImageList" and clicking one of these actually just fires a standard routed click event, something that can be emulated programmatically. Sounded like a plan to me!
The Solution
So, my general plan of action was along the lines of
- Gain access to and traverse the XAML (somehow) to find Images the users can click, buttons of type ImageButton
- Figure out exactly which image is currently selected (probably through one of the properties on the containing ListView such as SelectedItem or SelectedIndex, or maybe check if any of the buttons contain the ListBoxItem.IsSelectedProperty dependency property and if its value is set to true)
- Programmatically perform a routed click event with one of several techniques, many of which are displayed in this seriously insightful StackOverflow question
Access the XAML
I had a couple of different methods in mind for how I could do this, one of which was latching on to the FrameworkElement member I had discovered earlier using reflection on the one of the structures exposed to me through the API, then traverse the XAML using the static VisualTreeHelper class from the PresentationCore assembly. I gave it a try, got it implemented in my scripting code, and Bam! Worked perfectly.
Get the private FrameworkElement member, which was named "_wpfElement"
var exposedObject = SomeStaticClass.GetExposedStructure("someObject");
FrameworkElement element = exposedObject.GetType()
.GetField("_wpfElement", BindingFlags.NonPublic | BindingFlags.Instance)
.GetValue(exposedObject) as FrameworkElement;
However, once I had this, I realized I might not need this code at all. I assumed that the main window is what I really needed anyway to make sure that I always started at the root of the XAML to ensure I only ever needed to recurse one way through the visual tree -- descending. So, instead, I remembered a much easier method of getting the current window as a FrameworkElement.
private static FrameworkElement GetWindowRoot()
{
return Window.GetWindow(Application.Current.MainWindow);
}
Now that I had a GetWindowRoot() method, I had a starting point for searching the visual tree.
Next, I needed a recursive search function to fill me a list of buttons I needed.
private static void FillButtonList(FrameworkElement currentElement, List<FrameworkElement> buttons)
{
if (currentElement.GetType().ToString().ToLower().Contains("imagebutton"))
{
buttons.Add(currentElement);
}
for (int i = 0; i < VisualTreeHelper.GetChildrenCount(currentElement); ++i)
{
var next = VisualTreeHelper.GetChild(currentElement, i) as FrameworkElement;
if (next != null)
{
FillButtonList(VisualTreeHelper.GetChild(currentElement, i) as FrameworkElement, buttons);
}
}
}
private static List<FrameworkElement> GetImageButtons(FrameworkElement rootElement)
{
List<FrameworkElement> buttonList = new List<FrameworkElement>();
FillNodeButtonList(rootElement, buttonList);
return buttonList;
}
Now that I've gotten the list of ImageButton's, I can move on to figuring out which is selected.
Get the selected image
So, as I mentioned above, my first instinct was to check the containing ListView's SelectedItem and SelectedIndex properties to find out which image is currently selected, however, no luck. SelectedItem was always null no matter what was selected and SelectedIndex was always -1.
My next thought was to check for the dependency properties ListBoxItem.IsSelectedProperty and ListViewItem.IsSelectedProperty, however that was unhelpful as well, as none of the buttons contained either of these.
I was still convinced, however, that the ImageButton class probably had some sort of property or field indicating it's selection status, so I stepped through the code with the debugger and analyzed the class members for the ImageButton objects I retrieved using the GetImageButtons function shown above. I was looking for a member named something like "selected" or maybe "status", and voila! I found a private boolean field called "_activated". Just to be sure this was what I was looking for, I checked the value of every button's _activated field against which ImageButton I actually had selected, and it was correct! The object representing the selected ImageButton had a value of true for the _activated field while the rest had a value of false.
I simply wrote a function to extract the index of the object with an _activated value of true to indicate which image was selected.
public static int GetCurrentPageIndex()
{
try
{
var rootElement = GetWindowRoot();
var buttons = GetImageButtons(rootElement);
for (int i = 0; i < buttons.Count; ++i)
{
var button = buttons[i];
bool activated =
button.GetType()
.GetField("_activated", BindingFlags.NonPublic | BindingFlags.Instance)
.GetValue(button) as bool;
if (activated)
{
return i;
}
}
return 0;
}
catch (Exception)
{
throw new Exception();
}
}
Awesome. Now that I have my list of ImageButton's and I can tell which is selected, I just need to create a method to select the next image, or really any image I want.
Set the selected image
So, like I mentioned earlier, there are several ways to programmatically invoke a click, or other RoutedEvent's, on an object in WPF, however my favorite, because of it's sheer simplicity and readability, is the technique suggested in the previously mentioned StackOverflow question using the RaiseEvent method.
public static void MoveToNextPage()
{
try
{
var currentlySelected = GetCurrentPageIndex();
var newSelectionIndex = ++currentlySelected;
SelectPage(newSelectionIndex);
}
catch (Exception)
{
}
}
public static void MoveToPreviousPage()
{
try
{
var currentlySelected = GetCurrentPageIndex();
var newSelectionIndex = currentlySelected == 0 ? currentlySelected : --currentlySelected;
SelectPage(newSelectionIndex);
}
catch (Exception)
{
}
}
private static void SelectPage(int selectionIndex)
{
var rootElement = GetWindowRoot();
var buttonList = GetImageButtons(rootElement);
int currentlySelected = GetCurrentPageIndex();
buttonList[selectionIndex].RaiseEvent(new RoutedEventArgs(ButtonBase.ClickEvent));
}
There you have it!
The main takeaway from this for me, and hopefully for anyone reading it, was that I realized I was able to access and utilize the application's XAML through use of the application's current window (or any available FrameworkElement such as those found through reflection) as a FrameworkElement to gain access to the visual tree, from which I was opened a whole new world of functionality from my code.
I was finally able to save the users a lot of time and keep from breaking their work flow by setting it to automatically select the next image whenever the currently selected image required no work. It also gave me a lot of room to solve issues that arose later, such as
- Sometimes when a set of images were loaded, they were able to know ahead of time that the first image required no work, and now instead of having to click to the next image, I could simply set the code to skip to the second image automatically
- I was able to set up a hot key that allowed the users to move back and forth through the images as they pleased without having to leave the keyboard
Was there a better way to do this?
Quite possibly, but this worked great for the situation. It allowed me to minimize reflection in code while recreating the exact user behavior I wanted. There were several different ways something similar could've been done including manipulating more private class members via reflection or possibly firing the RoutedEvent a different way, but this is, as is most everything in programming, simply another tool in the toolset. Something different could work better -- this just worked for me at the time.
The UIAutomation framework might've been a fine alternative to some of this as well, however I like the above solution because it reads a bit more straightforward to me and for anyone who has no desire to utilize the UIAutomation framework for whatever reason, this is a good solution.
Disclaimer
I don't see anything about this that might be illegal, however everything can be a bit touchy when dealing with proprietary software, so as always, the choice of if and how to utilize this technique or not is yours.
History
Keep a running update of any changes or improvements you've made here.
