If you want to affect the current instance of the form, you just pass the current instance:
private void button1_Click(object sender, EventArgs e)
{
AppBackgroundColorDialog.ShowDialog();
globalvariables.DefaultBackgroundColor = AppBackgroundColorDialog.Color;
globalvariables.SetBackColor(this);
}
If what you are trying to do is alter the form that called the settings form from within the settings form, then it's more complex - and what you should do is have the settings form raise an event which the main form handles - and that changes it's own colour (or passes it's own instance to your SetBackColor method).
That sounds complicated, but it really isn't:
Transferring information between two forms, Part 2: Child to Parent[
^] explains how.