All GUI applications on Windows are driven by getting messages from the operating system, and translating them into events for the user to handle. So when the mouse moves, a window is given a message telling it that the mouse has moved, and here are its current coordinates. Or when the user presses a key, a different message is generated and a window is alerted. This message dispatching is how all events are fired in REALbasic -- the system lets the framework know something has happened, and the framework passes the information along to you in the form of an event.
If you are going to try writing a GUI component in REALbasic, you will most likely need a way to hook into this magic function that the OS calls in order to receive notifications of what the user is doing. For example, if you wanted to write a TrayIcon class you might want to be notified when your icon is double-clicked on so that you can fire an action event. Before we get into the details of how to accomplish this, you should be brought up to speed on the terminology and concepts involved.
If you're coming from the world of C/C++, .NET or even Visual Basic, the magic function being described may be familiar to you. It is traditionally called a WndProc (pronounced 'wind-proc') and it's short for window procedure. You let the system know what WndProc to call by specifying one in a window's class definition (called a WNDCLASS) structure, which is then registered with the system. Every window created with that class definition will have its WndProc called by the system for all messages. The OS knows when to call the WndProc because there's a loop in the application known as the message pump which handles all the message passing. A typical message pump will look something like this (in C):
while (!done) {
MSG msg;
if (GetMessage( &msg, NULL, 0, 0 )) {
if (msg.message == WM_QUIT) done = true;
TranslateMessage( &msg );
DispatchMessage( &msg );
}
}
In some languages, such as C or C++, the message pump is something you must write in order for your GUI application to work properly. In other languages, such as REALbasic or VB, this message pump is handled by the underlying framework so you never have to deal with it. Every application has its own message queue, and the operating system determines what messages go into which application's queue. So you will not get messages for other applications in your message queue.
What this code does is loop until it gets a WM_QUIT message. If there's a message to be found, it removes it from the operating system's message queue and stuffs the information into the MSG structure. We check to see if the message is a WM_QUIT, and if it is, then we know we need to exit the loop. Otherwise, it translates the message (which changes virtual key messages into actual character messages) and dispatches the message. It's the act of dispatching that actually sends the message to the WndProc. The WndProc's function definition will look something like this (in C):
LRESULT CALLBACK WndProc( HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam )
This may look a bit scary at first, but it's really quite easy to understand. DispatchMessage takes the MSG structure and it splits its information into multiple parts for you to use. The first parameter is the window handle of the window which should be getting the message. This corresponds to the Window.Handle property in REALbasic. The second parameter is the actual message identifier. Remember when we checked msg.message == WM_QUIT in the message pump? WM_QUIT would be the msg parameter in the WndProc. Each message is then given two 32-bit pieces of extra information to use -- the wParam and lParam parameters. The values stuffed into these parameters depends on what msg you've been given. For example, if you get a WM_COMMAND message, the high-order word of wParam specifies the notification code if the message is from a control. If the message is from an accelerator, this value is 1. If the message is from a menu, this value is zero. The low-order word of wParam specifies the identifier of the menu item, control, or accelerator. And the lParam is the handle to the control sending the message if the message is from a control. Otherwise, it is nil. The return parameter (LRESULT) is the result of whether you've handled the message or not, but its meaning depends on the message passed in. Finally, the CALLBACK part is just syntactic sugar basically meaning that this is a function that the OS calls.
A typical WndProc is just going to consist of a giant select case block where you are able to handle each of the messages you care about. There's one more standard detail which lives in the else clause for that select case block -- what to do for messages you don't care about. Windows provides you with a default WndProc to call thru to for all the messages you don't care about. It will look something like this (in C):
return DefWindowProc( hWnd, msg, wParam, lParam );
This passes the message information along to the catch-all window procedure so that the OS can handle a lot of the dirty work for you. For example, if the application doesn't need to handle the WM_SIZING message because it doesn't need to limit the size of the window, then that message can be handled by the default window procedure and the window will continue to be resized.
So now that you have a more firm understanding of how message processing happens under the hood, let's start going into how you would hook yourself into this process. Hooking yourself into the WndProc calling chain is accomplished by a process commonly known as window subclassing. The basic thought behind it is easy. You pick a window that you want to hook into and obtain a handle to it (also known as its HWND). In REALbasic, this is very easy to do since every Window instance has a Handle property to it. This window knows what WndProc it dispatches messages to by default since that information was provided by the window class when it was being created. You use a declare to the SetWindowLong Win32 API to stuff your own function's address into the window so that you can handle the messages instead. Voila! You're hooked in. But it's not quite that simple... The problem is, you've just overwritten the default WndProc for that window by stuffing in your own window procedure, so the old window procedure never be called again. Chances are, the window is not going to work the way it used to since the application was written under the assumption that its window procedure would be called. In REALbasic, this means that none of the events are going to be called for that Window instance, so your application is not going to behave the same. But don't worry, there are just a few small steps to take care of these issues.
Back up a step. Before you store the WndProc you want to call via the SetWindowLong call, you should grab the old WndProc and keep its address around. You do this with a declare to the GetWindowLong Win32 API. So now you know what function to call once you're done processing your messages. The only trick now is, how do you call the function you just retrieved? Simple: Windows exposes another API that lets you pass the buck (so to speak), aptly named CallWindowProc. So when you're done processing your messages, you use CallWindowProc with the value you saved from the window, and now you've properly subclassed the window. The only thing left to remember is that when you no longer need to hook into the window, you should remove yourself from the calling chain. This is simply a matter of calling SetWindowLong one last time and setting the old WndProc back on the window.
In theory, this is how things happen -- let's turn that theory into something more concrete and do it ourselves.
What we need to do is have a helper function that properly subclasses the window, another function that is the new WndProc, and lastly, a function to unsubclass the window when we're done with it. So first, let's make a new module that we can put our helper functions on. This also gives us a place to put our WndProc since it's a technically a callback function (which cannot live on the class scope in REALbasic). We'll also make a class interface for callers to implement so that we can pass the messages along in a sensible fashion. The class interface and module should look something like this:
Interface WndProcSubclass
Function WndProc( hWnd as Integer, msg as Integer, wParam as Integer, lParam as Integer, _
ByRef handled as Boolean ) as Integer
End Interface
(In case you're not familiar with how interfaces work, or what they're meant for, I would suggest reading up on them at http://www.quantum-meruit.com/RB/Articles/ClassInterfaces/.)
Module WndProcHelpers
Protected Sub Subclass( wnd as Window, proc as WndProcSubclass )
Protected Sub Unsubclass( wnd as Window )
Private Function WndProc( hWnd as Integer, msg as Integer, wParam as Integer, _
lParam as Integer ) as Integer
Private mOldWndProc as Dictionary
Private mSubClass as Dictionary
End Module
To reiterate what's going to happen: WndProcHelpers.Subclass is responsible for getting and storing the old window procedure from the window passed in. Then it will set up the new window procedure and point it at WndProcHelpers.WndProc. It needs to do this because we can't have callbacks into classes. All the window's messages will be routed into WndProcHelpers.WndProc, which will then call the interface WndProc when appropriate. It's also responsible for passing along messages to the window's old window procedure as well. Finally, WndProcHelpers.Unsubclass will restore the old window procedure for the window given.
Let's start by showing how the Subclass function works.
WndProcHelpers.Subclass( wnd as Window, proc as WndProcSubclass )
// Do a sanity check
if wnd = nil or proc = nil then return
// Now make sure we have the wnd procs and subclasses
if mOldWndProc = nil then mOldWndProc = new Dictionary
if mSubClass = nil then mSubClass = new Dictionary
// Check to see if we've already subclassed this window.
if mOldWndProc.HasKey( wnd.Handle ) then return
// Now we want to set the new window procedure. This call will return
// the old window procedure, so we don't need to call GetWindowLong
Declare Function SetWindowLongA Lib "User32" ( hwnd as Integer, index as Integer, _
newValue as Ptr ) as Integer
Const GWL_WNDPROC = -4
dim oldWndProc as Integer = SetWindowLongA( wnd.Handle, GWL_WNDPROC, AddressOf WndProc )
// Now save the old window procedure into our list. We'll
// reference it by the window's handle
mOldWndProc.Value( wnd.Handle ) = oldWndProc
// We'll also save off the interface to call, again, referencing
// it by the window handle
mSubClass.Value( wnd.Handle ) = proc
End Sub
You'll notice that we don't need to call GetWindowLong to get the old value of the window procedure. This is because SetWindowLong is designed to return the old value that's being replaced. You'll also notice that we only allow a window to be subclassed one time. This is an artificial limitation for code clarity -- you can subclass a window as many times as you'd like if you account for it.
The Unsubclass method is going to look very similar, except that it's going to be restoring the old window procedure and removing things from our dictionaries. It will look like this:
WndProcHelpers.Unsubclass( wnd as Window )
// Sanity check
if wnd = nil then return
// Check to see if we've subclassed this window
if not mOldWndProc.HasKey( wnd.Handle ) then return
// Get the old window procedure handle
dim oldWndProc as Integer = mOldWndProc.Value( wnd.Handle )
// We want to restore the old window procedure to the window. We
// don't care about the return value either
Declare Sub SetWindowLongA Lib "User32" ( hwnd as Integer, index as Integer, newValue as Integer )
Const GWL_WNDPROC = -4
SetWindowLongA( wnd.Handle, GWL_WNDPROC, oldWndProc )
// Now remove this window from our subclass list
mOldWndProc.Remove( wnd.Handle )
mSubClass.Remove( wnd.Handle )
End Sub
So now that you've seen how to subclass and unsubclass the window, you're ready to see how to handle the actual window procedure code itself. You're in for quite a surprise with it -- it's not nearly as difficult as you might suspect. Let's see how it's done.
WndProcHelpers.WndProc( hwnd as Integer, msg as Integer, wParam as Integer, lParam as Integer ) as Integer
Declare Function DefWindowProcA Lib "User32" ( hwnd as Integer, msg as Integer, _
wParam as Integer, lParam as Integer ) as Integer
// Do a sanity check to see if we've subclassed this window or not.
if not mSubclass.HasKey( hwnd ) then
// Something's not right here, so just call the default window procedure
return DefWindowProcA( hwnd, msg, wParam, lParam )
end if
// We know that we've got a subclass, so call it's window procedure.
dim handled as Boolean
dim subclass as WndProcSubclass = mSubclass.Value( hwnd )
dim ret as Integer = subclass.WndProc( hwnd, msg, wParam, lParam, handled )
// If the user handled the function, then we want to return
// with the value they passed us
if handled then return ret
// Otherwise we want to pass the function along to the old window
// procedure so it can handle it
Declare Function CallWindowProcA Lib "User32" ( proc as Integer, hwnd as Integer, _
msg as Integer, wParam as Integer, lParam as Integer ) as Integer
return CallWindowProcA( mOldWndProc.Value( hwnd ), hwnd, msg, wParam, lParam )
End Function
The function doesn't have to do that much. It simply relegates the decisions off to other callers. If we can't find a subclassed window for the window handle passed into us, we simply call the default window procedure and return its return value. Otherwise, we let the subclass interface have a crack at handling the message. If it handles it, then we're done and return its value. Finally, if the message hasn't been handled, we call the next window procedure in the calling chain, which is the old window procedure for the window and return its return value.
So as you can see, the real meat of window subclassing is mostly just housekeeping issues. Making sure that the right window procedures get setup, called and restored is all window subclassing boils down to. The real guts of what you're trying to solve happen within the interface's WndProc method. That's the part that handles all the actual messages and decides which ones to handle or not. Let's take a look at a hypothetical window procedure method for dealing with a tray icon's messages.
Tray icon messages come in the form of a user-defined message property that is set up when the icon is added. So let's go off the assumption that the user has defined WM_SYS_TRAY_MESSAGE as being WM_USER (whose value is &h400) plus some constant (whose value is arbitrary, but let's say &h150 for this example). This message will be passed to the window by the system whenever anything interesting happens to the tray icon. The true message (the one that causes the system to send a WM_SYS_TRAY_MESSAGE) is stuffed into the lParam. So the interface's window procedure code may look something like this:
Function MyTrayIcon.WndProc( hwnd as Integer, msg as Integer, wParam as Integer, _
lParam as Integer, ByRef handled as Boolean ) as Integer
// Check to see whether this is our message or not
Const WM_SYS_TRAY_MESSAGE = &h400 + &h150
if msg <> WM_SYS_TRAY_MESSAGE then
handled = false
return 0
end if
// Now we know this is our message, so we're handling it
handled = true
Const WM_LBUTTONDOWN = &h201
Const WM_RBUTTONDOWN = &h204
Const WM_MOUSEMOVE = &h200
// Check to see what the true message is and call one of our user
// defined events as appropriate
select case lParam
case WM_LBUTTONDOWN
MouseDown( kLeftButton )
case WM_RBUTTONDOWN
MouseDown( kRightButton )
case WM_MOUSEMOVE
MouseMove
end select
return 1
End Function
Processing messages is simply a matter of figuring out whether you care about what the operating system is telling you or not. In the above example, we only care about WM_SYS_TRAY_MESSAGES, so we check to see what the msg parameter is. If it's not the message we're interested in, we set handled to false and bail out. If we are processing the message, then we set handled to true and process the message however we see fit and return whatever value makes sense.
As you can see, subclassing a window isn't a terribly difficult process. It just takes a firm understanding of how Windows handles notifying applications of events, and a little bit of housekeeping. You should be able to take what you've learned today and use it to provide custom classes in REALbasic for just about any GUI control in Windows.
That's all there is to it! If you have any questions or comments, feel free to send them to aaron@realsoftware.com I hope you found this article helpful! To download the example that goes along with this, click here.