Andreas Tennert

A long time ago

I started using Java in the early years of my university studies. A little later, I found myself writing my first Java application with a graphical user interface. Back then I was running everything in the same thread and because I did I got the warning not to run GUI stuff (like updating text fields) in the regular thread, but in the UI thread instead. I didn't care a lot, because a) I didn't understand the concept of threading very well that time, b) I had no idea why I should do it and c) everything was running just fine without it. About two years later I was working with C#, which (as far as I can remember) enforced the use of delegators to interact with the GUI. It was a mystery and a nuisance to me. Why should I have to do this? At that point I was still writing rather small applications.

Basics

My understanding of the matter changed in my first job. I was writing infotainment applications for cars. One of the most important goals was responsiveness. For any action that the user was doing, there should be a response in a certain maximum amount of time. This time was in most cases too short to actually complete the computations that were necessary to show the final result.

The result was that the GUI needed to be decoupled from the computations, which had to run in the background. Of course, the GUI couldn't wait for the computations to finish. So in the meantime full or partial loading screens were shown to the user if the required data wasn't available. Those could be displayed instantaneously as first reaction after a user action. Then the computation was triggered. Finally, the screen was updated once the computation or parts of it were finished. Because the GUI had its own thread, the user could click any buttons while computations were running. In a single-threaded environment it would have been blocked or not reacting. Of course, new actions triggered new computations and the old ones needed to be aborted or their results ignored.

Responsiveness is a huge deal in web development as well. Managing computation time can be an issue here too, but the usecase that is always existent for every website is to load data from servers. The amount of the loading time depends on the quality of the connection to the server and the amount of data to load. Instead of loading everything at once, it's possible to load data later with requests made in JavaScript. Loading data with JavaScript is already handled asynchronously by the browsers. The remaining questions are what to load initially and what can be saved on the device in order to not load it again from the server later.

A recipe website is a classic example. The interesting content in the overview might be the recipes with the recipe name, a photo and maybe a short description for each of them. If the website generated by the server as full page it may take a long time to load the whole page. This will be especially frustrating for the user when the looked-for content is at the very bottom of a page with 100 recipes and the device is having a bad connection. One alternative would be to load the page without the recipes first so the user gets a quick feedback that something is happening. This static content can easily be saved in the browser cache with service workers as well so it is available immediately next time. Secondly, the recipe data may be loaded and displayed with place holder images, which are shown until the photos are loaded in the last step. (Use placeholders to avoid jumping around of content when more actual content is loaded.)

Summarizing this part, two different kinds of GUI decoupling were discussed:

• The GUI should run in its own thread separate from long running computations to keep it responsive.
• When loading data to display, it's necessary to think about what to load in which order. What is the most important information for the user?

In both cases, there should be a something to give to the user as quick response until the computation is finished or the data loaded. Otherwise user are left wondering what is happening. They might even start randomly clicking buttons to check whether the software still works. The result of that might be anything from confusing application responses to lost data.

Matlab ...

Decoupling the UI by using a separate UI thread is sometimes a required part of programming languages or frameworks and sometimes you need to keep it in mind yourself. Sometimes it is not possible at all (I'm looking at you, Matlab). I'm currently maintaining a legacy application written in Matlab. Matlab offers a framework to create GUIs using a built in editor called GUIDE. The programmatic interface to GUIDE-GUIs is a set of functions that are automatically generated. There is one for starting/setting up the figure (term for a window). One function is for the close button from the window frame. And there is functions for button clicks, editing of fields, ... all the handled user actions. What Matlab doesn't have is a UI thread. There is no particular threading concept at all except some basic parallel computing methods. So the setup function is called in the main thread and all the user actions as well. What does this lead to? When Matlab is busy then the UI is dead. That means that actions like button clicks are registered, but they will be executed whenever Matlab finished doing what it is doing at the moment.

In the case of our project one use of the Matlab application is to provide a user interface for handling computations on Simulink models. Those often take time. Unfortunately, it's not possible to display a loading information after a user action that is made while Matlab is busy on the main thread with another computation. This is not only an issue with GUI versus computations that the user started. We also integrate third party software that makes calls to Matlab itself. Calls from outside can only be executed on the main thread as well, so the user and the third party software compete against each other for the main thread. While users may accept to wait for actions to complete that they started themselves they will be really annoyed when the environment seems to be not responding randomly.

One solution is to use a differen GUI framework. For now our plan is to build the GUI layer in Java, because the Java integration into Matlab is working quite well except for a few edge cases. We can then work with a UI thread in Java and computations that need to be done in Matlab can be executed there. One thing to note is the limitations with modal windows. A modal window can only be modal with respect to its parent window. This can not be done across frameworks, meaning we can't have a modal window set up in Java with a parent window created in Matlab's GUIDE, or vice versa. When planing tasks that include GUI building or GUI refactoring it's important to useful this dependency in mind. Of course, modal Matlab windows block the Matlab main application thread.

Conclusion

In this article there are two discussed use cases of decoupling the GUI: keeping the GUI responsive next to long running computations and loading parts of the GUI in a way to put it together on user side nicely even when the data takes time to load. A quick response to the user, even if it is only a loading screen can improve user satisfaction a lot, because he/she isn't left wondering about what is going on.

Focusing on the GUI thread topic, Matlab is rather horrible to work with due to its strict one thread design. But there are ways to work around it by using different languages for the GUI handling. Btw., one of our most celebrated features in the last months was an improved waitbar and an icon in the status bar showing a message when computations are finished.