Idea on system-level debugging solution

[Johannes Schlatow]

Hi,

as I'm recently developing more sophisticated applications with Genode, I had an idea about how to augment the framework with some system-level debugging functionality that I'd like to share and discuss here:

First of all, the idea is motivated by the observation that most Genode components are of rather low complexity by themselves. Hence, debugging not only involves the code of a single component, but the operation of the component in context of its interaction with other components. I have therefore been thinking about a generic mechanism to instrument multiple Genode components in a way that allows halting the operation of a component on an incoming/outgoing signal or RPC, and printing the status of certain objects of that component.

The major concept I imagine is a hook-based approach that can be used to halt a particular component on demand. The demand would be indicated by a debug ROM that is provided for each component. Once the component is halted, it can react to changes of the debug ROM and be instructed to print debug information or continue the operation for instance. In addition, we would have a debug component as a central controller and user interface. Based on the user input, the debug controller populates the debug ROMs that are evaluated by the debug logic of the component(s) under test. With this mechanism, we could eventually set "breakpoints" at the hooks of the debugged components and interactively (or automatically) print the debug information of the components.

The debug logic can probably be implemented as a debug library that implements the hooks. A component is being instrumented by adding calls to the library functions and registering callback functions that print the debug information. Debugging is disabled by linking against a dummy library. Maybe this could also be implemented as a trace policy.

Disclaimer: I currently do not have the resources to implement such a feature, hence anyone keen on this idea is kindly invited to do so ;-)

[Adrien de GRAEVE]

Hi,

I had quiet the same kind of idea some time ago, and figured out that a
system like DTrace adapted to capabilities system could be very adapted and
a somehow standardized solution.

The only point is that it takes some efforts to implement...

Regards,

Post by Johannes Schlatow
Hi,
as I'm recently developing more sophisticated applications with Genode, I
had an idea about how to augment the framework with some system-level
First of all, the idea is motivated by the observation that most Genode
components are of rather low complexity by themselves. Hence, debugging not
only involves the code of a single component, but the operation of the
component in context of its interaction with other components. I have
therefore been thinking about a generic mechanism to instrument multiple
Genode components in a way that allows halting the operation of a component
on an incoming/outgoing signal or RPC, and printing the status of certain
objects of that component.
The major concept I imagine is a hook-based approach that can be used to
halt a particular component on demand. The demand would be indicated by a
debug ROM that is provided for each component. Once the component is
halted, it can react to changes of the debug ROM and be instructed to print
debug information or continue the operation for instance. In addition, we
would have a debug component as a central controller and user interface.
Based on the user input, the debug controller populates the debug ROMs that
are evaluated by the debug logic of the component(s) under test. With this
mechanism, we could eventually set "breakpoints" at the hooks of the
debugged components and interactively (or automatically) print the debug
information of the components.
The debug logic can probably be implemented as a debug library that
implements the hooks. A component is being instrumented by adding calls to
the library functions and registering callback functions that print the
debug information. Debugging is disabled by linking against a dummy
library. Maybe this could also be implemented as a trace policy.
Disclaimer: I currently do not have the resources to implement such a
feature, hence anyone keen on this idea is kindly invited to do so ;-)
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

[Norman Feske]

Hi Johannes,

thanks for posting your idea. I share your sentiment that the debugging
challenges increasingly move from individual programs to entire
compositions of components. Here, the classical tools like GDB start to
become inefficient.

On the other hand, I made some interesting observations while developing
the GUI stack, which consists of many components that interact with each
other. As you said, each of those components is typically quite simple.
In fact, from a high-level view, a component like the window layouter is
merely a state machine that consumes 'ROM' data and produces 'Report'
data. Since I realized the propagation of state between the components
via the report_rom server, a new opportunity for debugging appeared: By
configuring the report_rom server to be verbose, each incoming report is
being printed to the LOG. Since all the components report their internal
states as reports, the output is a nice sequence of state transitions.
Most problems turned out to be very easy to spot in this trace.

I can think of several ways to make this style of debugging even more
usable for interactive scenarios. For example, a graphical tool could
consume those reports and produce a graphical view of each report
presented as the internal state of the originating component. Since the
report_rom server also knows the consumers of each report (the ROM
session clients), we could even show the information flow between the
components graphically. This view could be dynamic, depending on clients
known by the report_rom service.

Post by Johannes Schlatow
The debug logic can probably be implemented as a debug library that
implements the hooks. A component is being instrumented by adding
calls to the library functions and registering callback functions
that print the debug information. Debugging is disabled by linking
against a dummy library. Maybe this could also be implemented as a
trace policy.

Leveraging the tracing mechanism sounds like a very cool idea.

Cheers
Norman