Query regarding extracting instruction which caused a data-abort exception

Discussion:

Abhishek Kumar

2017-06-13 09:17:02 UTC

Hello
I am trying to modify genode trustzone. I want to read the instruction
which lead to data abort exception in normal world, in the `dump` function
in tz_vmm. I have value of all the registers through `_state` register. We
tried with `_state->ip`. On converting 16 bits stored at the address
pointed by _state->ip, we got ARM Thumb instruction:

STRH R0, [R0, #6]
But the value (R0) + 6, doesn't match dfar. We're not sure if _state->ip is
the register to go with. We tried with _state->mode[2].lr which is lr_abt
register. But the address stored in lr_abt, lr_abt-16, lr_abt-32 all have
0s.

Which is right register to get the address of the instruction which caused
the data-abort exception?

Thanks
Abhishek

Stefan Kalkowski

2017-06-14 12:24:41 UTC

Permalink

Hello,

Post by Abhishek Kumar
Hello
I am trying to modify genode trustzone. I want to read the instruction
which lead to data abort exception in normal world, in the `dump`
function in tz_vmm. I have value of all the registers through `_state`
register. We tried with `_state->ip`. On converting 16 bits stored at
STRH R0, [R0, #6]
But the value (R0) + 6, doesn't match dfar. We're not sure if _state->ip
is the register to go with. We tried with _state->mode[2].lr which is
lr_abt register. But the address stored in lr_abt, lr_abt-16, lr_abt-32
all have 0s.
Which is right register to get the address of the instruction which
caused the data-abort exception?

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.

On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.

Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.

Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
the past, and are mostly answered in our TrustZone report:

https://genode.org/documentation/articles/trustzone

and in the discussions of our mailing list:

https://sourceforge.net/p/genode/mailman/search/?q=trustzone

Regards
Stefan

Post by Abhishek Kumar
Thanks
Abhishek
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

--
Stefan Kalkowski
Genode Labs

https://github.com/skalk · http://genode.org/

r***@mpi-sws.org

2017-06-14 12:40:49 UTC

Permalink

Thanks Stefan!

We are actually trying to do device emulation in the secure world genode,
for peripherals marked as secure (UART or ESDHC) in csu_config.h. We got
encouraged by the following discussion at
http://genode.org/documentation/articles/trustzone

==========================================================================
The basic idea of emulating device access is to let the hypervisor pass
control to the VMM as soon as the non-secure OS accesses an address
outside the permitted physical address ranges. The VMM can then inspect
the address in question and the program counter of the non-secure OS that
raised the access violation. Given the program counter value, the VMM can
fetch and decode the faulting instruction and emulate it in software.
Because ARM is a RISC architecture, the instruction decoding is rather
simple. The instruction in question can only be a load or a store
instruction. No other instruction would raise an access fault. For read
operations, the VMM would provide the result of the operation by changing
the corresponding entry of the VM state structure.

That said, we found that the trap-and-execute emulation model is not
possible to implement with the TrustZone protection mechanisms in general.
Dependent on the concrete platform, the CPU will not immediately enter the
hypervisor when the fault occurs but attempts to perform the bus
transaction. This transaction will trigger an external data abort. This
abort is similar to a device interrupt. It principally raises an exception
(so the violation can be detected) but not always immediately. Therefore,
there is no way to uniquely reconstruct what happened in between the
invalid access and the reception of the external abort exception in the
hypervisor. Neither can the hypervisor recover the non-secure world to a
useful state.

A noteworthy advantage of the CSU compared to the ARM TZ protection
controller within the ARM Cortex A9 reference board is the way of how
access violations are handled. As stated in Device emulation, the ARM TZ
protection controller responds to invalid accesses with an asynchronous
external abort exception, similar to a device interrupt. Upon the
execution of an offending instruction, the TZ protection controller
detects the violation, yet the CPU would continue the execution of further
instructions until the flagged violation eventually reaches the CPU,
triggering an external abort exception. This scheme effectively rules out
any attempt to emulate device accesses. In contrast, the i.MX CSU responds
to access violations by synchronously yielding control to the exception
handler. So when such an exception occurs, the offending instruction can
be determined and emulated in software. However, even though device
emulation using the CSU is principally possible, we haven't investigated
this opportunity further.
===========================================================================

We want to read the instruction faulting in NW linux in tz_vmm,
disassemble it, emulate it in genode code and restart the VM at the next
instruction of the normal world. Do you think this is feasible, or your
comments about "synchronous data abort in IMX53 vs. asynchronous aborts in
Versatile Express" don't hold always?

Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

--
Stefan Kalkowski
Genode Labs
https://github.com/skalk · http://genode.org/
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

r***@mpi-sws.org

2017-06-14 12:48:18 UTC

Permalink

Interestingly, the DFAR contains an UART address when UART is made secure
in csu_config.h and an eSDHC address when eSDHC is made secure in
csu_config.h. But this might still be a case of asynchronous abort, only
that DFAR is still pointing to the correct address where the instruction
faulted (not more data aborts have occurred since then), but the other
registers have changed because of the asynchronous nature of the abort.

How can we know if the abort is asynchronous vs. synchronous? Is an lr_abt
with all 0's already a clear indication that this is asynchronous, and
there is no hope for device emulation?

Thanks!
Riju

Post by r***@mpi-sws.org
Thanks Stefan!
We are actually trying to do device emulation in the secure world genode,
for peripherals marked as secure (UART or ESDHC) in csu_config.h. We got
encouraged by the following discussion at
http://genode.org/documentation/articles/trustzone
==========================================================================
The basic idea of emulating device access is to let the hypervisor pass
control to the VMM as soon as the non-secure OS accesses an address
outside the permitted physical address ranges. The VMM can then inspect
the address in question and the program counter of the non-secure OS that
raised the access violation. Given the program counter value, the VMM can
fetch and decode the faulting instruction and emulate it in software.
Because ARM is a RISC architecture, the instruction decoding is rather
simple. The instruction in question can only be a load or a store
instruction. No other instruction would raise an access fault. For read
operations, the VMM would provide the result of the operation by changing
the corresponding entry of the VM state structure.
That said, we found that the trap-and-execute emulation model is not
possible to implement with the TrustZone protection mechanisms in general.
Dependent on the concrete platform, the CPU will not immediately enter the
hypervisor when the fault occurs but attempts to perform the bus
transaction. This transaction will trigger an external data abort. This
abort is similar to a device interrupt. It principally raises an exception
(so the violation can be detected) but not always immediately. Therefore,
there is no way to uniquely reconstruct what happened in between the
invalid access and the reception of the external abort exception in the
hypervisor. Neither can the hypervisor recover the non-secure world to a
useful state.
A noteworthy advantage of the CSU compared to the ARM TZ protection
controller within the ARM Cortex A9 reference board is the way of how
access violations are handled. As stated in Device emulation, the ARM TZ
protection controller responds to invalid accesses with an asynchronous
external abort exception, similar to a device interrupt. Upon the
execution of an offending instruction, the TZ protection controller
detects the violation, yet the CPU would continue the execution of further
instructions until the flagged violation eventually reaches the CPU,
triggering an external abort exception. This scheme effectively rules out
any attempt to emulate device accesses. In contrast, the i.MX CSU responds
to access violations by synchronously yielding control to the exception
handler. So when such an exception occurs, the offending instruction can
be determined and emulated in software. However, even though device
emulation using the CSU is principally possible, we haven't investigated
this opportunity further.
===========================================================================
We want to read the instruction faulting in NW linux in tz_vmm,
disassemble it, emulate it in genode code and restart the VM at the next
instruction of the normal world. Do you think this is feasible, or your
comments about "synchronous data abort in IMX53 vs. asynchronous aborts in
Versatile Express" don't hold always?
Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

Stefan Kalkowski

2017-06-14 13:12:17 UTC

Permalink

Hi,

Post by r***@mpi-sws.org
Interestingly, the DFAR contains an UART address when UART is made secure
in csu_config.h and an eSDHC address when eSDHC is made secure in
csu_config.h. But this might still be a case of asynchronous abort, only
that DFAR is still pointing to the correct address where the instruction
faulted (not more data aborts have occurred since then), but the other
registers have changed because of the asynchronous nature of the abort.
How can we know if the abort is asynchronous vs. synchronous? Is an lr_abt
with all 0's already a clear indication that this is asynchronous, and
there is no hope for device emulation?

I would not generalize or depend on anything related to the "exception"
case, because TrustZone is not meant as a virtualization solution.
Normally, you should just turn off the device or reboot in emergency case.

Having said this: Obviously, in case of the i.MX53 the secure world's
DFAR is updated to the faulting address if the "normal" world provokes a
security violation "on the bus". Therefore, I could imagine the DFSR
register is updated too. Normally, this register contains information
about the type of data-fault. Have a look at:

repos/base-hw/src/core/spec/arm_v7/trustzone/kernel/vm.cc

where the DFAR is read and set to the VM state structure, and you could
print it, or write it to an arbitrary register for debugging purposes.

Regards
Stefan

Post by r***@mpi-sws.org
Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

--
Stefan Kalkowski
Genode Labs

https://github.com/skalk · http://genode.org/

Stefan Kalkowski

2017-06-14 12:55:05 UTC

Permalink

Hi,

Post by r***@mpi-sws.org
We want to read the instruction faulting in NW linux in tz_vmm,
disassemble it, emulate it in genode code and restart the VM at the next
instruction of the normal world. Do you think this is feasible, or your
comments about "synchronous data abort in IMX53 vs. asynchronous aborts in
Versatile Express" don't hold always?

As I said in my previous mail: I only observed synchronous data-abort on
i.MX53. So I think this is not the show-stopper.

Anyway please read my whole mail especially the section regarding the
caching issues. Being in your position, I would first correlate the
instruction pointer values with your Linux binary, e.g. using objdump,
before you start to do instruction decoding on cache-incoherent memory.

Regards
Stefan

Post by r***@mpi-sws.org
Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

--
Stefan Kalkowski
Genode Labs

https://github.com/skalk · http://genode.org/

r***@mpi-sws.org

2017-06-14 12:58:01 UTC

Permalink

Thanks a lot Stefan, Will correlate with the linux binary.

Riju

Post by Stefan Kalkowski
Hi,

As I said in my previous mail: I only observed synchronous data-abort on
i.MX53. So I think this is not the show-stopper.
Anyway please read my whole mail especially the section regarding the
caching issues. Being in your position, I would first correlate the
instruction pointer values with your Linux binary, e.g. using objdump,
before you start to do instruction decoding on cache-incoherent memory.
Regards
Stefan

Post by r***@mpi-sws.org
Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

Abhishek Kumar

2017-06-14 13:15:01 UTC

Permalink

Hello Stefan
Thanks for your reply. I have a quick clarification to ask for. As you
mentioned about problems with cache-incoherent memory. What we are trying
to read is in code segment, I don't see how it can be changed. Since it is
not changing cache coherency should not be an issue, this is my
understanding. Can you please help me see where I might be wrong?

Thanks
Abhishek

Post by r***@mpi-sws.org
Thanks a lot Stefan, Will correlate with the linux binary.
Riju

Post by Stefan Kalkowski
Hi,

As I said in my previous mail: I only observed synchronous data-abort on
i.MX53. So I think this is not the show-stopper.
Anyway please read my whole mail especially the section regarding the
caching issues. Being in your position, I would first correlate the
instruction pointer values with your Linux binary, e.g. using objdump,
before you start to do instruction decoding on cache-incoherent memory.
Regards
Stefan

Post by r***@mpi-sws.org
Thanks!
Riju

Post by Stefan Kalkowski
Hello,

As long as you get an synchronous data-abort from the normal world,
reading the current instruction pointer of the 'state' structure is
perfectly fine. The mode-specific lr register is useful for the handling
of MMU faults within the "normal" world itself. They are not modified,
as long as the "normal" world MMU can resolve an access, but some bus
resp. CSU is answering that the access is not allowed. This will not
change the "normal" world register set.
On the other hand, in general a bus fault triggered by unallowed access
of the "normal" world does not necessarily mean a synchronous
data-abort, although on i.MX53 I only observed those. In general, it can
also provoke an asynchronous external data-abort, which means that the
instruction pointer is not necessarily pointing to the instruction that
triggered the fault.
Moreover, looking at the "normal" world's memory from the secure side is
troublesome. Because the normal and secure world's memory view is not
cache-coherent. Cache entries are always tagged by the NS bit. That
means you have to take care to flush caches yourself. If you want to
debug instructions, you should instead look at the Linux binary itself
and not into the memory on the secure side. To me it looks strange that
you identify a Thumb instruction in the kernel here.
Btw. these kind of TrustZone/i.MX53 questions were asked repeatedly in
https://genode.org/documentation/articles/trustzone
https://sourceforge.net/p/genode/mailman/search/?q=trustzone
Regards
Stefan

Post by Abhishek Kumar
Thanks
Abhishek
------------------------------------------------------------

------------------