Writing to UART mapped memory address

[Abhishek Kumar]

Hello
I am running trustzone Genode, with linux in normal world. I have made UART
secure and I'm getting the fault address in DFAR register in dump in VM
monitor. I want to write to a memory address which is mapped to UART i.e.
physical address is in range [53FBC000, 53FFFFFF]. I am getting following
error when I try to access the address:

```
no RM attachment (faulter 203128 with IP 7001e7d0 attempts to read from
address 53fbc080)
init -> tz_vmm -> vmm: unresolved pagefault at ip=7001e7d0 sp=e01fed10
fault address=53fbc080
core -> pager_ep: cannot submit unknown signal context
```


I'm using va_to_pa to get the physical address. I'm not sure how Genode
maps the virtual addresses to I/O mapped physical region, or how can I
write to UART mapped physical address?



Thanks
Abhishek

[r***@mpi-sws.org]

In os/src/server/tz_vmm/spec/imx53/main.cc, added the following enums
(1) UART_BASE = 0x53FBC000,
(2) UART_SIZE = 0x43FFF

In os/src/server/tz_vmm/include/vm_base.h
In Vm_base constructor, added
(3)_ram_iomem_uart(uart_base, uart_size),
(4)_ram_uart(uart_base, uart_size,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem_uart.dataspace())),

In dump(), added
(5)printf("The memory location is %08lx\n", va_to_pa(_state->dfar));
(6) printf("_ram.local() is %08lx\n",_ram.local());
(7) printf("_ram.local() contains %08x\n",*(unsigned int*)(_ram.local()));
(8) printf("_ram_uart.local() is %08lx\n",_ram_uart.local());
(9) printf("_ram_uart.local() contains %08x\n",*(unsigned
int*)(_ram_uart.local()));

The output:
[init -> tz_vmm] The memory location is 53fbc080
[init -> tz_vmm] _ram.local() is 80000000
[init -> tz_vmm] _ram.local() contains eded1bb6
[init -> tz_vmm] _ram_uart.local() is 70078000
init -> tz_vmm -> vmm: raised unhandled data abort DFSR=0x00001008
ISFR=0x00000007 DFAR=0x70078000 ip=0x7001f440 sp=0xe01fed10

Thus though _ram and _ram_uart are assigned in the same way (over
different physical addresses), _ram.local() can be de-referenced and the
content read, while _ram_uart.local() gives a fault. How are these UART
addresses starting at 0x53FBC000 different from the
Trustzone::NONSECURE_RAM_BASE physical addresses?

Funnily, the DFSR=0x00001008, which is the same value as the abort in the
normal world linux. That can occur due to CSU configuration of UART made
secure. But the vmm is in secure world, why is that getting the same abort
at that same UART physical address?

Thanks!
Riju

[r***@mpi-sws.org]

In base-hw/src/core/include/spec/imx53/trustzone/csu.h, SECURE = 0x33.

So SECURE_UART = 1 in
base-hw/src/core/include/spec/imx53_qsb/trustzone/csu_config.h should
initialize UART as RD+WR for secure user and spvr models as given in Page
956 of http://www.nxp.com/docs/en/reference-manual/iMX53RM.pdf.

tz_vmm is in the secure user mode, sho it should be able to read an UART
address?

Thanks!
Riju

[r***@mpi-sws.org]

In base-hw/src/core/spec/arm_v7/trustzone/mode_transition.s,
_nonsecure_kernel_entry:, where we go in the monitor handler after a DABT,
we have
mov r1, #0
mcr p15, 0, r1, c1, c1, 0 //sets SCR all 0s
cps #SVC_MODE //changes processor mode to SVC

In base-hw/src/core/spec/arm_v7/trustzone/kernel/vm.cc,
in void Vm::exception(unsigned const cpu),
case Genode::Cpu_state::DATA_ABORT:,
Cpu::Scr::read() gives 00000000 and Cpu::Psr::read() gives 600001d3. So
the NS bit in Scr is 0 and the mode bits in CPSR are 10011=19, which is
the SVC mode. So we are getting the correct values that the monitor DABT
handler sets.

We are in the secure SVC mode, when we try to access the uart physical
address. So what is causing a data abort now?

Thanks!
Riju

[Stefan Kalkowski]

Post by r***@mpi-sws.org
In base-hw/src/core/spec/arm_v7/trustzone/mode_transition.s,
_nonsecure_kernel_entry:, where we go in the monitor handler after a DABT,
we have
mov r1, #0
mcr p15, 0, r1, c1, c1, 0 //sets SCR all 0s
cps #SVC_MODE //changes processor mode to SVC
In base-hw/src/core/spec/arm_v7/trustzone/kernel/vm.cc,
in void Vm::exception(unsigned const cpu),
case Genode::Cpu_state::DATA_ABORT:,
Cpu::Scr::read() gives 00000000 and Cpu::Psr::read() gives 600001d3. So
the NS bit in Scr is 0 and the mode bits in CPSR are 10011=19, which is
the SVC mode. So we are getting the correct values that the monitor DABT
handler sets.
We are in the secure SVC mode, when we try to access the uart physical
address. So what is causing a data abort now?

No, you are in secure SVC mode within the kernel when entering this
exception routine. Your access within the tz_vmm component is in secure
USR mode.

Anyway, I doubt that it has something to do with kernel/user
secure/non-secure issues, because otherwise all other userland drivers
should fault too. Another possibility is that the page-table entries are
not refering to the right address due to some wrong semantics in your
added code.
Or it is mapped with different caching attributes in kernel and
user-land (please: do not use the device in kernel and userland), but I
can't imagine how, if it's really an IOMEM dataspace in your code.
Well, to be honest, I do not know, why it should fail, but again it
would help if you provide a working branch.

Regards Stefan

Post by r***@mpi-sws.org
Thanks!
Riju
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

[r***@mpi-sws.org]

Thanks Stefan. You are already helping a lot, so I don't want to actually
debug code.

We could do instruction emulation for a large class of peripherals mapped
to CSU (for example USB), but not all.

For the ones that we couln't, the issue is we cannot access the
corresponding peripheral addresses in tz_vmm. We get a data abort at the
following printf in the Vm_base constructor, which is called from

static Vm vm("linux" .... in main() at
repos/os/src/server/tz_vmm/spec/imx53/main.cc.

So this is before copying linux kernel and everything else in
repos/os/src/server/tz_vmm/include/vm_base.h.

(1) _ram_iomem_peripheral(peripheral_base, peripheral_size)
(2) _ram_peripheral(peripheral_base, peripheral_size,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem_peripheral.dataspace()))
(3) Genode::printf("_ram_peripheral.local() contains %08x\n",*(unsigned
int*)(_ram_peripheral.local()))

The following are the peripherals, with peripheral_base and
peripheral_size, that have this behavior, along with their CSL mappings.
CSL are all set to UNSECURE i.e. 0xff, so everything should be accessible.
The other 50+ peripherals work fine and print the values held in those
peripheral memory addresses in the printf mentioned above.
===========================================================================
(1) write<Csl26::Slave_a>(Csl00::UNSECURE); UART3, 5000C000,3FFF
(2) write<Csl27::Slave_a>(Csl00::UNSECURE); ESAI, 50018000, 3FFF
(3) write<Csl02::Slave_b>(Csl00::UNSECURE); KPP, 53F94000, 3FFF
(4) write<Csl03::Slave_a>(Csl00::UNSECURE); WDOG1, 53F98000,3FFF
(5) write<Csl03::Slave_b>(Csl00::UNSECURE); WDOG2, 53F9C000,3FFF
(6) write<Csl07::Slave_b>(Csl00::UNSECURE); UART1, 53FBC000, 3FFF
(7) write<Csl08::Slave_a>(Csl00::UNSECURE); UART2, 53FC0000, 3FFF
(8) write<Csl05::Slave_b>(Csl00::UNSECURE); CAN1, 53FC8000, 3FFF
(9) write<Csl06::Slave_a>(Csl00::UNSECURE); CAN2, 53FCC000, 3FFF
(10) write<Csl09::Slave_b>(Csl00::UNSECURE); CAN2, 53FCC000, 3FFF
(11) write<Csl30::Slave_b>(Csl00::UNSECURE); UART4, 53FF0000, 3FFF
(12) write<Csl19::Slave_a>(Csl00::UNSECURE); UART5, 63F90000, 3FFF
(13) write<Csl16::Slave_a>(Csl00::UNSECURE); ROMCP, 63FB8000, 3FFF
(14) write<Csl17::Slave_b>(Csl00::UNSECURE); I2C2, 63FC4000, 3FFF
(15) write<Csl18::Slave_a>(Csl00::UNSECURE); I2C1, 63FC8000, 3FFF
(16) write<Csl23::Slave_b>(Csl00::UNSECURE); SAHARA, 63FF8000, 3FFF
(17) write<Csl11::Slave_b>(Csl00::UNSECURE); AHBMAX/PL301_2x2/PL301_4x1,
63F94000, 3FFF, 63FDC000, 3FFF, 63FE0000, 3FFF (only AHBMAX)
(18) write<Csl09::Slave_a>(Csl00::UNSECURE); CCM/SRC/GPC/DPLLIP1-4/OWIRE,
53FD4000, 3FFF, 53FD0000, 3FFF, 53FD8000, 3FFF, 63F80000, 3FFF, 63F84000,
3FFF, 63F88000, 3FFF, 63F8C000, 3FFF, 63FA4000, 3FFF (only OWIRE gives
secure world abort)
===========================================================================

The other 50+ peripherals also work fine if we make CSL for that
peripheral secure (0x33), access that memory in linux, trap data abort to
monitor, come to tz_vmm, get the instruction from ip, decode it and do the
LDR/STR from the DFAR into the necessary _state->register and set
ip->ip+4.

Have these peripherals like UART, I2C, WDOG, SAHARA, CAN etc. been
specially set up in any way, that tz_vmm (secure world user mode) cannot
access them? Do these peripherals have anything in common?

Thanks!
Riju

[Stefan Kalkowski]

Post by r***@mpi-sws.org
Thanks Stefan. You are already helping a lot, so I don't want to actually
debug code.
We could do instruction emulation for a large class of peripherals mapped
to CSU (for example USB), but not all.
For the ones that we couln't, the issue is we cannot access the
corresponding peripheral addresses in tz_vmm. We get a data abort at the
following printf in the Vm_base constructor, which is called from
static Vm vm("linux" .... in main() at
repos/os/src/server/tz_vmm/spec/imx53/main.cc.
So this is before copying linux kernel and everything else in
repos/os/src/server/tz_vmm/include/vm_base.h.
(1) _ram_iomem_peripheral(peripheral_base, peripheral_size)
(2) _ram_peripheral(peripheral_base, peripheral_size,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem_peripheral.dataspace()))
(3) Genode::printf("_ram_peripheral.local() contains %08x\n",*(unsigned
int*)(_ram_peripheral.local()))
The following are the peripherals, with peripheral_base and
peripheral_size, that have this behavior, along with their CSL mappings.
CSL are all set to UNSECURE i.e. 0xff, so everything should be accessible.
The other 50+ peripherals work fine and print the values held in those
peripheral memory addresses in the printf mentioned above.
===========================================================================
(1) write<Csl26::Slave_a>(Csl00::UNSECURE); UART3, 5000C000,3FFF
(2) write<Csl27::Slave_a>(Csl00::UNSECURE); ESAI, 50018000, 3FFF
(3) write<Csl02::Slave_b>(Csl00::UNSECURE); KPP, 53F94000, 3FFF
(4) write<Csl03::Slave_a>(Csl00::UNSECURE); WDOG1, 53F98000,3FFF
(5) write<Csl03::Slave_b>(Csl00::UNSECURE); WDOG2, 53F9C000,3FFF
(6) write<Csl07::Slave_b>(Csl00::UNSECURE); UART1, 53FBC000, 3FFF
(7) write<Csl08::Slave_a>(Csl00::UNSECURE); UART2, 53FC0000, 3FFF
(8) write<Csl05::Slave_b>(Csl00::UNSECURE); CAN1, 53FC8000, 3FFF
(9) write<Csl06::Slave_a>(Csl00::UNSECURE); CAN2, 53FCC000, 3FFF
(10) write<Csl09::Slave_b>(Csl00::UNSECURE); CAN2, 53FCC000, 3FFF
(11) write<Csl30::Slave_b>(Csl00::UNSECURE); UART4, 53FF0000, 3FFF
(12) write<Csl19::Slave_a>(Csl00::UNSECURE); UART5, 63F90000, 3FFF
(13) write<Csl16::Slave_a>(Csl00::UNSECURE); ROMCP, 63FB8000, 3FFF
(14) write<Csl17::Slave_b>(Csl00::UNSECURE); I2C2, 63FC4000, 3FFF
(15) write<Csl18::Slave_a>(Csl00::UNSECURE); I2C1, 63FC8000, 3FFF
(16) write<Csl23::Slave_b>(Csl00::UNSECURE); SAHARA, 63FF8000, 3FFF
(17) write<Csl11::Slave_b>(Csl00::UNSECURE); AHBMAX/PL301_2x2/PL301_4x1,
63F94000, 3FFF, 63FDC000, 3FFF, 63FE0000, 3FFF (only AHBMAX)
(18) write<Csl09::Slave_a>(Csl00::UNSECURE); CCM/SRC/GPC/DPLLIP1-4/OWIRE,
53FD4000, 3FFF, 53FD0000, 3FFF, 53FD8000, 3FFF, 63F80000, 3FFF, 63F84000,
3FFF, 63F88000, 3FFF, 63F8C000, 3FFF, 63FA4000, 3FFF (only OWIRE gives
secure world abort)
===========================================================================
The other 50+ peripherals also work fine if we make CSL for that
peripheral secure (0x33), access that memory in linux, trap data abort to
monitor, come to tz_vmm, get the instruction from ip, decode it and do the
LDR/STR from the DFAR into the necessary _state->register and set
ip->ip+4.
Have these peripherals like UART, I2C, WDOG, SAHARA, CAN etc. been
specially set up in any way, that tz_vmm (secure world user mode) cannot
access them? Do these peripherals have anything in common?

They are not setup specially in Genode, and I do not see what they have
in common.

Regards
Stefan

Post by r***@mpi-sws.org
Thanks!
Riju
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

[r***@mpi-sws.org]

Hi,
To get virtual address corresponding to any physical address in genode, we
generally use:

Genode::Io_mem_connection _ram_iomem(phy_addr, 4);
Ram _ram(phy_addr, 4,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem.dataspace()));

_ram.local() then gives the necessary virtual address. This works
vm_base.h for run/tz_vmm demo or vm.h for run/vmm, i.e. in the tz vmm user
space.

What can we do to get virtual addresses for physical addresses which are
already mapped? E.g. in the imx53 sabre tablet demo, the 7 GPIO banks are
already mapped to virtual addresses in os/src/drivers/gpio/imx53/driver.h.

If we have a physical address (from DFAR) corresponding to the GPIO
address ranges, how can I get the virtual address in user space files
corresponding to tz vmm?

Thanks!
Riju

[Stefan Kalkowski]

Post by r***@mpi-sws.org
Hi,
To get virtual address corresponding to any physical address in genode, we
Genode::Io_mem_connection _ram_iomem(phy_addr, 4);
Ram _ram(phy_addr, 4,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem.dataspace()));
_ram.local() then gives the necessary virtual address. This works
vm_base.h for run/tz_vmm demo or vm.h for run/vmm, i.e. in the tz vmm user
space.
What can we do to get virtual addresses for physical addresses which are
already mapped?

You can just calculate them for locally mapped devices. In principle
this "Ram" class is nothing more than a helper utility that saves the
virtual and physical start address of a portion of RAM, and helps to
calculate offsets.

Post by r***@mpi-sws.org
E.g. in the imx53 sabre tablet demo, the 7 GPIO banks are
already mapped to virtual addresses in os/src/drivers/gpio/imx53/driver.h.
If we have a physical address (from DFAR) corresponding to the GPIO
address ranges, how can I get the virtual address in user space files
corresponding to tz vmm?

In case of the GPIO banks used by the GPIO driver the situation is
different. GPIO driver and VMM are different components with different
address spaces. Here you are sliding into the same problem like with the
UART used by the kernel. You cannot simply alter the memory-mapped I/O
registers within the VMM. I mean technically you can, by adding it as
local device like every other device as well, but you will end up in
devil's kitchen, when altering the registers from two different
"drivers" concurrently.
Instead, you have to analyse semantically what the Linux OS tries to do
with the device, and use the GPIO drivers API appropriatedly.

Post by r***@mpi-sws.org
Thanks!
Riju
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

[Stefan Kalkowski]

Post by r***@mpi-sws.org
In base-hw/src/core/include/spec/imx53/trustzone/csu.h, SECURE = 0x33.
So SECURE_UART = 1 in
base-hw/src/core/include/spec/imx53_qsb/trustzone/csu_config.h should
initialize UART as RD+WR for secure user and spvr models as given in Page
956 of http://www.nxp.com/docs/en/reference-manual/iMX53RM.pdf.
tz_vmm is in the secure user mode, sho it should be able to read an UART
address?

correct.

Post by r***@mpi-sws.org
Thanks!
Riju
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main

[Stefan Kalkowski]

Hi,

Post by r***@mpi-sws.org
In os/src/server/tz_vmm/spec/imx53/main.cc, added the following enums
(1) UART_BASE = 0x53FBC000,
(2) UART_SIZE = 0x43FFF
In os/src/server/tz_vmm/include/vm_base.h
In Vm_base constructor, added
(3)_ram_iomem_uart(uart_base, uart_size),
(4)_ram_uart(uart_base, uart_size,
(Genode::addr_t)Genode::env()->rm_session()->attach(_ram_iomem_uart.dataspace())),
In dump(), added
(5)printf("The memory location is %08lx\n", va_to_pa(_state->dfar));
(6) printf("_ram.local() is %08lx\n",_ram.local());
(7) printf("_ram.local() contains %08x\n",*(unsigned int*)(_ram.local()));
(8) printf("_ram_uart.local() is %08lx\n",_ram_uart.local());
(9) printf("_ram_uart.local() contains %08x\n",*(unsigned
int*)(_ram_uart.local()));
[init -> tz_vmm] The memory location is 53fbc080
[init -> tz_vmm] _ram.local() is 80000000
[init -> tz_vmm] _ram.local() contains eded1bb6
[init -> tz_vmm] _ram_uart.local() is 70078000
init -> tz_vmm -> vmm: raised unhandled data abort DFSR=0x00001008
ISFR=0x00000007 DFAR=0x70078000 ip=0x7001f440 sp=0xe01fed10
Thus though _ram and _ram_uart are assigned in the same way (over
different physical addresses), _ram.local() can be de-referenced and the
content read, while _ram_uart.local() gives a fault. How are these UART
addresses starting at 0x53FBC000 different from the
Trustzone::NONSECURE_RAM_BASE physical addresses?
Funnily, the DFSR=0x00001008, which is the same value as the abort in the
normal world linux. That can occur due to CSU configuration of UART made
secure. But the vmm is in secure world, why is that getting the same abort
at that same UART physical address?

To be honest, it's a bit exhausting to follow the trap-emulate path that
I discouraged you to follow from the first moment. We already
investigated that, and published a detailed report showing that
TrustZone indeed is not useful for doing device emulation. To me it is
not comprehensible why you are still focussing this solution.

Having said that, what you are trying to do is accessing a device
already in use by the kernel/core concurrently. You are trying to access
the same UART device directly that is used to print all messages of
core's LOG service. Although, that is certainly not the reason that you
are getting the page-fault. Anyway, it is a very bad idea to do so!

It seems to me that you are using an older version of Genode? Looking at
the current 'tz_vmm' component's source code, I can't clearly see what
constructs you are refering to, when adding (3) and (4). It would be
helpful to provide the whole source of the changed component. Without
any further insights, above lines seem reasonable.

Post by r***@mpi-sws.org
Thanks!
Riju
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
genode-main mailing list
https://lists.sourceforge.net/lists/listinfo/genode-main