Socket fs

[Boris Mulder]

Hi all,

I've managed to port one of our existing network applications to use the
socket fs from 17.02 now, starting from the echo_udp scenario and got it
working. It looks a lot better now.

Following this, we would like to share our findings about the socket fs
so far:

1. In the release notes you describe the support for async file I/O.
However, it seems that while asynchronous I/O works now, synchronous I/O
is no longer possible. When I remove the select() call in the echo
scenario, the call to recvfrom() will fail saying the socket is not
connected (even though it is an UDP socket and should not have the
notion of connections at all).

Of course this means it is not posix compatible; the intended behaviour
is to block until data becomes available.

2. Currently there are two ways an application can use tcp/ip sockets
from the lxip stack: By using libc sockets, and by directly opening a
file system session and writing to the right socket files. For any new
Genode application that does not already depend on libc, the first way
means you unnecessarily add the entirety of libc to your component. So
the second way seems preferred.

However, when programming an application using a file system session
directly adds a lot of code with many open()s, read()s and write()s that
could be abbreviated into single calls such as connect() and recv().
Have you thought of providing a more convenient API (such an
object-oriented Socket class that hides the vfs calls from the
application programmer)? Or do you consider using the fs session
interface to be better? If so, why?

[Christian Helmuth]

Hello Boris,

Post by Boris Mulder
1. In the release notes you describe the support for async file I/O.
However, it seems that while asynchronous I/O works now, synchronous I/O
is no longer possible. When I remove the select() call in the echo
scenario, the call to recvfrom() will fail saying the socket is not
connected (even though it is an UDP socket and should not have the
notion of connections at all).
Of course this means it is not posix compatible; the intended behaviour
is to block until data becomes available.

Thanks for your investigation. You're right the current state of
socket support has some rough edges (esp. POSIX-correct error
handling) that we plan to smoothen in near term. I'd like to invite
you to open an issue at Github to aggregate your findings.

Post by Boris Mulder
2. Currently there are two ways an application can use tcp/ip sockets
from the lxip stack: By using libc sockets, and by directly opening a
file system session and writing to the right socket files. For any new
Genode application that does not already depend on libc, the first way
means you unnecessarily add the entirety of libc to your component. So
the second way seems preferred.
However, when programming an application using a file system session
directly adds a lot of code with many open()s, read()s and write()s that
could be abbreviated into single calls such as connect() and recv().
Have you thought of providing a more convenient API (such an
object-oriented Socket class that hides the vfs calls from the
application programmer)? Or do you consider using the fs session
interface to be better? If so, why?

We do not plan to implement an abstraction layer or object-oriented
Socket interface currently. The mere reason for our recent
developments was to enable more POSIX network applications to run on
Genode and to support those components to share one network stack.
Also, we imagine to mount the socket FS into Noux and use traditional
tools in shell scripts to interoperate with the network.

I must admit that we (the Genode Labs team) do not assess ourselves as
networking experts and, therefore, don't fancy to design and implement
such a library. But as always nothing is set in stone and priorities
may shift over time.

Regards

[Emery Hemingway]

Hello Boris,

In my opinion, once you start using TCP/IP, you cross a complexity
threshold where it makes sense to use high-level langauges or
networking libraries.

The alternative would be to use the lwIP raw API and avoid libc, but
this is not exposed right now. Another option would be the Rust way, I
expect there is atleast Rust TCP stack in the works, so in theory a
Rust component could be crafted to use Nic sessions directly.


Cheers,
Emery


On Mon, 27 Mar 2017 13:25:43 +0200

Post by Christian Helmuth
Hello Boris,

Post by Boris Mulder
1. In the release notes you describe the support for async file I/O.
However, it seems that while asynchronous I/O works now,
synchronous I/O is no longer possible. When I remove the select()
call in the echo scenario, the call to recvfrom() will fail saying
the socket is not connected (even though it is an UDP socket and
should not have the notion of connections at all).
Of course this means it is not posix compatible; the intended
behaviour is to block until data becomes available.

Thanks for your investigation. You're right the current state of
socket support has some rough edges (esp. POSIX-correct error
handling) that we plan to smoothen in near term. I'd like to invite
you to open an issue at Github to aggregate your findings.

Post by Boris Mulder
2. Currently there are two ways an application can use tcp/ip
sockets from the lxip stack: By using libc sockets, and by directly
opening a file system session and writing to the right socket
files. For any new Genode application that does not already depend
on libc, the first way means you unnecessarily add the entirety of
libc to your component. So the second way seems preferred.
However, when programming an application using a file system
session directly adds a lot of code with many open()s, read()s and
write()s that could be abbreviated into single calls such as
connect() and recv(). Have you thought of providing a more
convenient API (such an object-oriented Socket class that hides the
vfs calls from the application programmer)? Or do you consider
using the fs session interface to be better? If so, why?

We do not plan to implement an abstraction layer or object-oriented
Socket interface currently. The mere reason for our recent
developments was to enable more POSIX network applications to run on
Genode and to support those components to share one network stack.
Also, we imagine to mount the socket FS into Noux and use traditional
tools in shell scripts to interoperate with the network.
I must admit that we (the Genode Labs team) do not assess ourselves as
networking experts and, therefore, don't fancy to design and implement
such a library. But as always nothing is set in stone and priorities
may shift over time.
Regards

[Boris Mulder]

That would be nice. For now, I've used the lxip stack in my program (and
wrote a convenient wrapper for it), since it depended on old C/C++ code
and therefore I needed a C++ API for opening UDP sockets.

But using lwip without libc could decrease the complexity and might be
better indeed.

I would guess a rust TCP stack would mainly be useful for Rust applications.

Post by Christian Helmuth
Hello Boris,
In my opinion, once you start using TCP/IP, you cross a complexity
threshold where it makes sense to use high-level langauges or
networking libraries.
The alternative would be to use the lwIP raw API and avoid libc, but
this is not exposed right now. Another option would be the Rust way, I
expect there is atleast Rust TCP stack in the works, so in theory a
Rust component could be crafted to use Nic sessions directly.
Cheers,
Emery