Crate xarxa 

xarxa

Xarxa (pronounced “sharsha”): “Network” in Catalan

xarxa is a standalone network stack designed for embedded, real-time systems.

It can work without std and without alloc.

The design goals are the following, in order of decreasing priority:

• No unsafe. xarxa is exposed to the network and does complex packet parsing and manipulation. We want the guarantee that there is no memory safety vulnerabilities.
• Well suited for small embedded systems. This means low RAM usage, small code size.
• High performance
• Scales up to larger systems with faster links, more sockets, larger buffers.

xarxa vs smoltcp

xarxa is a rewrite/refactor of smoltcp aiming to address some design shortcomings that I felt were holding smoltcp back. (where “I” is Dario Nieuwenhuis (@dirbaio), smoltcp maintainer since 2020 until starting this project).

xarxa is basically a “port” of smoltcp to the new design. Many parts are not affected and are ported mostly unmodified (e.g TCP, wire), others look more different (e.g. the main Stack) but are a 1:1 port of the original logic wherever possible.

There’s two main design decisions where xarxa differs:

Memory management

smoltcp manages buffers in the following way:

• The device implementation owns a buffer pool where it stores Ethernet frames that are being received/sent.
• The device hands out borrows of them to the network stack via the TxToken/RxToken traits.
• All socket kinds (TCP, UDP, ICMP, raw) own one RX and one TX ring buffer.
• The network stack copies between the device buffers and the socket buffers.

This has a few implications:

• Zero-copy is impossible. You must do one copy between device and socket buffers.
• Multi-interface is very hard to implement. (smoltcp is currently single-interface).
  • TxToken/RxToken make the Device trait not dyn-compatible.
  • The “receive gives you a TxToken to send the reply” trick doesn’t work anymore because the reply to a packet may need to go out another interface.
• It’s very memory-inefficient.
  • You need to pay at least 2x MTU (1500*2 = 3kb) of RAM for each UDP/raw socket you create. This is why smoltcp implements DNS and DHCP sockets as dedicated DnsSocket and DhcpSocket types instead of building them on top of UDP and raw sockets. This is not very elegant.
  • For multi-interface you need dedicated pools per interface.

Instead, xarxa has a single global packet pool. It passes owned handles around, so any part of the stack (device, sockets, everything else) can easily own packets. This unlocks many improvements:

• Multi-interface becomes trivial.
• Zero-copy is now possible. An interface writes a received packet into a buffer, which then goes through dispatch, gets queued in a socket, and then gets handed to the user. Same for egress.
• It allows fixing “structurally unfixable” bugs like the tx queue clog when sending to multiple IPs from a single socket.

“Repr” structs

smoltcp defines plain old Rust structs such as IpRepr, UdpRepr. At ingress, it reads the packet wire bytes and deserializes it into reprs. At egress, the reprs are serialized to bytes. The entire core works with these repr structs.

xarxa makes the core work with the packet bytes directly instead. Why?

• It’s faster. Serializing and deserializing is work that doesn’t add value. It just loads from RAM in one format and writes in another format. The compiler is not good at optimizing it out.
• Smaller code size, for the same reason.
• Allows for actually-raw raw sockets. smoltcp raw sockets drop fields from the IP header because packets get deserialized and reserialized. The repr structs are intentionally incomplete, they don’t contain fields that the stack doesn’t look at. Adding them would hurt code size and perf. I’ve attempted refactors to avoid this reserialization in the past but they ended up too invasive and ugly since you basically need to add a way to thread raw bytes through the whole stack.

Benchmarks

xarxa is faster and smaller than smoltcp. Code size and TCP perf don’t quite reach lwIP.

Notes:

• lwIP TCP RX is zero-copy, which is why it’s so much faster. The API is different, it hands the data to the user by calling a callback synchronously from ingress code. This is unfair against xarxa and smoltcp: they could also easily do zero-copy TCP RX if they were also allowed to have such a terrible API.
• lwIP TCP TX is one-copy (it could be zero copy but it would again be unfair, since it forces the application to let the buffer live until the hardware is done with it). I haven’t yet investigated why it’s so fast even if it does one copy.

Benchmark source code is available here.

Features

• Multiple interface support
  • Add/remove interfaces dynamically to the network stack.
  • Each interface has its own configuration (like IP addresses)
  • A route table chooses which interface to use on egress.
  • Mixing mediums in is supported.
  • The driver reports its hardware address and link state to the stack.
• Ethernet interface medium (feature medium-ethernet)
  • Does IPv4 ARP, IPv6 NDISC.
  • Neighbor cache with expiry, renewal on use.
  • The network stack buffers egress packets pending network resolution. Unreachable neighbors don’t clog sockets.
• Pure IP interface medium (feature medium-ip)
• IEEE 802.15.4 interface medium (feature medium-ieee802154)
  • 6LoWPAN header compression: IPHC for the IPv6 header, NHC for UDP and extension headers, done in place in the packet buffer.
  • Address contexts for decompression.
  • NDISC over 802.15.4, link-local address from the extended address.
  • 6LoWPAN fragmentation (feature sixlowpan-fragmentation)
  • 6LoWPAN reassembly (feature sixlowpan-reassembly)
• IPv4 (feature ipv4)
  • DHCP client (feature dhcpv4)
    • Raw access to all lease options by option number. (feature dhcpv4-options)
  • Fragmentation (feature ipv4-fragmentation)
  • Reassembly (feature ipv4-reassembly)
• IPv6 (feature ipv6)
  • Link-local address automatically derived from the MAC address (EUI-64).
  • SLAAC: addresses and default routes from router advertisements, with lifetimes. (feature slaac)
• ICMP
  • Automatically replies to pings. (feature icmp-ping-reply)
  • Incoming ICMP errors are routed to the socket that caused them. (feature icmp-errors)
• UDP sockets (feature udp)
  • zero-copy on both TX and RX
  • Supports all binding modes Linux supports, including unconnected (receives from any IP) and connected sockets (receives from one fixed remote IP+port).
• Raw sockets (feature raw)
  • zero-copy on both TX and RX
  • Ethernet-layer raw sockets transmit/receive raw Ethernet frames. No routing, you choose the interface manually.
  • IP-layer raw sockets transmit/receive raw IP packets. The stack handles routing same as other socket types.
  • IP headers are byte-copied instead of parsed+re-emitted, so all fields and options are kept, even those unsupported by xarxa.
• TCP sockets (feature tcp)
  • Full TCP implementation
  • TCP listeners implement an accept queue. Buffers are not allocated until you accept() a connection. (feature tcp-listener)
  • Window scaling
  • Configurable keepalive.
  • RTT estimation automatically tunes retransmission timeout
  • Compile time selection of CUBIC, Reno or no congestion control.
  • Nagle’s algorithm (defaults to enabled, can be turned off)
  • Delayed ACK (defaults to enabled, can be turned off)
  • Zero-window probes
  • TCP Timestamps (feature tcp-timestamps)
  • TCP SACK, sending ranges only (feature tcp-sack)
• DNS client (feature dns)
  • Multiple servers, retransmission with backoff.
  • Multicast DNS for .local names (feature mdns)
• IP multicast (feature multicast)
  • Join and leave multicast groups per interface.
  • IGMPv1/IGMPv2 (IPv4) and MLDv2 (IPv6): membership is reported on join and leave, and in response to router queries.
  • The IPv6 solicited-node groups of the interface’s addresses are joined automatically.
• Packet metadata
  • Support for hardware timestamping on both RX and TX. Allows implementing protocols like PTP, NTP. (feature packetmeta-timestamp)
  • Opaque ID for correlating packets through the stack. (feature packetmeta-id)
• Checksum offload: interfaces report which checksums they can validate/calculate and the stack skips them.

Not yet implemented

All of the below is planned. Please open an issue or reach out on the Matrix chat if you want to work on one of these so we don’t duplicate work.

• DHCP server
• Acting on link state: skipping down interfaces on egress routing, restarting DHCP/SLAAC on link-up.
• IPv6 DAD (duplicate address detection)
• IPv6 RDNSS (DNS servers from router advertisements)
• an equivalent to smoltcp’s any_ip
• IPv6 fragmentation and reassembly
• TCP segmentation offload
• TCP SACK, acting on ranges received from the peer.
• Store sockets on a hashmap so packet dispatch is O(1) instead of O(n). (would be optional with a Cargo feature, it’s only worth if you have thousands of sockets, i.e. not on embedded)
• Maybe multithreading. Would require per-socket mutexes etc. (again, optional, would require std)

License

xarxa is distributed under the terms of 0-clause BSD license.

See LICENSE-0BSD for details.

Feature flags

• alloc (enabled by default) — Use the heap. Used for:
  • Address, route tables.
  • Socket slabs
  • Owned interfaces
  • Owned TCP socket buffers
• std (enabled by default) — Enable functionality that requires the Rust standard library: the driver_impls module (TunTapDriver, RawSocketDriver, wait) and Instant::now().
• async (enabled by default) — Waker registration on sockets, for building Futures on top of the stack.

Protocol support

• medium-ethernet (enabled by default) — Support interfaces that send and receive Ethernet frames, with the link-layer machinery that goes with them (ARP, NDISC, the neighbor cache).
• medium-ip (enabled by default) — Support interfaces that send and receive bare IP packets, without a link-layer header.
• medium-ieee802154 (enabled by default) — Support IEEE 802.15.4 interfaces carrying 6LoWPAN (RFC 4944, RFC 6282). Needs ipv6.
• sixlowpan-fragmentation (enabled by default) — Fragment outgoing 6LoWPAN packets larger than one 802.15.4 frame. Needs medium-ieee802154.
• sixlowpan-reassembly (enabled by default) — Reassemble incoming 6LoWPAN fragments. Needs medium-ieee802154.
• ipv4 (enabled by default) — Support IPv4 (and, with medium-ethernet, ARP).
• ipv4-fragmentation (enabled by default) — Fragment outgoing IPv4 packets larger than the interface MTU. Needs ipv4.
• ipv4-reassembly (enabled by default) — Reassemble incoming IPv4 fragments. Needs ipv4.
• ipv6 (enabled by default) — Support IPv6 (and, with medium-ethernet, NDISC).
• raw (enabled by default) — Raw sockets.
• udp (enabled by default) — UDP sockets.
• tcp (enabled by default) — TCP sockets.
• tcp-listener (enabled by default) — TCP listeners, for accepting incoming connections.
• dhcpv4 (enabled by default) — DHCPv4 client, built into the interface. Needs ipv4 and medium-ethernet.
• dhcpv4-options (enabled by default) — Allow reading received DHCP options. Requires a small buffer. Size is set by the dhcp-options-buf-size-N features.
• slaac (enabled by default) — IPv6 stateless address autoconfiguration (SLAAC). Needs ipv6, and medium-ethernet or medium-ieee802154.
• dns (enabled by default) — DNS client.
• mdns (enabled by default) — Resolve .local names with multicast DNS in the DNS client.
• multicast (enabled by default) — Join IP multicast groups, with IGMP (IPv4) and MLD (IPv6) membership reports.
• tcp-timestamps (enabled by default) — Send and receive the TCP timestamp option (RFC 7323).
• tcp-sack (enabled by default) — Send selective acknowledgement ranges (RFC 2018).
• icmp-ping-reply (enabled by default) — Automatically reply to pings (ICMP echo requests).
• icmp-errors (enabled by default) — Deliver incoming ICMP error messages (destination unreachable, packet too big, …) to the sockets that provoked them.

TCP congestion control

Enable one of these features to enable congestion control. No feature enabled means no congestion control. You may enable at most one.

• tcp-reno — Use the Reno congestion control algorithm on TCP sockets.
• tcp-cubic (enabled by default) — Use the CUBIC congestion control algorithm on TCP sockets. Warning: it uses f64 arithmetic which can be slow and pull in soft-float code depending on the target.

Packet metadata

• packetmeta-id (enabled by default) — Enable the PacketMeta::id field: an opaque number that travels with a packet through the whole stack.

• packetmeta-timestamp (enabled by default) — Enable the PacketMeta::timestamp and PacketMeta::request_timestamp fields, and Driver::poll_tx_timestamp.

  Transmit timestamps are correlated back to the packet that produced them by PacketMeta::id, so this implies packetmeta-id.

Logging

• log (enabled by default) — Log with the log crate.
• defmt — Log with the defmt crate.
• packet-log — Log every packet received and sent, decoded. Needs log or defmt to print anything.

Re-exports

pub use xarxa_driver as driver;

Modules

config

Compile-time configuration.

dns

DNS client.

driver_impls

Driver implementations for the host OS.

iface

Network interfaces.

raw

Raw sockets.

route

IP routing.

tcp

TCP sockets.

time

Time structures.

udp

UDP sockets.

wire

Low-level packet access and construction.

Structs

Full

A table, slab or queue has no room for another item.

Neighbor

An entry in the NeighborCache.

NeighborCache

The neighbor cache: the stack’s map of IP addresses to hardware addresses.

Stack

A network stack.

Enums

IcmpError

ICMP error reported against a socket.

NeighborState

State of a Neighbor entry.