General Information.

Still waiting for interface finalization and proper integration.

See user-space device drivers for generic discussion related to user-space device drivers.

Disk Drivers

Not yet supported.

The plan is to use libstore parted for accessing partitions.

Upstream Status

IRC, freenode, #hurd, 2012-02-08

After the microkernel devroom at FOSDEM 2012:

<antrik> there was quite some talk about DDE. I learnt that there are newer
  versions in Genode and in Minix (as opposed to the DROPS one we are
<antrik> but apparently none of the guys involved is interested in creating
  a proper upstream project with central repository and communication
  channels :-(
<antrik> the original DDE creator was also there, but said he isn't working
  on it anymore
<tschwinge> OK, and the other two projects basically have their own forks.
<tschwinge> Or are they actively cooperating?
<tschwinge> (If you know about it.)
<antrik> well, Genode is also part of the Dresden L4 group; but apart from
  that, I'd rather call it a fork...
<antrik> hm... actually, I'm not sure anymore whether the guy I talked to
  was from Genode or Nova...
<antrik> (both from the Dresdem L4 group)

IRC, freenode, #hurd, 2012-08-12

<antrik> I wonder whether the very detailed explanation was prompted by our
  DDE discussions at FOSDEM...
<pinotree> antrik: one could think about approaching them to develop the
  common dde libs + dde_linux together
<antrik> pinotree: that's what I did at FOSDEM -- they weren't interested
<pinotree> antrik: this year's one? why weren't they?
<pinotree> maybe at that time dde was not integrated properly yet (netdde
  is just few months "old")
<braunr> do you really consider it integrated properly ?
<pinotree> no, but a bit better than last year
<antrik> I don't see what our integration has to do with anything...
<antrik> they just prefer hacking thing ad-hoc than having some central
<pinotree> the helenos people?
<antrik> err... how did helenos come into the picture?...
<antrik> we are talking about genode
<pinotree> sorry, confused wrong microkernel OS
<antrik> actually, I don't remember exactly who said what; there were
  people from genode there and from one or more other DDE projects... but
  none of them seemed interested in a common DDE
<antrik> err... one or two other L4 projects

IRC, freenode, #hurd, 2012-02-19

<youpi> antrik: do we know exactly which DDE version Zheng Da took as a
  base ?
<youpi> (so as to be able to merge new changes easily)
<antrik> youpi: not sure... but from what I gathered at FOSDEM, the version
  he based on (from DROPS) is not really actively developed right now; if
  we want to go for newer versions, we probably have to look at other
  projects (like Genode or Nova or Minix)
<youpi> there's no central project for dde ?
<youpi> that sucks
<antrik> no... and nobody seemed interested in having one :-(
<youpi> pff
<antrik> which makes me seriously question the original decision to build
  on DDE...
<braunr> ..
<antrik> if we have to basically maintain it ourselfs anyways, we could
  just as well have gone with custom glue
<youpi> well, the advantage of DDE is that it already exists now
<antrik> on the positive side, one of the projcets (not sure which)
  apparently have both USB and SATA working with some variant of DDE

IRC, freenode, #hurd, 2012-11-03

<mcsim> DrChaos: there is DDEUSB framework for L4. You could port it, if
  you want. It uses Linux 2.6.26 usb subsystem.

IRC, freenode, #hurd, 2013-02-15

After the microkernel devroom at FOSDEM 2013.

<pinotree> youpi: speaking of dde, was there any will among other
  microkernel os developers to eventually develop one single dde (with
  every team handling the custom glue of the own kernel)?
<youpi> well, there is still upstream dde actually
<youpi> in dresden
<youpi> nothing was really decided or anything (it was a round table, not a
<youpi> but conversation converged into sharing the DDE maintenance, yes
<youpi> and dresden would be the logical central place
<youpi> pb is that they don't have the habit of being very open
<youpi>  has a recent
  enough version
<youpi> which macsim confirmed having all the latest commits from the
  internal repository
<pinotree> i see
<youpi> so it seems a viable solution on the medium term
<youpi> the long term might need a real visible open source project
<youpi> but we should probably still keep basing on dresden work
<youpi> (better take work being done anywhere)
<pinotree> well, if the upstream is not really open, microkernel teams
  could just fork it and all work on it
<youpi> that's what I mean
<pinotree> should still be a win than everybody maintaining their own dde
<youpi> sure
<pinotree> ah yes, i was writing and i'm slow at it :)
<youpi> but at least we can try to work with dresden
<youpi> see how open they could become by just asking :)
<pinotree> right

IRC, OFTC, #debian-hurd, 2012-02-15

<pinotree> i have no idea how the dde system works
<youpi> gnumach patch to provide access to physical memory and interrupts
<youpi> then userland accesses i/o ports by hand to drive things
<youpi> but  that assumes that no kernel driver is interfering
<youpi> so one has to disable kernel drivers
<pinotree> how are dde drivers used? can they be loaded on their own
  automatically, or you have to settrans yourself to setup a device?
<youpi> there's no autoloader for now
<youpi> we'd need a bus arbitrer that'd do autoprobing

PCI arbiter.

<pinotree> i see
<pinotree> (you see i'm not really that low level, so pardon the flood of
  posssibly-noobish questions ;) )
<youpi> I haven't set it up yet, but IIRC you need to specify which driver
  to be used
<youpi> well, I mostly have the same questions actually :)
<youpi> I just have some guesswork here :)
<pinotree> i wonder whether the following could be feasible:
<youpi> I'm wondering how we'll manage to make it work in d-i
<pinotree> a) you create a package which would b-d on linux-source, build a
  selection of (network only for now) drivers and install them in
<youpi> probably through a choice at the boot menu
<youpi> I wouldn't dare depending on linux-source
<youpi> dde is usually not up-to-date
<pinotree> b) add a small utility over the actual fsys_settrans() which
  would pick the driver from /hurd/dde/
<pinotree> ... so you could do `set-dde-driver b43 <device>` (or something
  like that)
<youpi> we can provide something like b) yes
<youpi> although documenting the settrans should be fine enough ;)
<pinotree> the a) would help/ease with the fact that you need to compile on
  your own the drivers
<pinotree> otherwise we would need to create a new linux-dde-sources-X.Y.Z
  only with the sources of the drivers we want from linux X.Y.Z
<pinotree> (or hurd-dde-linux-X.Y.Z)
<CIA-4> samuel.thibault * raccdec3 gnumach/debian/ (changelog
  patches/70_dde.patch patches/series): 
<CIA-4> Add DDE experimental support
<CIA-4> * debian/patches/70_dde.patch: Add experimental support for irq
  passing and
<CIA-4> physical memory allocation for DDE. Also adds nonetdev boot
  parameter to
<CIA-4> disable network device drivers.
<youpi> ok, boots fine with the additional nonetdev option
<youpi> now I need to try that dde hurd branch :)
<CIA-4> samuel.thibault * rf8b9426 gnumach/debian/patches/70_dde.patch: Add
  experimental.defs to gnuamch-dev

IRC, freenode, #hurd, 2012-02-19

* youpi got dde almost working
<youpi> it's able to send packets, but apparently not receive them
<youpi> (e1000)
<youpi> ok, rtl8139 works
<youpi> antrik: the wiki instructions are correct
<youpi> with e1000 I haven't investigated
<antrik> (Archhurd guys also reported problems with e1000 IIRC... the one I
  built a while back works fine though on my T40p with real e1000 NIC)
<antrik> maybe I should try with current versions... something might got
  broken by later changes :-(
<youpi> at least testing could tell the changeset which breaks it
<youpi> Mmm, it's very odd
<youpi> with the debian package, pfinet's call to device_set_filter returns
<youpi> and indeed devnode.c returns that
<youpi> ah but it's libmachdev which is supposed to answer here
<antrik> youpi: so, regarding the failing device_set_filter... I guess you
  are using some wrong combination of gnumach and pfinet
<youpi> no it's actually that my pfinet was not using bpf
<youpi> I've now fixed it
<antrik> the DDE drivers rely on zhengda's modified pfinet, which uses
  devnode, but also switched to using proper BPF filters. so you also need
  his BPF additions/fixes in gnumach
<antrik> OK
<youpi> that's the latter
<youpi> I had already fixed the devnode part
<youpi> but hadn't seen that the filter was different
<antrik> err... did I say gnumach? that of course doesn't come into play
<antrik> so yes, you just need a pfinet using BPF
<youpi> libmachdev does ;)
<antrik> I'm just using pfinet from zhengda's DDE branch... I think devnode
  and BPF are the only modifications
<youpi> there's also a libpcap modification in the incubator
<youpi> probably for tcpdump etc.
<antrik> libpcap is used by the modified pfinet to compile the filter rule
<youpi> why does pfinet need to compile the rule ?
<youpi> it's libbpf which is used in the dde driver
<antrik> it doesn't strictly need to... but I guess zhengda considered it
  more elegant to put the source rule in pfinet on compile it live, rather
  than the compiled blob
<antrik> I probably discussed this with him myself a few years back... but
  my memory on this is rather hazy ;-)
<antrik> err... and compile it live
<youpi> ah, right, it's only used when asking pfinet to change its filter
<youpi> but it does not need it for the default filter
<youpi> which is hardcoded
<antrik> I see
<antrik> when would pfinet change its filter?
* youpi now completely converting his hurd box to debian packages with dde
<youpi> on SIOCSIFADDR apparently
<youpi> to set "arp or (ip host %s)",
<antrik> well, that sounds like the default filter...
<youpi> the default filter does not choose an IP
<antrik> oh, right... pfinet has to readjust the filter when setting the IP
<youpi> arg we lack support for kernel options for gnumach in update-grub
<antrik> again, I have a vague recollection of discussing this
* youpi crosses fingers
<youpi> yay, works
<antrik> so we *do* need libpcap in pfinet to set proper rules... though I
  guess it can also work with a static catchall rule (like it did before
  zhengda's changes), only less efficient
<youpi> well in the past we were already catching everything anyway, so at
  least it's not a regression :)
<antrik> right

PCI Arbiter

IRC, freenode, #hurd, 2012-02-21

<youpi> since all drivers need is interrupts, io ports and iomem
<youpi> the latter was already available through /dev/mem
<youpi> io ports through the i386 rpcs
<youpi> the changes provide both interrupts, and physical-contiguous
<youpi> it should be way enough
<braunr> youpi: ok
<braunr> youpi: thanks for the details :)
<antrik> braunr: this was mentioned in the context of the interrupt
  forwarding interface... the original one implemented by zhengda isn't
  suitable for a PCI server; but the ones proposed by youpi and tschwinge
  would work
<antrik> same for the physical memory interface: the current implementation
  doesn't allow delegation; but I already said that it's wrong

IRC, freenode, #hurd, 2012-02-20

<youpi> I was a bit wary of including the ton of dde headers in hurd-dev
<youpi> maybe adding another package for that
<youpi> but that would have delayed introducing the dde binaries
<youpi> probably we can do that for next upload
<pinotree> i can try to work on it, if is feasible (ie if the dde drivers
  can currently be built from outside the hurd source tree)
<youpi> it should be, it's a matter of pointing its makefile to a place
  where the make scripts and include headers are
<youpi> (and the libraries)
<pinotree> ok
<antrik> youpi: you mean DDEKit headers?
<antrik> pinotree: actually it doesn't matter where the dde-ified Linux
  drivers are built -- libdde_linux26 and the actual drivers use a
  completetly different build system anyways
<antrik> in fact we concluded at some point that they should live in a
  separate repository -- but that change never happened
<antrik> only the base stuff (ddekit, libmachdev etc.) belong in the Hurd
  source tree
<youpi> antrik: yes
<youpi> antrik: err, not really completely different
<youpi> the actual drivers' Makefile include the libdde_linux26 mk files
<youpi> the build itself is separate, though
<antrik> youpi: yes, I mean both libdde_linux26 and the drivers use a build
  system that is completely distinct from the Hurd one
<youpi> ah, yes
<youpi> libdde_linux26 should however be shipped in the system
<antrik> ideally libdde_linux26 and all the drivers should be built in one
  go I'd say...
<youpi> it should be easily feasible to also have a separate driver too
<youpi> e.g. to quickly try a 2.6 driver
<antrik> youpi: I'm not sure about that. it's not even dynamically linked
<youpi> with scripts to build it
<youpi> it's not
<youpi> but that doesn't mean it can't be separate
<youpi> .a files are usually shipped in -dev packages
<antrik> youpi: ideally we should try to come with a build system that
  reuses the original Linux makefile snippets to build all the drivers
  automatically without any manual per-driver work
<youpi> there's usually no modification of the drivers themselves?
<youpi> but yeah
<youpi> "ideally", when somebody takes the time to do it
<antrik> unfortunately, it's necessary to include one particular
  Hurd/DDE-specific header file in each driver :-(
<youpi> can't it be done through gcc's -include option?
<antrik> zhengda didn't find a way to avoid this... though I still hope
  that it must be possible somehow
<antrik> I think the problem is that it has to be included *after* the
  other Linux headers. don't remember the details though
<youpi> uh
<youpi> well, a good script can add a  line after the last occurrence of
<antrik> yeah... rather hacky, but might work
<youpi> even with a bit of grep, tail, cut, and sed it should work :)
<antrik> note that this is Hurd-specific; the L4 guys didn't need that
<youpi> what is it?
<antrik> don't remember off-hand

IRC, freenode, #hurd, 2012-02-22

<youpi> antrik: AIUI, it should be possible to include all network drivers
  in just one binary?
<youpi> that'd permit to use it in d-i
<youpi> and completely replace the mach drivers
<youpi> we just need to make sure to include at least what the mach drivers
<youpi> (all DDE network drivers, I mean)
<youpi> of course that doesn't hinder from people to carefully separate
  drivers in several binaries if they wish
<youpi> antrik: it does link at least, I'll give a try later
<youpi> yes it works!
<youpi> that looks like a plan
<youpi> throw all network drivers in a /hurd/dde_net
<youpi> settrans it on /dev/dde_net, and settrans devnode on /dev/eth[0-9]
<youpi> I'm also uploading a version of hurd which includes headers &
  libraries, so you just need a make in dde_{e100,e1000,etc,net}
<youpi> (uploading it with the dde driver itself :) )
<youpi> btw, a nice thing is that we don't really care that all drivers are
  stuffed into a single binary, since it's a normal process only the useful
  pages are mapped and actually take memory :)
<Tekk_> is that really a nice thing though? compared to other systems I
<Tekk_> I know on linux it only loads the modules I need, for example. It's
  definitely a step up for hurd though :D
<youpi> that's actually precisely what I mean
<youpi> you don't need to load only the modules you need
<youpi> you just load them all
<youpi> and paging eliminates automatically what's not useful
<youpi> even parts of the driver that your device will not need
<Tekk_> ooh
<Tekk_> awesome
<youpi> (actually, it's not even loaded, but the pci tables of the drivers
  are loaded, then paged out)

IRC, freenode, #hurd, 2012-02-24

<youpi> antrik_: about the #include <ddekit/timer.h>, I see the issue, it's
  about jiffies
<youpi> it wouldn't be a very good thing to have a jiffies variable which
  we'd have to update 100times per second
<youpi> so ZhengDa preferred to make jiffies a macro which calls a function
  which reads the mapped time

Mapped-time interface.

<youpi> however, that break any use of the work "jiffies", e.g. structure
  members & such
<youpi> actually it's not only after headers that the #include has to be
  done, but after any code what uses the word "jiffies" for something else
  than the variable
<youpi> pb is: it has to be done *before* any code that uses the word
  "jiffies" for the variable, e.g. inline functions in headers
<youpi> in l4dde, there's already the jiffies variable so it's not a

IRC, OFTC, #debian-hurd, 2012-02-27

<tschwinge> I plan to do some light performance testing w.r.t. DDE
  Ethernet.  That is DDE vs. Mach, etc.
<youpi> that'd be good, indeed
<youpi> I'm getting 4MiB/s with dde
<youpi> I don't remember with mach
<tschwinge> Yes.  It just shouldn't regress too much.
<tschwinge> Aha, OK.

IRC, freenode, #hurd, 2012-02-27

<youpi> tschwinge: nttcp tells me ~80Mbps for mach-rtl8139, ~72Mbps for
  dde-rtl8139, ~72Mbps for dde-e1000
<youpi> civodul: ↑ btw
<ArneBab> youpi: so the dde network device is not much slower than the
<civodul> youpi: yes, looks good
<ArneBab> rather almost the same speed
<youpi> apparently
<ArneBab> that’s quite a deal. 
<ArneBab> what speed should it have as maximum?
<ArneBab> (means: does the mach version get out all that’s possible?)
<ArneBab> differently put: What speed would GNU/Linux get?
<youpi> I'm checking that right now
<ArneBab> cool!
<ArneBab> we need those numbers for the moth after the next
<youpi> Mmm, I'm not sure you really want the linux number :)
<youpi> 1.6Gbps :)
<ArneBab> oh…
<youpi> let me check with udp rather than tcp
<ArneBab> so the Hurd is a “tiny bit” worse at using the network… 
<youpi> it might simply be an issue with tcp tuning in pfinet
<ArneBab> hm, yes
<ArneBab> tcp is not that cheap
<ArneBab> and has some pretty advanced stuff for getting to high speeds
<youpi> well, I'm not thinking about being cheap
<youpi> but using more recent tuning
<youpi> that does not believe only 1Mbps network cards exist :)
<ArneBab> like adaptive windows and such?
<ArneBab> :)
<youpi> yes
* ArneBab remembers that TCP was invented when the connections went over
    phone lines - by audio :)
<youpi> yep
<ArneBab> what’s the system load while doing the test?
<youpi> yes, udp seems not so bad
<ArneBab> ah, cool!
<youpi> it's very variable (300-3000Mbps), but like on linux
<ArneBab> that pushing it into user space has so low cost is pretty nice.
* ArneBab thinks that that’s a point where Hurd pays off
<youpi> that's actually what AST said to fosdem
<youpi> he doesn't care about putting an RPC for each and every port i/o
<youpi> because hardware is slow anyway
<ArneBab> jupp
<ArneBab> but it is important to see that in real life

IRC, freenode, #hurd, 2012-04-01

<youpi> antrik: I wonder whether you could actually not route the IRQs to a
  non-zero ring, AIUI you can in the x86 IDT table
<antrik> youpi: you mean having a userspace server for each (non-timer)
<antrik> youpi: how would a userspace IRQ handler interact with the
<youpi> antrik: it doesn't necessarily have to
<youpi> provided that it's trusted
<antrik> youpi: how would you do CPU time accounting if there is no
  interaction with the scheduler?...
<youpi> antrik: you don't necessarily want to care about it
<antrik> youpi: well, that would mean that all drivers handling interrupts
  would have to be trusted to not use more than a very small part of CPU
<youpi> yes
<youpi> which is usually needed for interrupt handlers anyway
<antrik> youpi: nah, the bottom handler only has to do very basic stuff;
  afterwards, we can pass off to "normal" driver processes, scheduled just
  like other processes... but that requires some interaction between the
  IRQ handler and the scheduler I think
<youpi> the IRQ handler can wake up a thread, yes
<youpi> no need for anything special there
<antrik> so the userspace IRQ server would just decide what process to wake
  up, and then call the scheduler to do a normal task switch? I guess
  that's possible; but I'm not sure it would buy much...
<youpi> it would permit userland to quickly react to the IRQ
<youpi> such as acknowledge it to the hardware etc.
<antrik> yeah, but my point is that I don't see much benefit in having this
  part of the code isolated in a userspace process... it has to be trusted
  anyways, and it's pretty trivial too
<youpi> I never said it was a good idea

IRC, freenode, #hurd, 2012-04-06

<braunr> oh i forgot about my work on pcap
<braunr> is devnode (or devopen or whatever) in the upstream repository now
<antrik> can't say for sure, but I'd be surprised... don't remember seeing
  any movement in that regard :-(
<braunr> wasn't it needed for dde ?
<antrik> hm... good point

IRC, freenode, #hurd, 2013-09-20

<braunr> i should take some time to integrate my pcap changes into the
  libpcap debian package at least
<pinotree> braunr: if upstream is active, i'd say to go there directly
<braunr> the problem with that approach is that netdde is still not part of
  our upstream code
<pinotree> don't understand the relation
<braunr> i don't want to send the pcap guys code for an interface that is
  still not considered upstream ...

IRC, freenode, #hurd, 2012-08-14

<braunr> it's amazing how much code just gets reimplemented needlessly ...
<braunr> libddekit has its own mutex, condition, semaphore etc.. objects
<braunr> with the *exact* same comment about the dequeueing-on-timeout
  problem found in libpthread
<braunr> *sigh*

IRC, freenode, #hurd, 2012-08-18

<braunr> hum, leaks and potential deadlocks in libddekit/thread.c :/

IRC, freenode, #hurd, 2012-08-18

<braunr> nice, dde relies on a race to start ..

IRC, freenode, #hurd, 2012-08-21

In context of libpthread.

<braunr> hm, i thought my pthreads patches introduced a deadlock, but
  actually this one is present in the current upstream/debian code :/
<braunr> (the deadlock occurs when receiving data fast with sftp)
<braunr> either in netdde or pfinet

IRC, freenode, #hurd, 2013-02-28

<braunr> (which needs the same kinds of fixes that libpthread got)
<braunr> actually i'm not sure why he didn't simply reuse the pthread
  functions :/
<youpi> which kind of fixes?
<youpi> cancellation?
<braunr> timeouts
<braunr> cancellation too but that's less an issue
<youpi> I'm not sure it really needs timeout work
<youpi> on what RPC?
<youpi> pfinet is just using the mach interface
<braunr> i don't know but it clearly copies some of the previous pthread
  code from pthread_cond_timedwait
<braunr> see libddekit/thread.c:_sem_timedwait_internal
<youpi> I recognize the comment indeed :)
<youpi> I guess he thought he might need some particular semantic that
  libpthread may not provide
<braunr> also, now that i think about it, he couldn't have used libpthread,
  could he ?
<braunr> and there was no condition_timedwait in cthreads
<braunr> there is a deadlock in netdde
<braunr> it occurs sometimes, at high network speeds
<braunr> (well high, 4 MiB/s or more)

IRC, freenode, #hurd, 2013-11-20

<braunr> for example, netdde needs more reviewing and polishing
<braunr> it is known to deadlock sometimes
<teythoon> what deadlocks ?
<braunr> i'm not sure
<teythoon> ah, netdde
<teythoon> right
<braunr> yes
<teythoon> I'm seeing that to on one of my vms
<teythoon> nasty one
<braunr> i know something is wrong with the condition_wait_timeout function
  for example
<teythoon> breaks sysvinit shutdown
<braunr> because it was taken without modification from libpthread
<braunr> it might be that, or something else
<teythoon> well, dhclient hangs releasing the lease
<braunr> that's still on my todo list
<teythoon> so I'm pretty sure it's related
<braunr> hm
<braunr> maybe
<braunr> :/

IRC, freenode, #hurd, 2014-02-11

<braunr> teythoon: looks like a netdde/pfinet freeze/deadlock
<braunr> yes a netdde deadlock
<braunr> i really have to fix that too one day :(
<teythoon> hehe :)
<braunr> the netdde locking privimites are copies of the "old" pthread
  ones, instead of reusing pthread
<braunr> primitives*

IRC, freenode, #hurd, 2014-03-08

<gg0> what to do if network freezes?
<teythoon> gg0: depends on what caused the freeze
<teythoon> gg0: you could try to kill the netdde process
<gg0> it's just apt-get'ing, download phase
<braunr> yess kill netdde
<braunr> there are known deadlocks in netdde

IRC, freenode, #hurd, 2012-08-18

<braunr> hm looks like if netdde crashes, the kernel doesn't handle it
  cleanly, and we can't attach another netdde instance


DDE for Filesystems

IRC, freenode, #hurd, 2012-10-07

* pinotree wonders whether the dde layer could aldo theorically support
    also file systems
<antrik> pinotree: yeah, I also brought up the idea of creating a DDE
  extension or DDE-like wrapper for Linux filesystems a while back... don't
  know enough about it though to decide whether it's doable
<antrik> OTOH, I'm not sure it would be worthwhile. we still should
  probably have a native (not GPLv2-only) implementation for the main FS at
  least; so the wrapper would only be for accessing external

IRC, freenode, #hurd, 2013-12-03

<gg0> how about porting linux block device layer via dde as mcsim wanted to
  do? then all linux filesystems could be brought in, right?
<braunr> gg0: that should be done, but we need to correctly deal with
  multiple pci devices in userspace and arbitration
<kilobug> wouldn't adding support to passive translator into Linux
  filesystems be quite some work ? IIRC ext2fs needs a special "owner =
  hurd" mode to handle them