<braunr> i guess it wouldn't be too hard to have a special mach kernel for 64 bits processors, but 32 bits userland only <youpi> well, it means tinkering with mig <braunr> like old sparc systems :p <youpi> to build the 32bit interface, not the 64bit one <braunr> ah yes <braunr> hm <braunr> i'm not sure <braunr> mig would assume a 32 bits kernel, like now <youpi> and you'll have all kinds of discrepancies in vm_size_t & such <braunr> yes <braunr> the 64 bits type should be completely internal <braunr> types* <braunr> but it would be far less work than changing all the userspace bits for 64 bit (ofc we'll do that some day but in the meanwhile ..) <youpi> yes <youpi> and it'd boost userland addrespace to 4GiB <braunr> yes <youpi> leaving time for a 64bit userland :)
<braunr> also, what's the best way to deal with types such as <braunr> type cache_info_t = struct of integer_t; <braunr> whereas cache_info_t contains longs, which are obviously not integer-wide on 64-bits processors <braunr> ? <youpi> you mean, to port mach to 64bit? <braunr> no, to make the RPC declaration portable <braunr> just in case :) <youpi> refine integer_t into something more precise <youpi> such as size_t, off_t, etc. <braunr> i can't use a single line then <braunr> struct cache_info contains ints, vm_size_t, longs <braunr> should i just use the maximum size it can get ? <braunr> or declare two sizes depending on the word size ? <youpi> well, I'd say three <braunr> youpi: three ? <youpi> the ints, the vm_size_ts, and the longs <braunr> youpi: i don't get it <braunr> youpi: how would i write it in mig language ? <youpi> I don't know the mig language <braunr> me neither :) <youpi> but I'd say don't lie <braunr> i just see struct of smething <braunr> the original zone_info struct includes both integer_t and vm_size_t, and declares it as <braunr> type zone_info_t = struct of integer_t; <braunr> in its mig defs file <braunr> i don't have a good example to reuse <youpi> which is lying <braunr> yes <braunr> which is why i was wondering if mach architects themselves actually solved that problem :) <braunr> "There is no way to specify the fields of a <braunr> C structure to MIG. The size and type-desc are just used to give the size of <braunr> the structure. <braunr> " <braunr> well, this sucks :/ <braunr> well, i'll do what the rest of the code seems to do, and let it rot until a viable solution is available <antrik> braunr: we discussed the problem of expressing structs with MIG in the libburn thread <antrik> (which I still need to follow up on... [sigh])
In context of memory management.
<tschwinge> Or with a 64-bit one? ;-P <braunr> tschwinge: i think we all had that idea in mind :) <pinotree> tschwinge: patches welcome :P <youpi> tschwinge: sure, please help us settle down with the mig stuff <youpi> what was blocking me was just deciding how to do it <braunr> hum, what's blocking x86_64, except time to work on it ? <youpi> deciding the mig types & such things <youpi> i.e. the RPC ABI <braunr> ok <braunr> easy answer: keep it the same <youpi> sorry, let me rephrase <youpi> decide what ABI is supposed to be on a 64bit system, so as to know which way to rewrite the types of the kernel MIG part to support 64/32 conversion <braunr> can't this be done in two steps ? <youpi> well, it'd mean revamping the whole kernel twice <youpi> as the types at stake are referenced in the whole RPC code <braunr> the first step i imagine would simply imply having an x86_64 kernel for 32-bits userspace, without any type change (unless restricting to 32-bits when a type is automatically enlarged on 64-bits) <youpi> it's not so simple <youpi> the RPC code is tricky <youpi> and there are alignments things that RPC code uses <youpi> which become different when build with a 64bit compiler <pinotree> there are also things like int[N] for io_stat_struct and so on <braunr> i see <youpi> making the code wrong for 32 <youpi> thus having to change the types <youpi> pinotree: yes <pinotree> (doesn't mig support structs, or it is too clumsy to be used in practice?) <braunr> pinotree: what's the problem with that (i explcitely said changing int to e.g. int32_t) <youpi> that won't fly for some of the calls <youpi> e.g. getting a thread state <braunr> pinotree: no it doesn't support struct <pinotree> braunr: that some types in struct stat are long, for instance <braunr> pinotree: same thing with longs <braunr> youpi: why wouldn't it ? <youpi> that wouldn't work on a 64bit system <youpi> so we can't make it int32_t in the interface definition <braunr> i understand the alignment issues and that the mig code adjusts the generated code, but not the content of what is transfered <braunr> well of course <braunr> i'm talking about the first step here <braunr> which targets a 32-bits userspace only <youpi> ok, so we agree <youpi> the second step would have to revamp the whole RPC code again <braunr> i imagine the first to be less costly <braunr> well, actually no <braunr> you're right, the mig stuff would be easy on the application side, but more complicated on the kernel side, since it would really mean dealing with 64-bits values there <braunr> (unless we keep a 3/1 split instead of giving the full 4g to applications)
See also memory management.
<youpi> (I don't see what that changes) <braunr> if the kernel still runs with 32-bits addresses, everything it recevies from or sends through mig can be stored with the user side 32-bits types <youpi> err, ok, but what's the point of the 64bit kernel then ? :) <braunr> and it simply uses 64-bits addresses to deal with physical memory <youpi> ok <youpi> that could even be a 3.5/0.5 split then <braunr> but the memory model forces us to run either at the low 2g or the highest ones <youpi> but linux has 3/1, so we don't need that <braunr> otherwise we need an mcmodel=medium <braunr> we could do with mcmodel=medium though, for a time <braunr> hm actually no, it would require mcmodel=large <braunr> hum, that's stupid, we can make the kernel run at -2g, and use 3g up to the sign extension hole for the kernel map
<azeem> I believe the main issue is redoing the RPCs in 64bit, i.e. the Mach/Hurd interface <braunr> mach has always been 64-bits capable <braunr> the problem is both mach and the hurd <braunr> it's at the system interface (the .defs of the RPCs) <braunr> azeem: ah, actually that's why you also say <braunr> but i consider it to be a hurd problem <braunr> the hurd itself is defined as being a set of interfaces and servers implementing them, i wouldn't exclude the interfaces <braunr> that's what*
<teythoon> is there a nice way to get structured data through mig that I haven't found yet? <teythoon> say an array of string triples <braunr> no <teythoon> :/ <braunr> but you shouldn't need that <teythoon> my use case is getting info about fs translators from init to procfs
<teythoon> should I go for an iterator like interface instead? <braunr> depends <braunr> how many do you need ? <braunr> you could go for a variable sized array too <braunr> have a look at what already exists <teythoon> records, maybe 10-15, depends on many fs translators are running <braunr> a variable sized array is ok if the size isn't too big (and when i say too big, i mean hundreds of MiB) <braunr> an iterator is ok too if there aren't too many items <braunr> you may want to combine both (i think that's what proc does) <braunr> be aware that the maximum size of a message is limited to 512 MiB <teythoon> yeah I saw the array of stuff stuff, but array of string_t does not work, I guess b/c string_t is also an array <teythoon> how would I send an array of variable length strings? <braunr> i'm not sure you can <braunr> or maybe out of line <teythoon> somehow I expected mig to serialize arbitrary data structures, maybe it's to old for that? <teythoon> yeah, I read about uot of line, but that seems overkill <braunr> it is old yes <braunr> and not very user friendly in the end <braunr> let me check <teythoon> we could stuff json into mig... <braunr> see proc_getallpids for example <braunr> we could get rid of low level serialization altogether :p <teythoon> hah, exactly what I was looking at <braunr> (which is what i'll do in x15) <braunr> type pidarray_t = array of pid_t; <teythoon> but that is trivial b/c its array of pid_t <braunr> and always have the server writing guide near you <teythoon> yes <braunr> well, make one big string and an array of lengths :p <teythoon> thought about that and said to myself, there must be a better way that I haven't found yet <braunr> or one big string filled with real null-terminated c strings that you keep parsing until you ate all input bytes <braunr> i'm almost certain there isn't <braunr> type string_t = c_string; /* XXX */ <teythoon> yes <braunr> even that isn't really variable sized <teythoon> you think anyone would object to me putting a json encoder in /hurd/init? it is probably better than me at serializing stuff... <braunr> try with mig anyway <braunr> the less dependencies we have for core stuff, the simpler it is <braunr> but i agree, mig is painful <teythoon> would it be too hacky if I abused the argz functions? they do exactly what I'd need
<teythoon> there is https://code.google.com/p/protobuf-c/ and it has a rpc mechanism and I believe one could plug arbitrary transports easily <braunr> please don't think about it <braunr> we really don't want to add another layer of serialization <braunr> it's better to completely redesign mach ipc anyway <braunr> and there is a project for that :p <teythoon> ive seen x15 <teythoon> just food for thought <braunr> i've studied google protocol buffers <braunr> and fyi, no, it wouldn't be easy to plug arbitrary transports on top of mach <braunr> there is a lot of knowledge about mach ports in mig
<teythoon> but again I face the challenge of serializing a arbitrary sized list of arbitrary sized strings <braunr> yes <teythoon> list of ports is easier ;) but I think its worthwile <teythoon> so what about abusing argz* for this? you think it's too bad a hack? <braunr> no since it's in glibc <teythoon> awesome :) <braunr> but i don't remember the details well and i'm not sure the way you use it is safe <teythoon> yeah, I might have got the details wrong, I hadn't had the chance to test it ;) <braunr> about this dynamic size problem <braunr> a "simple" varying size array should do <braunr> you can easily put all your strings in there <teythoon> seperated by 0? <braunr> yes <teythoon> that's exactly what the argz stuff does <braunr> you'll get the size of the array anyway, and consume it until there is no byte left <braunr> good <braunr> but be careful with this too <braunr> since translators can be run by users, they somtimes can't be trusted <braunr> and even a translator running as root may behave badly <braunr> so careful with parsing <teythoon> noted