Features Download
From: Slava Pestov <slava-ZiOttN0IXrP2JYTphGNI2Q <at> public.gmane.org>
Subject: Re: Thread safe factor VM
Newsgroups: gmane.comp.lang.factor.general
Date: Friday 24th July 2009 21:39:26 UTC (over 7 years ago)
Hi Phil,

This looks like a good start. Full native threading won't happen until
after 1.0, but we can do a series of half-assed steps in that
direction in the meantime.

As we discussed in the channel, a potentially cleaner approach would
be to put all global VM state in a struct (or a series of structs)
instead of making them thread-local variables. Then you can either
pass this struct around, or store it in a single thread-local

Another change that would be pretty easy is to move the co-operative
thread scheduler into the VM. This will speed up thread switching,
since instead of using continuations (which copy stacks) it will just
flip the stack pointers over to the new thread.

Finally, it would be easy to add a 'VM lock' which would allow the
possibility of spawning a Factor co-operative thread in a new native
thread. FFI calls would release the lock. This wouldn't enable
multiprocessing but it would allow non-blocking FFI calls to take
place, as well as callbacks from threads other than the primary
thread, and eliminate the need for the VM's stdin thread hack. Of
course by default, the in-thread combinator and similar would still
spawn a co-operative lightweight thread; you'd only use the native
threads for FFI calls and such.

To make your multiple-VM-threads feature more usable, you could work
on setting up some communication channels between multiple VMs in a
single process. Then you can shoehorn concurrency.distributed to use
this facility, allowing semi-transparent passing of Factor data
between threads.

So having the option of multiple VMs per process with full
multiprocessing, as well as non-blocking FFI calls, and faster
co-operative thread scheduling, would make concurrency a bit more
usable than it is now. Of course real concurrency with multiple native
threads and one shared VM would be ideal but that won't happen until
after 1.0.

As for 1.0 I really do want to release it soon as possible. Basically
my plan is that after I finish with some compiler changes, I want to
make another formal release; let's say 0.92; and from that point on,
pretty much focus just on bug fixing and documentation.

If people help out, some of the above threading-related changes could
make it in, too.


On Fri, Jul 24, 2009 at 3:08 PM, Phil
Dawes wrote:
> Hi Slava, Hello everyone,
> Here's my attempt at making Factor reentrant, which actually turned out
> to be a lot easier than I thought. (that's more a testament to how clean
> Slava's vm code is than skill on my part)
> http://github.com/phildawes/factor/tree/reentrantvm
> Caveat: I've only tried this on ubuntu 9.04 64bit as I don't have a
> windows or mac box, sorry. I'd appreciate it if people with those
> platforms could give it a go.
> I made the following modifications to the vm:
> - Modified all global variables so that they're thread local
> - In code_heap.cpp I moved initialisation of the code heap forwarding
> map to the init fn (rather than load time)
> - In os-unix.cpp I hacked the nonblockingio stdin_loop thread to work in
> an MT environment
> - In factor.cpp I added a 'start_standalone_factor_in_new_thread'
> function to spawn a new vm
> I think that's it. I haven't tested this heavily so there may be stuff
> I've missed, but it looks like it works ok. If you want to try it I've
> put a scratch vocab called 'osthreadhack' in resource:work:
> ---- osthreadhack.factor ----
> USING: alien.syntax io.encodings.utf8 kernel sequences unix.utilities ;
> IN: osthreadhack
> FUNCTION: void start_standalone_factor_in_new_thread ( int argc, char**
> argv ) ;
> : start-vm-in-os-thread ( argv -- )
>     [ length ] [ utf8 strings>alien ] bi
> start_standalone_factor_in_new_thread ;
> : start-tetris-in-os-thread ( -- )
>     { "./factor" "-run=tetris" } start-vm-in-os-thread ;
> ----
> So to test you can fire up the factor gui or listener and type:
>        USE: osthreadhack
>        4 [ start-tetris-in-os-thread ] times
> and four tetris windows pop up. Sweet. For an added bonus if you don't
> have tetris in your your image you can witness all your cpu cores
> melting as 4 factor vms compile stuff in parallel.
> Obviously this is just a bit of a hack at the moment so I don't expect
> this to go into trunk, but I'd be interested in knowing if you're open
> to the idea of getting some OS-thread features in factor before 1.0?
> Cheers,
> Phil
> _______________________________________________
> Factor-talk mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/factor-talk

CD: 3ms