Because of the relatively poor state of multithreading in Nim and the reliance on external libraries like Arraymancer for heavy numerical workloads (also the performance issues with boxed values due to `ref object` everywhere), I started writing a language from scratch, with built-in support for concurrency via parallel blocks (without macros) and a C backend, similar to Nim.
GC is optional and the stdlib will work with or without the GC.
Interesing idea. I am wondering what are the use cases on top of your head? I am asking because in my understanding people who care concurrency and parallelism are often those who care performance.
Like I said, the use case is heavy numerical workloads with, e.g. dataframes, in a context where the data is too big for something like python to handle. Using Nim for this is quite difficult too due to value unboxing overhead. It is easier to optimize for things like cache locality and avoid unnecessary allocations using this tool.
I wouldn't reply except you mentioned this unboxing twice and I think people might get the wrong idea. I almost never use ref T/ref object in Nim. I find if you annotate functions at the C level gcc can even autovectorize a lot, like all four central moments: https://github.com/c-blake/adix/blob/3bee09a24313f8d92c185c9... - the relevance being that redoing your own BLAS level 1/2 stuff is really not so bad if the backend is autovectorizing it for you, and full SVD/matrix multiplying/factorizing can be a lot of work. Anyway, as per the link, the Nim itself is just straight, idiomatic Nim.
Parallel/threads is some whole other can of worms, of course. It is unfortunate that the stdlib is weak, both here and for numerics, and for other things, and that people are as dependency allergic as in C culture.
Anyway, "easier to optimize" is often subjective, and I don't mean to discourage you.
The goal is to not need to write C just to get performant code, especially for things like concurrency and numerics. While Nim can be fast without `ref object`, and you can guide the compiler to autovectorize in some cases, it often requires deep knowledge of both the compiler and backend behaviour. That’s not a good developer experience for many users.
Multithreading in Nim is bad to say the least and has been for a while. The standard library option for multithreading is deprecated, and most alternatives like weave are either unmaintained or insanely limited (taskpools, malebolgia, etc.). There's no straightforward, idiomatic way to write data-parallel or task-parallel code in Nim today.
The idea of the project is to make shared-memory parallelism as simple as writing a `parallel:` block, without macros or unsafe hacks, and with synchronization handled automatically by the compiler.
Course, performance can be dragged out of Nim with effort, but there's a need for a language where fast, concurrent, GC-optional code is the default, not something one has to wrestle into existence.
FWIW, I almost always have to look at the generated assembly (never mind C) to even verify things are vectorized. I hope you realize the promise of a PLang where that doesn't need checking or some kind of "autovec-miss" compiler flags or so on and wish you all the best.
Repo: https://github.com/navid-m/scar
Because of the relatively poor state of multithreading in Nim and the reliance on external libraries like Arraymancer for heavy numerical workloads (also the performance issues with boxed values due to `ref object` everywhere), I started writing a language from scratch, with built-in support for concurrency via parallel blocks (without macros) and a C backend, similar to Nim.
GC is optional and the stdlib will work with or without the GC.
Example:
The idea is to make things like accessing shared memory concurrently a trivial process by automating the generation of thread synchronization code.Also there are parallel fors, like so:
It is not ready for use at all currently, though will likely see further development until it is.Compiler implemented in Go, originally with Participle, recursive-descent approach. All examples in the examples directory compile.
Interesing idea. I am wondering what are the use cases on top of your head? I am asking because in my understanding people who care concurrency and parallelism are often those who care performance.
Like I said, the use case is heavy numerical workloads with, e.g. dataframes, in a context where the data is too big for something like python to handle. Using Nim for this is quite difficult too due to value unboxing overhead. It is easier to optimize for things like cache locality and avoid unnecessary allocations using this tool.
I wouldn't reply except you mentioned this unboxing twice and I think people might get the wrong idea. I almost never use ref T/ref object in Nim. I find if you annotate functions at the C level gcc can even autovectorize a lot, like all four central moments: https://github.com/c-blake/adix/blob/3bee09a24313f8d92c185c9... - the relevance being that redoing your own BLAS level 1/2 stuff is really not so bad if the backend is autovectorizing it for you, and full SVD/matrix multiplying/factorizing can be a lot of work. Anyway, as per the link, the Nim itself is just straight, idiomatic Nim.
Parallel/threads is some whole other can of worms, of course. It is unfortunate that the stdlib is weak, both here and for numerics, and for other things, and that people are as dependency allergic as in C culture.
Anyway, "easier to optimize" is often subjective, and I don't mean to discourage you.
The goal is to not need to write C just to get performant code, especially for things like concurrency and numerics. While Nim can be fast without `ref object`, and you can guide the compiler to autovectorize in some cases, it often requires deep knowledge of both the compiler and backend behaviour. That’s not a good developer experience for many users.
Multithreading in Nim is bad to say the least and has been for a while. The standard library option for multithreading is deprecated, and most alternatives like weave are either unmaintained or insanely limited (taskpools, malebolgia, etc.). There's no straightforward, idiomatic way to write data-parallel or task-parallel code in Nim today.
The idea of the project is to make shared-memory parallelism as simple as writing a `parallel:` block, without macros or unsafe hacks, and with synchronization handled automatically by the compiler.
Course, performance can be dragged out of Nim with effort, but there's a need for a language where fast, concurrent, GC-optional code is the default, not something one has to wrestle into existence.
FWIW, I almost always have to look at the generated assembly (never mind C) to even verify things are vectorized. I hope you realize the promise of a PLang where that doesn't need checking or some kind of "autovec-miss" compiler flags or so on and wish you all the best.
I'm curious if you have looked at Chapel: https://chapel-lang.org/
If not, you might find some thing there inspiring.
That's a minor detail but C's "%d" isn't something to copy in a new language, python's format/template string is much better.