7K views · May 17, 2018 · 26:34 · 4 HN recommenders · 5 mentions · discussed 2018–2019
Kiran Pamnany's Cambridge Julia Meetup talk on Julia's shared memory parallelism and its new task scheduler. It argues that fork-join and work-stealing models nest poorly and presents depth-first scheduling for composable parallelism.
Why Hacker News points people here: Recommended in Julia threading threads as an insightful, simple explanation of the new task scheduler.
“I found his talk to be an insightful overview of this work”HN comment, 2018
“I found his talk to be an insightful overview of this work”
Insightful overview of Julia's parallel task runtime work
This work has been a monumental collaborative effort and I think the eventual outcome will be pretty spectacular. I gave a talk at an MIT Julia meetup a few months ago that the author of this PR also spoke at; I found his talk to be an insightful overview of this work: this video
“which explains the new features of the task scheduler in a simple fashion.”
Explains Julia's new task scheduler simply
checkout this video which explains the new features of the task scheduler in a simple fashion. Basically Julia is employing a new algorithm to schedule tasks, where traditional methods would get stuck/blocked. Essentially, if you run a job with parallel tasks and one of the threads finishes, it's extremely difficult to schedulate the free thread, since the data may still be used by the other threads that are busy; thus you need to have clever way to determine how to allocate computational resources. Julia is implementing results of a new paper on how to optimally do this.
“See this talk by Kiran Pamnany for more details”
Depth-first scheduling for composable multithreaded parallelism
Consider a machine with N cores running a program which spawns N parallel tasks, each of which in turn spawns N parallel tasks. Which subtasks do the cores work on initially? If it's the first subtask of each top-level task, that's breadth-first scheduling. If it's all N subtasks of the first top-level task, that's depth-first scheduling. The latter is better for high-performance computing because it tends to be the lower level libraries that are really carefully tuned to make optimal use of the machine (think BLAS/FTT/parallel sum/prefix sum). Previously it was unknown how to implement this kind of scheduling efficiently. See this talk by Kiran Pamnany for more details: this video
“watching for a talk about some unique stuff that's being worked on with Julia's multi-threading”
Talk on unique Julia multi-threading work
By far the most interesting part of this release is the new multi-threading features, I recommend reading link for an overview, and watching this video for a talk about some unique stuff that's being worked on with Julia's multi-threading (though the depth-first scheduling is not in 1.3 afaik).
Speaker described serialized inner parallel loops problem
Does the Rust Rayon library have the issue the speaker was describing, where parallel inner loops mostly get executed serially once an outer loop is parallelized? (I assume it does, from its Cilk heritage?) Are there workarounds for when that is a problem? Edit: Dumb mistake on my part. I was looking at the video linked in the top comment ( this video ) and then I came back to this thread and thought it was the topic.
