Schedulable lightweight virtual processes and concurrent loops P&L: -11.5 (≃ -982 CNY)

An approach to building concurrent software
YAML 项目 产品

Imagine a nested for loop, it is composed of multiple loops. We can simulate concurrency by executing parts of loops and rapidly switch between them.

This is what a kernel scheduler does and I think it's a very interesting part of an operating system. Unfortunately most programs are not written to be multithreaded so computers use very little of resources available to them.


年度 内部收益率: -1.0000
最新 净现值@贴现率=0.1: -11.4888 ħ (-981.23 CNY)
(别通知) 请,登录

(别通知) 请,登录

(别通知) 请,登录

(别通知) 请,登录

(别通知) 请,登录

(别通知) 请,登录

(别通知) (可选) 请,登录

我意识到我的逻辑错误。 线程 0 负责循环迭代 0-8 线程1负责8-16 线程2负责16-24 等等 由于不处理环绕的数学逻辑,我目前正在错误的线程上执行循环迭代。我使用模运算符。 理想情况下,每个股票只在线程上调用其范围。

I realised my bug in my logic. Thread 0 is responsible for loop iterations 0-8 Thread 1 is responsible for 8-16 Thread 2 is responsible for 16-24 And so on I'm currently executing loop iterations on the wrong thread due to a mathematical logic of not handling wraparound. I use the modulo operator. Ideally each ticker is only called on the thread for its range.

(别通知) 请,登录

我有一个问题需要解决。这与并行性无关。

想象一下,我有一个看起来像这样的嵌套循环

``` 对于信中的信件:

对于数字中的数字:

打印(字母编号)

对于符号中的符号:

打印(字母符号) ```

我希望这三个循环同时运行。 一种方法是根据索引选择要运行的函数!并将数字和字母循环合并在一起。 所以我们得到

A1

A2

这种方法的问题在于循环不是分开的。它们是一个已合并的循环。

我想我可以通过使集合成为一个多集合来解决这个问题。 [字母、[数字、符号] 并从每个子列表循环中挑选并将循环作为单独的对象公开。

I have a problem I need to solve. Which isn't related to the parallellism.

Imagine I have a nested loop that looks like this

``` For letter In letters:

For number in numbers:

Print(letter + number)

For symbol in symbols:

Print(letter + symbol) ```

I want these three loops to run concurrently. One approach is to pick the function to run based on the indexes! And merge the numbers and letters together round robin. So we get

A1

A2

The problem with this approach is that the loops aren't separate. They're one loop that has been merged.

I think I can solve this by causing collections to be a multi collection. [Letters, [numbers, symbols] And picking from each sublist round robin and exposing the loops as separate objects.

(别通知) 请,登录

我创建了一个没有 Joinable 循环的多线程版本。但此时我正在休息。 我需要以某种方式将多线程与可连接循环联系起来。 当 Joinable 循环收到 2 个或更多值时,它会继续处理。这允许在管道处理中进行拆分和合并。 我希望加入的进程尽快发生并且是多线程的。 在我的设计中——我还没有在 Java 中实现——每个线程都有一个调度程序,它重新检查可以勾选的内容并勾选它。 问题是在线程上拆分滴答声,如果只有一层代码,这很容易。 您一次只能勾选 8 个批次。在单独的线程中。 我的示例有一个包含 3 个集合的嵌套循环,并拆分为两个任务,然后这两个任务合并为一个任务以打印出结果。 我希望分批执行嵌套循环、单独任务和连接任务。 需要一种将并发循环发送到旧线程的方法。可以有一个称为滴答池的共享集合,由所有线程检查。 当前产品编号与线程相关联。 我选择 ÷ 8 是因为 64 是 8 的倍数。清理每个线程的批次数。

```

虽然(真){

报价池中的 For 循环:

如果 current[loop] < thisThreadN[loop]:

当前[循环] = 当前[循环] 1

NewTickersOrResult = Loop.tick()

如果 NewTickersOrResult.isTickers:

对于 NewTickersOrResult.tickers 中的 newTicker:

 Current[loop] = threadNum × newTicker.size() / threadCount

 thisThreadN[loop] = threadNum × newTicker.size() / threadCount threadCount

Pool.extend(NewTickersOrResult.tickers)

别的:

打印(NewTickersOrResult.result)

}

```

I created a multithreading version of this without the Joinable loops. But at this time I am taking a break. I need to somehow tie the multithreading to the joinable loop. When Joinable loop received 2 or more values, it continues processing. This allows a split and merge in pipeline processing. I want joined processes to occur as soon as they can and to be multithreaded. In my design - that I am yet to implement in Java - each thread has a scheduler that rechecks what can be ticked and ticks it. The problem is splitting ticks over threads, it's easy if there is only one level of tickers. You can just tick batches of 8 at a time. In separate threads. My example has a nested loop of 3 collections and a split into two tasks and then those two tasks join into one task to print out the results. I want the nested loops, separate tasks and the joined tasks to be executed in batches. Kind of need a way to send concurrent loops to old threads. Could have a shared collection called the tick pool which is checked by all threads. The current product number is associated with a thread. I picked ÷ 8 due to the 64 being a multiple of 8. Clean number of batches per thread.

```

While (true) {

For loop in tick pool:

If current[loop] < thisThreadN[loop]:

current[loop] = current[loop] + 1

NewTickersOrResult = Loop.tick()

If NewTickersOrResult.isTickers:

For newTicker in NewTickersOrResult.tickers:

 Current[loop] = threadNum × newTicker.size() / threadCount

 thisThreadN[loop] = threadNum × newTicker.size() / threadCount + threadCount

Pool.extend(NewTickersOrResult.tickers)

Else:

Print(NewTickersOrResult.result)

}

```

(别通知) 请,登录

我正在考虑如何使用多个 CPU 内核的性能。它需要对我们如何编写代码进行彻底的重新思考!

我突然想到,循环可以与我的设计简单地并行化。

N = 0 .. N = 3×3×3

如果您在每个 N 的所有线程上运行 tick 方法,您可以一次运行整个循环。

I am thinking of how to use the performance of multiple CPU cores. It requires a drastic rethinking of how we write code!

It occurred to me that loops could be trivially parallelized with my design.

N = 0 .. N = 3×3×3

If you ran the tick method on all threads with every N, you could run the entire loop in one go.

(别通知) 请,登录

我需要编写一个 Joinable Ticker 来等待来自多个代码的输入,然后再发送输出。 这让我们可以创建拆分和合并的管道。

I need to write a Joinable Ticker that waits for inputs from multiple tickers before sending output along. This lets us create pipelines that split and merge.

(别通知) 请,登录

我要补充一点,每个线程只需要一个 while (true) 循环。 其他一切都可以在一个线程中同时调度。

I shall add that you only need one while (true) loop per thread. Everything else can be concurrently scheduled within a thread.



    :  -- 
    : Mindey
    :  -- 
    

chronological,
(别通知) 请,登录

这就是我称之为虚拟并发的原因。 您需要使用线程来提供 IO 并发 任何永远循环或阻塞的东西都不能组合。因此,我将编写我所有的代码,以尝试从不阻塞和重入。这是我在开发多线程调度程序时学到的重要一课。

阻塞对程序是不可见的。该程序不知道它正在阻塞。您需要知道方法可以阻止以解决它。 我的另一个想法是并行 while do 循环,它通过语法糖将阻塞变为非阻塞。 看起来像这样

A1 = 并行同时 { Buf = socket.recv(1024) } 做 { 并行 for (Socket socket : sockets) { Socket.send(buf) } } 交叉合并 A1

此语法启动多个线程以阻止接收数据。对于每一个,它都会在线程池中启动一个线程来处理它并将其并行发送到每个连接。

我的另一个想法是更改优先执行顺序。该代码中的调度程序每次都以相同的顺序轮询代码。没有理由我们不能比其他人更多地执行循环。

That's why I call it virtual concurrency. You would need to use threads to provide IO concurrency Anything that loops forever or blocks cannot be composed. So I shall write all my code going forward to try never block and be reetrant. This is an important lesson I learned while developing my multithreaded scheduler.

Blocking is invisible to the program. The program isn't aware it is blocking. You need to know that a method can block to work around it. My other idea is a parallel while do loop which changes blocking to be non blocking through syntactic sugar. It looks like this

A1 = parallel while { Buf = socket.recv(1024) } Do { Parallel for (Socket socket : sockets) { Socket.send(buf) } } Crossmerge A1

This syntax spins up multiple threads to block on receiving data. And for each one it spins up a thread in a thread pool to handle it and send it to every connection in parallel.

Another idea I have is to change the priority execution order. That scheduler in that code round robins the tickers in the same order every time. There's no reason why we cannot execute the loops some more than others.



    : Mindey
    :  -- 
    :  -- 
    

chronological,
(别通知) 请,登录

有趣的。你没有使用任何原生支持 concurrency in Python,并且只使用了基本的枚举、索引、赋值。我想,这就是理解所需要的。这具有教育价值。

Interesting. You did not use any native support for concurrency in Python, and used only basic enumeration, indexing, assignment. I guess, that is what it takes to understand. This has educational value.

(别通知) 请,登录