Distributed data storage

If you have many nodes and want to store data - how do you store data?

YAML 问题

This category is to explore the various approaches to distributed data storage. One popular technique is the Blockchain - used by bitcoin, Ethereum and the dfinity internet computer.

There are other techniques available to us too! I think it is unfair on Mindey has to host 0oo.li by himself and take the cost of maintaining it by himself. We should be able to distribute arbitrary data across machines.






投票 (可选) (别通知) (可选)
请,登录

有ipfs。任何人都可以托管 ipfs 节点。不知道重构oo使用ipfs需要多少工作

There's ipfs. Anyone can host ipfs node. Don't know how much work to refactor oo to use ipfs


是的,我正在考虑这个问题,我认为正确的道路是——

  1. 研究数据库模型作为 [此处] (https://0oo.li/invite/0xE34A06A52C1b57CBEe4001f8dF6991BCAffDbE10/) 所描述的最佳本体。 (如果本体优化了合作决策和经济活动,那么寻找扩展它的方法并使节点去中心化它是有意义的。)

  2. 去中心化。

正如我所见,去中心化的“方式”是——寻找替代功能的 dApp,并为所有这些 dApp 提供一个通用的 UI,这将共同相当于 Infinity 的单体应用程序。

另一种方法是从头开始构建一个新的去中心化系统,使用诸如 gun.js 之类的东西,其中每个用户都将同时成为播种者。

//skihappy:有ipfs。

这属于对替换功能的“dApps”的搜索。例如,IPFS 就是这样一个用于文件存储的 dApp。但是,作为应用程序的无穷大需要将关系数据库作为 dApp :) 有没有类似的东西? (包括模式迁移)

Yeah, I'm thinking of this problem, and I think the right path would be --

  1. Researching the database models as an optimal ontology for what is described here. (If the ontology optimizes cooperative decision-making and economic activity, then it makes sense to look for ways to scale it and make nodes decentralize it.)

  2. Decentralize it.

The "how" of decentralization is, as I see -- searching for dApps that replace functionality, and providing a common UI for all those dApps, that would collectively be equivalent to the monolith application that the Infinity is.

Another way would be to build a new decentralized system from ground up, using things like gun.js, where each of users would be seeders at the same time.

// skihappy: There's ipfs.

This falls under the search for "dApps" that replace functionality. For example, IPFS is such a dApp for file storage. However, infinity as application requires the relational database as a dApp :) is there anything like that? (including schema migrations)


关系链接可以是数据存储之上的一层,形成一个图。图比关系数据库更能代表关系数据库。关系链接可以是数据集的一部分。这就是图数据库的工作原理。也许,已经有这样的 ipfs 图解决方案了

Relational links can be a layer above data storage, forming a graph. Graphs are much better to represent relational database then relational database. Relational links can be part of data set. That's how graph db works. Perhaps, there's already an ipfs graph solution like that already


Mindey 人们可以做的最简单的事情是使用转储 URL 备份数据库,我有一个每天运行的脚本来备份数据。不幸的是,我的主计算机已关闭,因此在接下来的 4 个月内我一直没有运行它。

我喜欢像 Neo4j 和 Dgraph 这样的图数据库。我什至写了一个简单的,使用矩阵乘法来做广度优先搜索,比如 Graphblas。

Mindey the simplest thing that people can do to help is to backup the database with the dump URL which I have a script that runs daily to take a backup of the data. My main computer has been switched off unfortunately so I haven't been running it for the next 4 months.

I like Graph databases like Neo4j and Dgraph. I have even written a simple one that uses matrix multiplication to do Breadth first search like Graphblas.


隐藏在这一类中的想法是,获得 P2P 行为的最简单方法是使数据存储层 P2P,其他一切都是传统的。

Hidden in this category is the idea that the easiest way to get P2P behaviour is to make the data storage layer P2P and everything else is traditional.



    : Mindey
    :  -- 
    :  -- 
    

chronological,

// 使数据存储层P2P,其他一切都是传统的

这是完全有道理的——大多数软件系统都是数据库系统的客户。

// make the data storage layer P2P and everything else is traditional

It makes perfect sense -- most of software systems are clients of database systems.



    : chronological
    :  -- 
    :  -- 
    

Mindey,

我刚刚发现项目 hyper Hyper space 这是一个有趣的项目。您拥有空间而不是文件,它们由默克尔有向无环图支持,该图以某种方式无缝合并。假设它使用 CRDT。

他们说你可以用它构建几乎任何应用程序,论坛、聊天、网站等

I just found the project hyper Hyper space it's an interesting project. Rather than files you have spaces and they are backed by a merkel directed acyclic graph that somehow merges seamlessly. Assuming it uses CRDTs.

They say you can build almost any app with it, forum, chat, website etc



    :  -- 
    : Mindey
    :  -- 
    

chronological,

Hyperhyperspace 的问题是我永远不会使用它,因为它接管了你用来存储数据的对象来实现它的功能。所以它需要一个糟糕的方法。

要在内存中创建简单的对象图,您必须使用称为 MutableReference 的类将对象链接在一起。对我来说感觉不对。

The problem with Hyperhyperspace is that I would never use it because it takes over the objects you use to store data to implement its functionality. So it takes a bad approach.

To create simple object graphs in memory you have to use a class called a MutableReference to link objects together. Feels wrong to me.



    :  -- 
    : Mindey
    :  -- 
    

chronological,

当超空间数据层与其他对等点同步空间内容时,它需要验证接收到的更改是否尊重应用程序的数据假设。是的,任何可变的东西都需要包装在一种知道如何将更改编组为类似 CRDT 的操作的类型中。

该项目是实验性的,也许我们需要更多的迭代才能弄清楚如何消除这种耦合。该模型已经对以前的模型进行了相当大的简化:)

When the Hyper Hyper Space data layer synchronizes a space contents with other peers, it needs to validate that the received changes respect the data assumptions of the application. And yeah, anything that's mutable needs to be wrapped in a type that knows how to marshal the changes as CRDT-like operations.

The project is experimental, maybe we need more iterations before we figure out how to remove that coupling. The model is already a considerable simplification over previous ones :)


sbazerque,

关于 Hyper Hyper Space 的一个小说明:您不需要使用 MutableReference 将对象链接在一起,只有当您希望以后能够用另一个对象替换您链接的对象时才需要它!

One small clarification about Hyper Hyper Space: you don't need to use MutableReference to link objects together, you need it only if you want to be able to replace the object you linked with another one later!



    :  -- 
    : Mindey
    :  -- 
    

sbazerque,