dip-1.markdown
211 lines
| 7.5 KiB
| text/x-markdown
|
MarkdownLexer
/ docs / dip-1.markdown
|
r1189 | # 0 Title # | ||
DIP-1 Common protocol description | ||||
# 1 Intro # | ||||
This document describes the Data Interchange Protocol (DIP), designed to | ||||
exchange filtered data that can be stored as a graph structure between | ||||
network nodes. | ||||
# 2 Purpose # | ||||
This protocol will be used to share the models (originally imageboard posts) | ||||
across multiple servers. The main differnce of this protocol is that the node | ||||
can specify what models it wants to get and from whom. The nodes can get | ||||
models from a specific server, or from all except some specific servers. Also | ||||
the models can be filtered by timestamps or tags. | ||||
# 3 Protocol description # | ||||
The node requests other node's changes list since some time (since epoch if | ||||
this is the start). The other node sends a list of post ids or posts in the | ||||
XML format. | ||||
Protocol version is the version of the sync api. Model version is the version | ||||
of data models. If at least one of them is different, the sync cannot be | ||||
performed. | ||||
The node signs the data with its keys. The receiving node saves the key at the | ||||
first sync and checks it every time. If the key has changed, the info won't be | ||||
saved from the node (or the node id must be changed). A model can be signed | ||||
with several keys but at least one of them must be the same as in the global | ||||
ID to verify the sender. | ||||
Each node can have several keys. Nodes can have shared keys to serve as a pool | ||||
(several nodes with the same key). | ||||
Each post has an ID in the unique format: key-type::key::local-id | ||||
All requests pass a request type, protocol and model versions, and a list of | ||||
optional arguments used for filtering. | ||||
Each request has its own version. Version consists of 2 numbers: first is | ||||
incompatible version (1.3 and 2.0 are not compatible and must not be in sync) | ||||
and the second one is minor and compatible (for example, new optional field | ||||
is added which will be igroned by those who don't support it yet). | ||||
Post edits and reflinks are not saved to the sync model. The replied post ID | ||||
can be got from the post text, and reflinks can be computed when loading | ||||
posts. The edit time is not saved because a foreign post can be 'edited' (new | ||||
replies are added) but the signature must not change (so we can't update the | ||||
content). The inner posts can be edited, and the signature will change then | ||||
but the local-id won't, so the other node can detect that and replace the post | ||||
instead of adding a new one. | ||||
## 3.1 Requests ## | ||||
There is no constraint on how the server should calculate the request. The | ||||
server can return any information by any filter and the requesting node is | ||||
responsible for validating it. | ||||
The server is required to return the status of request. See 3.2 for details. | ||||
|
r1566 | ### 3.1.1 list ### | ||
|
r1189 | |||
|
r1566 | "list" request gets the desired model id list by the given filter (e.g. thread, tags, | ||
|
r1189 | author) | ||
Sample request is as follows: | ||||
|
r1237 | <?xml version="1.1" encoding="UTF-8" ?> | ||
|
r1566 | <request version="1.0" type="list"> | ||
|
r1237 | <model version="1.0" name="post"> | ||
<timestamp_from>0</timestamp_from> | ||||
<timestamp_to>0</timestamp_to> | ||||
<tags> | ||||
<tag>tag1</tag> | ||||
</tags> | ||||
<sender> | ||||
<allow> | ||||
<key>abcehy3h9t</key> | ||||
<key>ehoehyoe</key> | ||||
</allow> | ||||
<!-- There can be only allow block (all other are denied) or deny block (all other are allowed) --> | ||||
</sender> | ||||
</model> | ||||
</request> | ||||
|
r1189 | |||
Under the <model> tag there are filters. Filters for the "post" model can | ||||
be found in DIP-2. | ||||
Sample response: | ||||
<?xml version="1.1" encoding="UTF-8" ?> | ||||
<response> | ||||
<status>success</status> | ||||
<models> | ||||
|
r1576 | <model> | ||
<id key="id1" type="ecdsa" local-id="1"> | ||||
<version>1</version> | ||||
</model> | ||||
<model> | ||||
<id key="id1" type="ecdsa" local-id="2" /> | ||||
</model> | ||||
<model> | ||||
<id key="id2" type="ecdsa" local-id="1" /> | ||||
<some-valuable-attr>value</some-valuable-attr> | ||||
</model> | ||||
|
r1189 | </models> | ||
</response> | ||||
### 3.1.2 get ### | ||||
"get" gets models by id list. | ||||
Sample request: | ||||
<?xml version="1.1" encoding="UTF-8" ?> | ||||
<request version="1.0" type="get"> | ||||
<model version="1.0" name="post"> | ||||
<id key="id1" type="ecdsa" local-id="1" /> | ||||
<id key="id1" type="ecdsa" local-id="2" /> | ||||
</model> | ||||
</request> | ||||
Id consists of a key, key type and local id. This key is used for signing and | ||||
validating of data in the model content. | ||||
Sample response: | ||||
<?xml version="1.1" encoding="UTF-8" ?> | ||||
<response> | ||||
<!-- | ||||
Valid statuses are 'success' and 'error'. | ||||
--> | ||||
<status>success</status> | ||||
<models> | ||||
<model name="post"> | ||||
<!-- | ||||
Content tag is the data that is signed by signatures and must | ||||
not be changed for the post from other node. | ||||
--> | ||||
<content> | ||||
<id key="id1" type="ecdsa" local-id="1" /> | ||||
<title>13</title> | ||||
<text>Thirteen</text> | ||||
<thread><id key="id1" type="ecdsa" local-id="2" /></thread> | ||||
<pub-time>12</pub-time> | ||||
<!-- | ||||
Images are saved as attachments and included in the | ||||
signature. | ||||
--> | ||||
<attachments> | ||||
|
r1561 | <attachment mimetype="image/png" id-type="md5">TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5I</attachment> | ||
|
r1800 | <attachment id-type="url">http://example.com/</attachment> | ||
|
r1189 | </attachments> | ||
</content> | ||||
<!-- | ||||
There can be several signatures for one model. At least one | ||||
signature must be made with the key used in global ID. | ||||
--> | ||||
<signatures> | ||||
<signature key="id1" type="ecdsa" value="dhefhtreh" /> | ||||
<signature key="id45" type="ecdsa" value="dsgfgdhefhtreh" /> | ||||
</signatures> | ||||
|
r1505 | <attachment-refs> | ||
<attachment-ref ref="TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0" | ||||
url="/media/images/12345.png" /> | ||||
</attachment-refs> | ||||
|
r1189 | </model> | ||
<model name="post"> | ||||
<content> | ||||
<id key="id1" type="ecdsa" local-id="id2" /> | ||||
<title>13</title> | ||||
<text>Thirteen</text> | ||||
<pub-time>12</pub-time> | ||||
<edit-time>13</edit-time> | ||||
<tags> | ||||
<tag>tag1</tag> | ||||
</tags> | ||||
</content> | ||||
<signatures> | ||||
<signature key="id2" type="ecdsa" value="dehdfh" /> | ||||
</signatures> | ||||
</model> | ||||
</models> | ||||
</response> | ||||
### 3.1.3 put ### | ||||
"put" gives a model to the given node (you have no guarantee the node takes | ||||
it, consider you are just advising the node to take your post). This request | ||||
type is useful in pool where all the nodes try to duplicate all of their data | ||||
across the pool. | ||||
## 3.2 Responses ## | ||||
### 3.2.1 "not supported" ### | ||||
If the request if completely not supported, a "not supported" status will be | ||||
returned. | ||||
### 3.2.2 "success" ### | ||||
"success" status means the request was processed and the result is returned. | ||||
### 3.2.3 "error" ### | ||||
If the server knows for sure that the operation cannot be processed, it sends | ||||
the "error" status. Additional tags describing the error may be <description> | ||||
and <stack>. | ||||