Show More
@@ -1,205 +1,203 b'' | |||||
1 | # 0 Title # |
|
1 | # 0 Title # | |
2 |
|
2 | |||
3 | DIP-1 Common protocol description |
|
3 | DIP-1 Common protocol description | |
4 |
|
4 | |||
5 | # 1 Intro # |
|
5 | # 1 Intro # | |
6 |
|
6 | |||
7 | This document describes the Data Interchange Protocol (DIP), designed to |
|
7 | This document describes the Data Interchange Protocol (DIP), designed to | |
8 | exchange filtered data that can be stored as a graph structure between |
|
8 | exchange filtered data that can be stored as a graph structure between | |
9 | network nodes. |
|
9 | network nodes. | |
10 |
|
10 | |||
11 | # 2 Purpose # |
|
11 | # 2 Purpose # | |
12 |
|
12 | |||
13 | This protocol will be used to share the models (originally imageboard posts) |
|
13 | This protocol will be used to share the models (originally imageboard posts) | |
14 | across multiple servers. The main differnce of this protocol is that the node |
|
14 | across multiple servers. The main differnce of this protocol is that the node | |
15 | can specify what models it wants to get and from whom. The nodes can get |
|
15 | can specify what models it wants to get and from whom. The nodes can get | |
16 | models from a specific server, or from all except some specific servers. Also |
|
16 | models from a specific server, or from all except some specific servers. Also | |
17 | the models can be filtered by timestamps or tags. |
|
17 | the models can be filtered by timestamps or tags. | |
18 |
|
18 | |||
19 | # 3 Protocol description # |
|
19 | # 3 Protocol description # | |
20 |
|
20 | |||
21 | The node requests other node's changes list since some time (since epoch if |
|
21 | The node requests other node's changes list since some time (since epoch if | |
22 | this is the start). The other node sends a list of post ids or posts in the |
|
22 | this is the start). The other node sends a list of post ids or posts in the | |
23 | XML format. |
|
23 | XML format. | |
24 |
|
24 | |||
25 | Protocol version is the version of the sync api. Model version is the version |
|
25 | Protocol version is the version of the sync api. Model version is the version | |
26 | of data models. If at least one of them is different, the sync cannot be |
|
26 | of data models. If at least one of them is different, the sync cannot be | |
27 | performed. |
|
27 | performed. | |
28 |
|
28 | |||
29 | The node signs the data with its keys. The receiving node saves the key at the |
|
29 | The node signs the data with its keys. The receiving node saves the key at the | |
30 | first sync and checks it every time. If the key has changed, the info won't be |
|
30 | first sync and checks it every time. If the key has changed, the info won't be | |
31 | saved from the node (or the node id must be changed). A model can be signed |
|
31 | saved from the node (or the node id must be changed). A model can be signed | |
32 | with several keys but at least one of them must be the same as in the global |
|
32 | with several keys but at least one of them must be the same as in the global | |
33 | ID to verify the sender. |
|
33 | ID to verify the sender. | |
34 |
|
34 | |||
35 | Each node can have several keys. Nodes can have shared keys to serve as a pool |
|
35 | Each node can have several keys. Nodes can have shared keys to serve as a pool | |
36 | (several nodes with the same key). |
|
36 | (several nodes with the same key). | |
37 |
|
37 | |||
38 | Each post has an ID in the unique format: key-type::key::local-id |
|
38 | Each post has an ID in the unique format: key-type::key::local-id | |
39 |
|
39 | |||
40 | All requests pass a request type, protocol and model versions, and a list of |
|
40 | All requests pass a request type, protocol and model versions, and a list of | |
41 | optional arguments used for filtering. |
|
41 | optional arguments used for filtering. | |
42 |
|
42 | |||
43 | Each request has its own version. Version consists of 2 numbers: first is |
|
43 | Each request has its own version. Version consists of 2 numbers: first is | |
44 | incompatible version (1.3 and 2.0 are not compatible and must not be in sync) |
|
44 | incompatible version (1.3 and 2.0 are not compatible and must not be in sync) | |
45 | and the second one is minor and compatible (for example, new optional field |
|
45 | and the second one is minor and compatible (for example, new optional field | |
46 | is added which will be igroned by those who don't support it yet). |
|
46 | is added which will be igroned by those who don't support it yet). | |
47 |
|
47 | |||
48 | Post edits and reflinks are not saved to the sync model. The replied post ID |
|
48 | Post edits and reflinks are not saved to the sync model. The replied post ID | |
49 | can be got from the post text, and reflinks can be computed when loading |
|
49 | can be got from the post text, and reflinks can be computed when loading | |
50 | posts. The edit time is not saved because a foreign post can be 'edited' (new |
|
50 | posts. The edit time is not saved because a foreign post can be 'edited' (new | |
51 | replies are added) but the signature must not change (so we can't update the |
|
51 | replies are added) but the signature must not change (so we can't update the | |
52 | content). The inner posts can be edited, and the signature will change then |
|
52 | content). The inner posts can be edited, and the signature will change then | |
53 | but the local-id won't, so the other node can detect that and replace the post |
|
53 | but the local-id won't, so the other node can detect that and replace the post | |
54 | instead of adding a new one. |
|
54 | instead of adding a new one. | |
55 |
|
55 | |||
56 | ## 3.1 Requests ## |
|
56 | ## 3.1 Requests ## | |
57 |
|
57 | |||
58 | There is no constraint on how the server should calculate the request. The |
|
58 | There is no constraint on how the server should calculate the request. The | |
59 | server can return any information by any filter and the requesting node is |
|
59 | server can return any information by any filter and the requesting node is | |
60 | responsible for validating it. |
|
60 | responsible for validating it. | |
61 |
|
61 | |||
62 | The server is required to return the status of request. See 3.2 for details. |
|
62 | The server is required to return the status of request. See 3.2 for details. | |
63 |
|
63 | |||
64 | ### 3.1.1 pull ### |
|
64 | ### 3.1.1 pull ### | |
65 |
|
65 | |||
66 | "pull" request gets the desired model id list by the given filter (e.g. thread, tags, |
|
66 | "pull" request gets the desired model id list by the given filter (e.g. thread, tags, | |
67 | author) |
|
67 | author) | |
68 |
|
68 | |||
69 | Sample request is as follows: |
|
69 | Sample request is as follows: | |
70 |
|
70 | |||
71 | <?xml version="1.1" encoding="UTF-8" ?> |
|
71 | <?xml version="1.1" encoding="UTF-8" ?> | |
72 | <request version="1.0" type="pull"> |
|
72 | <request version="1.0" type="pull"> | |
73 | <model version="1.0" name="post"> |
|
73 | <model version="1.0" name="post"> | |
74 | <timestamp_from>0</timestamp_from> |
|
74 | <timestamp_from>0</timestamp_from> | |
75 | <timestamp_to>0</timestamp_to> |
|
75 | <timestamp_to>0</timestamp_to> | |
76 | <tags> |
|
76 | <tags> | |
77 | <tag>tag1</tag> |
|
77 | <tag>tag1</tag> | |
78 | </tags> |
|
78 | </tags> | |
79 | <sender> |
|
79 | <sender> | |
80 | <allow> |
|
80 | <allow> | |
81 | <key>abcehy3h9t</key> |
|
81 | <key>abcehy3h9t</key> | |
82 | <key>ehoehyoe</key> |
|
82 | <key>ehoehyoe</key> | |
83 | </allow> |
|
83 | </allow> | |
84 | <!-- There can be only allow block (all other are denied) or deny block (all other are allowed) --> |
|
84 | <!-- There can be only allow block (all other are denied) or deny block (all other are allowed) --> | |
85 | </sender> |
|
85 | </sender> | |
86 | </model> |
|
86 | </model> | |
87 | </request> |
|
87 | </request> | |
88 |
|
88 | |||
89 | Under the <model> tag there are filters. Filters for the "post" model can |
|
89 | Under the <model> tag there are filters. Filters for the "post" model can | |
90 | be found in DIP-2. |
|
90 | be found in DIP-2. | |
91 |
|
91 | |||
92 | Sample response: |
|
92 | Sample response: | |
93 |
|
93 | |||
94 | <?xml version="1.1" encoding="UTF-8" ?> |
|
94 | <?xml version="1.1" encoding="UTF-8" ?> | |
95 | <response> |
|
95 | <response> | |
96 | <status>success</status> |
|
96 | <status>success</status> | |
97 | <models> |
|
97 | <models> | |
98 | <id key="id1" type="ecdsa" local-id="1" /> |
|
98 | <id key="id1" type="ecdsa" local-id="1" /> | |
99 | <id key="id1" type="ecdsa" local-id="2" /> |
|
99 | <id key="id1" type="ecdsa" local-id="2" /> | |
100 | <id key="id2" type="ecdsa" local-id="1" /> |
|
100 | <id key="id2" type="ecdsa" local-id="1" /> | |
101 | <id key="id2" type="ecdsa" local-id="5" /> |
|
101 | <id key="id2" type="ecdsa" local-id="5" /> | |
102 | </models> |
|
102 | </models> | |
103 | </response> |
|
103 | </response> | |
104 |
|
104 | |||
105 | ### 3.1.2 get ### |
|
105 | ### 3.1.2 get ### | |
106 |
|
106 | |||
107 | "get" gets models by id list. |
|
107 | "get" gets models by id list. | |
108 |
|
108 | |||
109 | Sample request: |
|
109 | Sample request: | |
110 |
|
110 | |||
111 | <?xml version="1.1" encoding="UTF-8" ?> |
|
111 | <?xml version="1.1" encoding="UTF-8" ?> | |
112 | <request version="1.0" type="get"> |
|
112 | <request version="1.0" type="get"> | |
113 | <model version="1.0" name="post"> |
|
113 | <model version="1.0" name="post"> | |
114 | <id key="id1" type="ecdsa" local-id="1" /> |
|
114 | <id key="id1" type="ecdsa" local-id="1" /> | |
115 | <id key="id1" type="ecdsa" local-id="2" /> |
|
115 | <id key="id1" type="ecdsa" local-id="2" /> | |
116 | </model> |
|
116 | </model> | |
117 | </request> |
|
117 | </request> | |
118 |
|
118 | |||
119 | Id consists of a key, key type and local id. This key is used for signing and |
|
119 | Id consists of a key, key type and local id. This key is used for signing and | |
120 | validating of data in the model content. |
|
120 | validating of data in the model content. | |
121 |
|
121 | |||
122 | Sample response: |
|
122 | Sample response: | |
123 |
|
123 | |||
124 | <?xml version="1.1" encoding="UTF-8" ?> |
|
124 | <?xml version="1.1" encoding="UTF-8" ?> | |
125 | <response> |
|
125 | <response> | |
126 | <!-- |
|
126 | <!-- | |
127 | Valid statuses are 'success' and 'error'. |
|
127 | Valid statuses are 'success' and 'error'. | |
128 | --> |
|
128 | --> | |
129 | <status>success</status> |
|
129 | <status>success</status> | |
130 | <models> |
|
130 | <models> | |
131 | <model name="post"> |
|
131 | <model name="post"> | |
132 | <!-- |
|
132 | <!-- | |
133 | Content tag is the data that is signed by signatures and must |
|
133 | Content tag is the data that is signed by signatures and must | |
134 | not be changed for the post from other node. |
|
134 | not be changed for the post from other node. | |
135 | --> |
|
135 | --> | |
136 | <content> |
|
136 | <content> | |
137 | <id key="id1" type="ecdsa" local-id="1" /> |
|
137 | <id key="id1" type="ecdsa" local-id="1" /> | |
138 | <title>13</title> |
|
138 | <title>13</title> | |
139 | <text>Thirteen</text> |
|
139 | <text>Thirteen</text> | |
140 | <thread><id key="id1" type="ecdsa" local-id="2" /></thread> |
|
140 | <thread><id key="id1" type="ecdsa" local-id="2" /></thread> | |
141 | <pub-time>12</pub-time> |
|
141 | <pub-time>12</pub-time> | |
142 | <!-- |
|
142 | <!-- | |
143 | Images are saved as attachments and included in the |
|
143 | Images are saved as attachments and included in the | |
144 | signature. |
|
144 | signature. | |
145 | --> |
|
145 | --> | |
146 | <attachments> |
|
146 | <attachments> | |
147 |
<attachment mimetype="image/png" |
|
147 | <attachment mimetype="image/png">TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5I</attachment> | |
148 | TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0 |
|
|||
149 | aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1 |
|
|||
150 | c3Qgb2YgdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdodCBpbiB0 |
|
|||
151 | aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdl |
|
|||
152 | LCBleGNlZWRzIHRoZSBzaG9ydCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4= |
|
|||
153 | </attachment> |
|
|||
154 | </attachments> |
|
148 | </attachments> | |
155 | </content> |
|
149 | </content> | |
156 | <!-- |
|
150 | <!-- | |
157 | There can be several signatures for one model. At least one |
|
151 | There can be several signatures for one model. At least one | |
158 | signature must be made with the key used in global ID. |
|
152 | signature must be made with the key used in global ID. | |
159 | --> |
|
153 | --> | |
160 | <signatures> |
|
154 | <signatures> | |
161 | <signature key="id1" type="ecdsa" value="dhefhtreh" /> |
|
155 | <signature key="id1" type="ecdsa" value="dhefhtreh" /> | |
162 | <signature key="id45" type="ecdsa" value="dsgfgdhefhtreh" /> |
|
156 | <signature key="id45" type="ecdsa" value="dsgfgdhefhtreh" /> | |
163 | </signatures> |
|
157 | </signatures> | |
|
158 | <attachment-refs> | |||
|
159 | <attachment-ref ref="TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0" | |||
|
160 | url="/media/images/12345.png" /> | |||
|
161 | </attachment-refs> | |||
164 | </model> |
|
162 | </model> | |
165 | <model name="post"> |
|
163 | <model name="post"> | |
166 | <content> |
|
164 | <content> | |
167 | <id key="id1" type="ecdsa" local-id="id2" /> |
|
165 | <id key="id1" type="ecdsa" local-id="id2" /> | |
168 | <title>13</title> |
|
166 | <title>13</title> | |
169 | <text>Thirteen</text> |
|
167 | <text>Thirteen</text> | |
170 | <pub-time>12</pub-time> |
|
168 | <pub-time>12</pub-time> | |
171 | <edit-time>13</edit-time> |
|
169 | <edit-time>13</edit-time> | |
172 | <tags> |
|
170 | <tags> | |
173 | <tag>tag1</tag> |
|
171 | <tag>tag1</tag> | |
174 | </tags> |
|
172 | </tags> | |
175 | </content> |
|
173 | </content> | |
176 | <signatures> |
|
174 | <signatures> | |
177 | <signature key="id2" type="ecdsa" value="dehdfh" /> |
|
175 | <signature key="id2" type="ecdsa" value="dehdfh" /> | |
178 | </signatures> |
|
176 | </signatures> | |
179 | </model> |
|
177 | </model> | |
180 | </models> |
|
178 | </models> | |
181 | </response> |
|
179 | </response> | |
182 |
|
180 | |||
183 | ### 3.1.3 put ### |
|
181 | ### 3.1.3 put ### | |
184 |
|
182 | |||
185 | "put" gives a model to the given node (you have no guarantee the node takes |
|
183 | "put" gives a model to the given node (you have no guarantee the node takes | |
186 | it, consider you are just advising the node to take your post). This request |
|
184 | it, consider you are just advising the node to take your post). This request | |
187 | type is useful in pool where all the nodes try to duplicate all of their data |
|
185 | type is useful in pool where all the nodes try to duplicate all of their data | |
188 | across the pool. |
|
186 | across the pool. | |
189 |
|
187 | |||
190 | ## 3.2 Responses ## |
|
188 | ## 3.2 Responses ## | |
191 |
|
189 | |||
192 | ### 3.2.1 "not supported" ### |
|
190 | ### 3.2.1 "not supported" ### | |
193 |
|
191 | |||
194 | If the request if completely not supported, a "not supported" status will be |
|
192 | If the request if completely not supported, a "not supported" status will be | |
195 | returned. |
|
193 | returned. | |
196 |
|
194 | |||
197 | ### 3.2.2 "success" ### |
|
195 | ### 3.2.2 "success" ### | |
198 |
|
196 | |||
199 | "success" status means the request was processed and the result is returned. |
|
197 | "success" status means the request was processed and the result is returned. | |
200 |
|
198 | |||
201 | ### 3.2.3 "error" ### |
|
199 | ### 3.2.3 "error" ### | |
202 |
|
200 | |||
203 | If the server knows for sure that the operation cannot be processed, it sends |
|
201 | If the server knows for sure that the operation cannot be processed, it sends | |
204 | the "error" status. Additional tags describing the error may be <description> |
|
202 | the "error" status. Additional tags describing the error may be <description> | |
205 | and <stack>. |
|
203 | and <stack>. |
General Comments 0
You need to be logged in to leave comments.
Login now