9. 用Rust手把手编写一个wmproxy(代理，内网穿透等), HTTP2改造篇之HPACK示例, 了解http2头信息如何处理
项目 ++wmproxy++
gite: https://gitee.com/tickbh/wmproxy
github: https://github.com/tickbh/wmproxy
关于HPACK相关数据的示例
长度编码的示例，用5位的前缀示例
0 1 2 3 4 5 6 7+---+---+---+---+---+---+---+---+| X | X | X | 0 | 1 | 0 | 1 | 0 | 10 stored on 5 bits+---+---+---+---+---+---+---+---+
- 1337大于2^5-1，故前5位填充
0 1 2 3 4 5 6 7+---+---+---+---+---+---+---+---+| X | X | X | 1 | 1 | 1 | 1 | 1 | 31+---+---+---+---+---+---+---+---+
- 1337 - 31 = 1306，大于128，故需要二次填充，用1306 mod 128 = 21，首位填充1，故8位填充为，当前偏移值为7
0 1 2 3 4 5 6 7+---+---+---+---+---+---+---+---+| 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 26+---+---+---+---+---+---+---+---+
- 对1301-21=1280，1280 / 128 = 10， 10 < 128，故已经完成，首位填0
0 1 2 3 4 5 6 7+---+---+---+---+---+---+---+---+| 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 10+---+---+---+---+---+---+---+---+
最终的填充值：
0 1 2 3 4 5 6 7+---+---+---+---+---+---+---+---+| X | X | X | 1 | 1 | 1 | 1 | 1 | Prefix = 31, I = 1306| 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 1306>=128, encode(154), I=1306/128| 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 10<128, encode(10), done+---+---+---+---+---+---+---+---+
头部编码示例
在静态列表中索引
以下待索引的值
:method: GET
十六进制表示值82，二进制表示10000010，表示取静态表2的值，查表为(:method, GET)
不在列表中，但请求索引，未使用HUFFMAN
以下示例
custom-key: custom-header
十六进制表示
400a 6375 7374 6f6d 2d6b 6579 0d63 7573 | @.custom-key.cus746f 6d2d 6865 6164 6572 | tom-header
解码过程
40 | == 01开头请求索引 ==0a | name (长度 10)6375 7374 6f6d 2d6b 6579 | custom-key0d | value (长度 13)6375 7374 6f6d 2d68 6561 6465 72 | custom-header | -> custom-key: | custom-header
动态表 (解码之后):
[ 1] (占用 55) custom-key: custom-header
占用长度: 10+13+32=55
名字在列表中，但不索引，未使用HUFFMAN
以下示例
:path: /sample/path
十六进制表示
040c 2f73 616d 706c 652f 7061 7468 | ../sample/path
解码过程
04 | == 0000开头，请求不索引 == | name从索引取 (idx = 4) | 值为:path0c | value (长度12)2f73 616d 706c 652f 7061 7468 | /sample/path | -> :path: /sample/path
永不索引，未使用HUFFMAN
以下示例
password: secret
十六进制表示
1008 7061 7373 776f 7264 0673 6563 7265 | ..password.secre74 | t
解码过程
10 | == 0001开头不索引 ==08 | name (长度8)7061 7373 776f 7264 | password06 | value (长度6)7365 6372 6574 | secret | -> password: secret
完整的请求示例，不使用HUFFMAN
以下几个示例将连接请求，后续的会用到前面的动态列表
第一次请求. 示例如下
:method: GET:scheme: http:path: /:authority: www.example.com
十六进制表示
8286 8441 0f77 7777 2e65 7861 6d70 6c65 | ...A.www.example2e63 6f6d | .com
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET86 | == Indexed - 静态表 == | idx = 6 | -> :scheme: http84 | == Indexed - 静态表 == | idx = 4 | -> :path: /41 | == 01开头请求索引 indexed == | Indexed name (idx = 1) | :authority0f | Literal value (长度15)7777 772e 6578 616d 706c 652e 636f 6d | www.example.com | -> :authority: | www.example.com
动态列表 (解码后):
[ 1->62] (s = 57) :authority: www.example.com
列表长度: 57
第二次请求. 示例如下，新加了cache-control字段，其它和第一次一样
:method: GET:scheme: http:path: /:authority: www.example.comcache-control: no-cache
十六进制表示
8286 84be 5808 6e6f 2d63 6163 6865 | ....X.no-cache
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET86 | == Indexed - 静态表 == | idx = 6 | -> :scheme: http84 | == Indexed - 静态表 == | idx = 4 | -> :path: /be | == Indexed - 动态表，索引值62及以上的为动态表 == | idx = 62 | -> :authority: | www.example.com58 | == Literal indexed == | Indexed name (idx = 24) | cache-control08 | Literal value (8)6e6f 2d63 6163 6865 | no-cache | -> cache-control: no-cache
动态列表 (解码后):
[ 1->62] (s = 53) cache-control: no-cache
[ 2->63] (s = 57) :authority: www.example.com
总长度: 110
第三次请求. 示例如下
:method: GET:scheme: https:path: /index.html:authority: www.example.comcustom-key: custom-value
十六进制表示
8287 85bf 400a 6375 7374 6f6d 2d6b 6579 | ....@.custom-key0c63 7573 746f 6d2d 7661 6c75 65 | .custom-value
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET87 | == Indexed - 静态表 == | idx = 7 | -> :scheme: https85 | == Indexed - 静态表 == | idx = 5 | -> :path: /index.htmlbf | == Indexed - 动态表 == | idx = 63 | -> :authority: | www.example.com40 | == Literal indexed ==0a | Literal name (长度10)6375 7374 6f6d 2d6b 6579 | custom-key0c | Literal value (长度12)6375 7374 6f6d 2d76 616c 7565 | custom-value | -> custom-key: | custom-value
动态列表 (解码后):
[ 1->62] (s = 54) custom-key: custom-value
[ 2->63] (s = 53) cache-control: no-cache
[ 3->64] (s = 57) :authority: www.example.com
总长度: 164
完整的请求示例（和上述例子一模一样，但是使用HUFFMAN)
以下几个示例将连接请求，后续的会用到前面的动态列表
第一次请求. 示例如下
:method: GET:scheme: http:path: /:authority: www.example.com
十六进制表示
8286 8441 8cf1 e3c2 e5f2 3a6b a0ab 90f4 | ...A......:k....ff | .
比之前少了3字节
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET86 | == Indexed - 静态表 == | idx = 6 | -> :scheme: http84 | == Indexed - 静态表 == | idx = 4 | -> :path: /41 | == Literal indexed == | Indexed name (idx = 1) | :authority8c | Literal value (长度12) | Huffman encoded:f1e3 c2e5 f23a 6ba0 ab90 f4ff | .....:k..... | Decoded: | www.example.com | -> :authority: | www.example.com
动态列表 (解码后):
[ 1->62] (s = 57) :authority: www.example.com 列表长度: 57
第二次请求. 示例如下，新加了cache-control字段，其它和第一次一样
:method: GET:scheme: http:path: /:authority: www.example.comcache-control: no-cache
十六进制表示
8286 84be 5886 a8eb 1064 9cbf | ....X....d..
比之前少了2字节
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET86 | == Indexed - 静态表 == | idx = 6 | -> :scheme: http84 | == Indexed - 静态表 == | idx = 4 | -> :path: /be | == Indexed - 动态表 == | idx = 62 | -> :authority: | www.example.com58 | == Literal indexed == | Indexed name (idx = 24) | cache-control86 | Literal value (长度6) | Huffman encoded:a8eb 1064 9cbf | ...d.. | Decoded: | no-cache | -> cache-control: no-cache
动态列表 (解码后):
[ 1->62] (s = 53) cache-control: no-cache[ 2->63] (s = 57) :authority: www.example.com 列表长度: 110
第三次请求. 示例如下
:method: GET:scheme: https:path: /index.html:authority: www.example.comcustom-key: custom-value
十六进制表示
8287 85bf 4088 25a8 49e9 5ba9 7d7f 8925 | ....@.%.I.[.}..%a849 e95b b8e8 b4bf | .I.[....
比之前少了5字节
解码过程
82 | == Indexed - 静态表 == | idx = 2 | -> :method: GET87 | == Indexed - 静态表 == | idx = 7 | -> :scheme: https85 | == Indexed - 静态表 == | idx = 5 | -> :path: /index.htmlbf | == Indexed - 动态表 == | idx = 63 | -> :authority: | www.example.com40 | == Literal indexed ==88 | Literal name (长度8) | Huffman encoded:25a8 49e9 5ba9 7d7f | %.I.[.}. | Decoded: | custom-key89 | Literal value (长度9) | Huffman encoded:25a8 49e9 5bb8 e8b4 bf | %.I.[.... | Decoded: | custom-value | -> custom-key: | custom-value
动态列表 (解码后):
[ 1->62] (s = 54) custom-key: custom-value[ 2->63] (s = 53) cache-control: no-cache[ 3->64] (s = 57) :authority: www.example.com 总长度: 164
HUFFMAN编码在于首次如果数据较大的时候优势会更加明显，如果数据较小，或者在后续的时候与普通编码命中索引时基本一致。
完整的返回示例(HUFFMAN)
HUFFMAN与普通的差别在于字符串编解码时的差别，这里只介绍一种，并且设置SETTINGS_HEADER_TABLE_SIZE为256
以下几个示例将连接请求，后续的会用到前面的动态列表
第一次返回. 示例如下
:status: 302cache-control: privatedate: Mon, 21 Oct 2013 20:13:21 GMTlocation: https://www.example.com
十六进制表示
4882 6402 5885 aec3 771a 4b61 96d0 7abe | H.d.X...w.Ka..z.9410 54d4 44a8 2005 9504 0b81 66e0 82a6 | ..T.D. .....f...2d1b ff6e 919d 29ad 1718 63c7 8f0b 97c8 | -..n..)...c.....e9ae 82ae 43d3 | ....C.
解码过程
48 | == Literal indexed == | Indexed name (idx = 8) | :status82 | Literal value (长度2) | Huffman encoded:6402 | d. | Decoded: | 302 | -> :status: 30258 | == Literal indexed == | Indexed name (idx = 24) | cache-control85 | Literal value (长度5) | Huffman encoded:aec3 771a 4b | ..w.K | Decoded: | private | -> cache-control: private61 | == Literal indexed == | Indexed name (idx = 33) | date96 | Literal value (长度22) | Huffman encoded:d07a be94 1054 d444 a820 0595 040b 8166 | .z...T.D. .....fe082 a62d 1bff | ...-.. | Decoded: | Mon, 21 Oct 2013 20:13:21 | GMT | -> date: Mon, 21 Oct 2013 | 20:13:21 GMT6e | == Literal indexed == | Indexed name (idx = 46) | location91 | Literal value (长度17) | Huffman encoded:9d29 ad17 1863 c78f 0b97 c8e9 ae82 ae43 | .)...c.........Cd3 | . | Decoded: | https://www.example.com | -> location: | https://www.example.com
动态列表 (解码后):
[ 1->62] (s = 63) location: https://www.example.com[ 2->63] (s = 65) date: Mon, 21 Oct 2013 20:13:21 GMT[ 3->64] (s = 52) cache-control: private[ 4->65] (s = 42) :status: 302 Table size: 222
第二次请求. 示例如下，只是状态码发生了变更
:status: 307cache-control: privatedate: Mon, 21 Oct 2013 20:13:21 GMTlocation: https://www.example.com
十六进制表示
4883 640e ffc1 c0bf | H.d.....
解码过程
48 | == Literal indexed == | Indexed name (idx = 8) | :status83 | Literal value (长度3) | Huffman encoded:640e ff | d.. | Decoded: | 307 | - evict: :status: 302 | -> :status: 307c1 | == Indexed - Add == | idx = 65 | -> cache-control: privatec0 | == Indexed - Add == | idx = 64 | -> date: Mon, 21 Oct 2013 | 20:13:21 GMTbf | == Indexed - Add == | idx = 63 | -> location: | https://www.example.com
动态列表 (解码后):
[ 1->62] (s = 42) :status: 307[ 2->63] (s = 63) location: https://www.example.com[ 3->64] (s = 65) date: Mon, 21 Oct 2013 20:13:21 GMT[ 4->65] (s = 52) cache-control: private Table size: 222
由于(:status, 302)的长度为42，且42+222=264>256，所以舍弃最大值
第三次请求. 示例如下
:status: 200cache-control: privatedate: Mon, 21 Oct 2013 20:13:22 GMTlocation: https://www.example.comcontent-encoding: gzipset-cookie: foo=ASDJKHQKBZXOQWEOPIUAXQWEOIU; max-age=3600; version=1
十六进制表示
88c1 6196 d07a be94 1054 d444 a820 0595 | ..a..z...T.D. ..040b 8166 e084 a62d 1bff c05a 839b d9ab | ...f...-...Z....77ad 94e7 821d d7f2 e6c7 b335 dfdf cd5b | w..........5...[3960 d5af 2708 7f36 72c1 ab27 0fb5 291f | 9`..'..6r..'..).9587 3160 65c0 03ed 4ee5 b106 3d50 07 | ..1`e...N...=P.
比之前少了5字节
解码过程
88 | == Indexed - 静态表 == | idx = 8 | -> :status: 200c1 | == Indexed - 动态表 == | idx = 65 | -> cache-control: private61 | == Literal indexed == | Indexed name (idx = 33) | date96 | Literal value (长度22) | Huffman encoded:d07a be94 1054 d444 a820 0595 040b 8166 | .z...T.D. .....fe084 a62d 1bff | ...-.. | Decoded: | Mon, 21 Oct 2013 20:13:22 | GMT | - evict: cache-control: | private | -> date: Mon, 21 Oct 2013 | 20:13:22 GMTc0 | == Indexed - Add == | idx = 64 | -> location: | https://www.example.com5a | == Literal indexed == | Indexed name (idx = 26) | content-encoding83 | Literal value (长度3) | Huffman encoded:9bd9 ab | ... | Decoded: | gzip | - evict: date: Mon, 21 Oct | 2013 20:13:21 GMT | -> content-encoding: gzip77 | == Literal indexed == | Indexed name (idx = 55) | set-cookiead | Literal value (长度45) | Huffman encoded:94e7 821d d7f2 e6c7 b335 dfdf cd5b 3960 | .........5...[9`d5af 2708 7f36 72c1 ab27 0fb5 291f 9587 | ..'..6r..'..)...3160 65c0 03ed 4ee5 b106 3d50 07 | 1`e...N...=P. | Decoded: | foo=ASDJKHQKBZXOQWEOPIUAXQ | WEOIU; max-age=3600; versi | on=1 | - evict: location: | https://www.example.com | - evict: :status: 307 | -> set-cookie: foo=ASDJKHQ | KBZXOQWEOPIUAXQWEOIU; ma | x-age=3600; version=1
动态列表 (解码后):
[ 1->62] (s = 98) set-cookie: foo=ASDJKHQKBZXOQWEOPIUAXQWEOIU; max-age=3600; version=1[ 2->63] (s = 52) content-encoding: gzip[ 3->64] (s = 65) date: Mon, 21 Oct 2013 20:13:22 GMT总长度: 215
动态列表保留着一个最大的缓存大小值，每一个键值对的计算为name的字节数+value的字节数+32为确定的大小值。超出大小部分则丢弃不缓存，默认大小为4096。
总结
HPACK管理着HTTP2的头部的协议部分，有着高压缩比和重复请求的高复用性，双方编码解码需要各自维持一份动态表，动态根据处理数据来动态拓展，保证双方维持的表一模一样。从而保证ID索引不会乱。Huffman编码把头里面需要用到字符串的数据进行进一步的压缩，相对来说整个过程复杂度比HTTP1高很多，但相对的对使用者完全透明，在不影响其使用的情况下提高传输效率，并减少带宽的使用量。
经验首页