经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » 数据库/运维 » MongoDB » 查看文章
MongoDB利用oplog恢复数据的方法
来源:jb51  时间:2022/4/11 8:44:58  对本文有异议

当我们对数据出现误操作的时候,可以利用oplog恢复数据。

使用前提:

  • 1、环境是副本集
  • 2、必须有全备
  • 2、全备后oplog没有被覆盖

数据全备

  1. mongodump -h 172.16.254.133 --port 27017 -o /mongodb/backup/backup

模拟故障

写入数据

  1. handong1:PRIMARY> for (var i = 1; i <= 100; i++) {
  2. ... db.test.insert( { id : i , name: "handong" , name1:"handong", name2:"handong", name3:"handong"} )
  3. ... }
  4. WriteResult({ "nInserted" : 1 })
  5. handong1:PRIMARY>
  6. handong1:PRIMARY> db.test.count()
  7. 100
  8.  

模拟误操作

  1. handong1:PRIMARY> db.test.remove({})
  2. WriteResult({ "nRemoved" : 100 })
  3. handong1:PRIMARY> db.test.count()
  4. 0
  5.  

所有文档被误删除。

恢复步骤

备份oplog

  1. mongodump -h 172.16.254.133 --port 27017 -d local -c oplog.rs -o /mongodb/backup
  2. 2021-04-30T18:32:29.077+0800 writing local.oplog.rs to /mongodb/backup/local/oplog.rs.bson
  3. 2021-04-30T18:32:32.039+0800 local.oplog.rs 7108
  4. 2021-04-30T18:32:35.038+0800 local.oplog.rs 17912
  5. 2021-04-30T18:32:38.041+0800 local.oplog.rs 28226
  6. 2021-04-30T18:32:41.039+0800 local.oplog.rs 38642
  7. 2021-04-30T18:32:44.042+0800 local.oplog.rs 50679
  8. 2021-04-30T18:32:47.040+0800 local.oplog.rs 64001
  9. 2021-04-30T18:32:50.040+0800 local.oplog.rs 77265
  10. 2021-04-30T18:32:53.038+0800 local.oplog.rs 89739
  11. 2021-04-30T18:32:56.038+0800 local.oplog.rs 102449
  12. 2021-04-30T18:32:57.697+0800 local.oplog.rs 132459
  13. 2021-04-30T18:32:57.697+0800 done dumping local.oplog.rs (132459 documents)
  14.  

解析oplog

  1. bsondump /mongodb/backup/local/oplog.rs.bson > /mongodb/backup/local/local.log
  2. 2021-04-30T18:34:27.612+0800 132459 objects found

将oplog备份和全备复制到standalone机

  1. scp -r backup/ 172.16.254.130:/mongodb/backup/
  1. scp -r local/ 172.16.254.130:/mongodb/backup/backup

查找误操作时间点

通过查看解析完的日志local.log发现误操作的时间点在1619778429。

进行数据恢复

  1. mongorestore -h 172.16.254.130 --port 27017 --oplogReplay --oplogLimit 1619778429:1 /mongodb/backup/backup
  2. 2021-04-30T19:00:11.099+0800 preparing collections to restore from
  3. 2021-04-30T19:00:11.100+0800 don't know what to do with file "/mongodb/backup/backup/local/111.log", skipping...
  4. 2021-04-30T19:00:11.100+0800 don't know what to do with file "/mongodb/backup/backup/local/local.log", skipping...
  5. 2021-04-30T19:00:11.116+0800 reading metadata for db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.metadata.json
  6. 2021-04-30T19:00:11.117+0800 reading metadata for ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.metadata.json
  7. 2021-04-30T19:00:11.119+0800 reading metadata for db3.db3 from /mongodb/backup/backup/db3/db3.metadata.json
  8. 2021-04-30T19:00:11.119+0800 reading metadata for ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.metadata.json
  9. 2021-04-30T19:00:11.170+0800 restoring ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.bson
  10. 2021-04-30T19:00:11.187+0800 restoring ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.bson
  11. 2021-04-30T19:00:11.391+0800 restoring db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.bson
  12. 2021-04-30T19:00:11.580+0800 restoring db3.db3 from /mongodb/backup/backup/db3/db3.bson
  13. 2021-04-30T19:00:11.661+0800 no indexes to restore
  14. 2021-04-30T19:00:11.661+0800 finished restoring db3.db3 (6 documents, 0 failures)
  15. 2021-04-30T19:00:11.662+0800 reading metadata for db5.test from /mongodb/backup/backup/db5/test.metadata.json
  16. 2021-04-30T19:00:12.545+0800 restoring db5.test from /mongodb/backup/backup/db5/test.bson
  17. 2021-04-30T19:00:12.548+0800 no indexes to restore
  18. 2021-04-30T19:00:12.548+0800 finished restoring db5.test (0 documents, 0 failures)
  19. 2021-04-30T19:00:12.548+0800 reading metadata for db4.db4 from /mongodb/backup/backup/db4/db4.metadata.json
  20. 2021-04-30T19:00:13.784+0800 no indexes to restore
  21. 2021-04-30T19:00:13.784+0800 finished restoring ycsb1.usertable (30370 documents, 0 failures)
  22. 2021-04-30T19:00:13.785+0800 reading metadata for db4.test1 from /mongodb/backup/backup/db4/test1.metadata.json
  23. 2021-04-30T19:00:14.099+0800 [###################.....] ycsb.usertable 47.8MB/58.9MB (81.2%)
  24. 2021-04-30T19:00:14.099+0800 [........................] db4.rcmd_1_tag_li_liao 79.8MB/3.32GB (2.3%)
  25. 2021-04-30T19:00:14.099+0800
  26. 2021-04-30T19:00:14.843+0800 [########################] ycsb.usertable 58.9MB/58.9MB (100.0%)
  27. 2021-04-30T19:00:14.843+0800 no indexes to restore
  28. 2021-04-30T19:00:14.843+0800 finished restoring ycsb.usertable (43458 documents, 0 failures)
  29. 2021-04-30T19:00:15.339+0800 restoring db4.db4 from /mongodb/backup/backup/db4/db4.bson
  30. 2021-04-30T19:00:17.077+0800 restoring db4.test1 from /mongodb/backup/backup/db4/test1.bson
  31. 2021-04-30T19:00:17.103+0800 [#.......................] db4.rcmd_1_tag_li_liao 167MB/3.32GB (4.9%)
  32. 2021-04-30T19:00:17.104+0800 [#.......................] db4.db4 8.11MB/137MB (5.9%)
  33. 2021-04-30T19:00:17.104+0800 [........................] db4.test1 0B/104MB (0.0%)
  34. 2021-04-30T19:00:17.104+0800
  35. 2021-04-30T19:00:20.099+0800 [#.......................] db4.rcmd_1_tag_li_liao 204MB/3.32GB (6.0%)
  36. 2021-04-30T19:00:20.099+0800 [####....................] db4.db4 24.5MB/137MB (18.0%)
  37. 2021-04-30T19:00:20.099+0800 [#.......................] db4.test1 4.47MB/104MB (4.3%)
  38. 2021-04-30T19:00:20.099+0800
  39. 2021-04-30T19:00:23.099+0800 [#.......................] db4.rcmd_1_tag_li_liao 272MB/3.32GB (8.0%)
  40. 2021-04-30T19:00:23.099+0800 [######..................] db4.db4 39.7MB/137MB (29.1%)
  41. 2021-04-30T19:00:23.099+0800 [####....................] db4.test1 20.1MB/104MB (19.3%)
  42. 2021-04-30T19:00:23.099+0800
  43. 2021-04-30T19:00:26.102+0800 [##......................] db4.rcmd_1_tag_li_liao 355MB/3.32GB (10.4%)
  44. 2021-04-30T19:00:26.102+0800 [##########..............] db4.db4 58.0MB/137MB (42.5%)
  45. 2021-04-30T19:00:26.102+0800 [########................] db4.test1 38.1MB/104MB (36.7%)
  46. 2021-04-30T19:00:26.102+0800
  47. 2021-04-30T19:00:29.098+0800 [##......................] db4.rcmd_1_tag_li_liao 401MB/3.32GB (11.8%)
  48. 2021-04-30T19:00:29.098+0800 [############............] db4.db4 73.1MB/137MB (53.5%)
  49. 2021-04-30T19:00:29.098+0800 [###########.............] db4.test1 51.8MB/104MB (49.8%)
  50. 2021-04-30T19:00:29.098+0800
  51. 2021-04-30T19:00:32.097+0800 [###.....................] db4.rcmd_1_tag_li_liao 494MB/3.32GB (14.5%)
  52. 2021-04-30T19:00:32.097+0800 [###############.........] db4.db4 90.8MB/137MB (66.5%)
  53. 2021-04-30T19:00:32.097+0800 [###############.........] db4.test1 67.3MB/104MB (64.7%)
  54. 2021-04-30T19:00:32.097+0800
  55. 2021-04-30T19:00:35.100+0800 [###.....................] db4.rcmd_1_tag_li_liao 556MB/3.32GB (16.3%)
  56. 2021-04-30T19:00:35.100+0800 [###################.....] db4.db4 110MB/137MB (80.5%)
  57. 2021-04-30T19:00:35.100+0800 [###################.....] db4.test1 86.1MB/104MB (82.8%)
  58. 2021-04-30T19:00:35.100+0800
  59. 2021-04-30T19:00:38.097+0800 [####....................] db4.rcmd_1_tag_li_liao 620MB/3.32GB (18.2%)
  60. 2021-04-30T19:00:38.097+0800 [#####################...] db4.db4 124MB/137MB (91.1%)
  61. 2021-04-30T19:00:38.097+0800 [#######################.] db4.test1 101MB/104MB (96.7%)
  62. 2021-04-30T19:00:38.097+0800
  63. 2021-04-30T19:00:39.023+0800 [########################] db4.test1 104MB/104MB (100.0%)
  64. 2021-04-30T19:00:39.023+0800 no indexes to restore
  65. 2021-04-30T19:00:39.023+0800 finished restoring db4.test1 (1000000 documents, 0 failures)
  66. 2021-04-30T19:00:40.386+0800 [########################] db4.db4 137MB/137MB (100.0%)
  67. 2021-04-30T19:00:40.386+0800 no indexes to restore
  68. 2021-04-30T19:00:40.386+0800 finished restoring db4.db4 (1313657 documents, 0 failures)
  69. 2021-04-30T19:00:41.097+0800 [####....................] db4.rcmd_1_tag_li_liao 684MB/3.32GB (20.1%)
  70. 2021-04-30T19:00:44.097+0800 [#####...................] db4.rcmd_1_tag_li_liao 760MB/3.32GB (22.3%)
  71. 2021-04-30T19:00:47.097+0800 [#####...................] db4.rcmd_1_tag_li_liao 836MB/3.32GB (24.6%)
  72. 2021-04-30T19:00:50.098+0800 [######..................] db4.rcmd_1_tag_li_liao 906MB/3.32GB (26.6%)
  73. 2021-04-30T19:00:53.098+0800 [#######.................] db4.rcmd_1_tag_li_liao 994MB/3.32GB (29.2%)
  74. 2021-04-30T19:00:56.098+0800 [#######.................] db4.rcmd_1_tag_li_liao 1.03GB/3.32GB (31.0%)
  75. 2021-04-30T19:00:59.098+0800 [########................] db4.rcmd_1_tag_li_liao 1.11GB/3.32GB (33.3%)
  76. 2021-04-30T19:01:02.097+0800 [########................] db4.rcmd_1_tag_li_liao 1.18GB/3.32GB (35.5%)
  77. 2021-04-30T19:01:05.101+0800 [#########...............] db4.rcmd_1_tag_li_liao 1.26GB/3.32GB (38.0%)
  78. 2021-04-30T19:01:08.097+0800 [#########...............] db4.rcmd_1_tag_li_liao 1.32GB/3.32GB (39.7%)
  79. 2021-04-30T19:01:11.100+0800 [#########...............] db4.rcmd_1_tag_li_liao 1.37GB/3.32GB (41.1%)
  80. 2021-04-30T19:01:14.098+0800 [##########..............] db4.rcmd_1_tag_li_liao 1.43GB/3.32GB (43.2%)
  81. 2021-04-30T19:01:17.097+0800 [##########..............] db4.rcmd_1_tag_li_liao 1.50GB/3.32GB (45.0%)
  82. 2021-04-30T19:01:20.098+0800 [###########.............] db4.rcmd_1_tag_li_liao 1.54GB/3.32GB (46.3%)
  83. 2021-04-30T19:01:23.098+0800 [###########.............] db4.rcmd_1_tag_li_liao 1.58GB/3.32GB (47.6%)
  84. 2021-04-30T19:01:26.098+0800 [###########.............] db4.rcmd_1_tag_li_liao 1.64GB/3.32GB (49.3%)
  85. 2021-04-30T19:01:29.097+0800 [############............] db4.rcmd_1_tag_li_liao 1.71GB/3.32GB (51.4%)
  86. 2021-04-30T19:01:32.097+0800 [############............] db4.rcmd_1_tag_li_liao 1.77GB/3.32GB (53.2%)
  87. 2021-04-30T19:01:35.098+0800 [#############...........] db4.rcmd_1_tag_li_liao 1.85GB/3.32GB (55.7%)
  88. 2021-04-30T19:01:38.097+0800 [#############...........] db4.rcmd_1_tag_li_liao 1.90GB/3.32GB (57.2%)
  89. 2021-04-30T19:01:41.097+0800 [##############..........] db4.rcmd_1_tag_li_liao 1.98GB/3.32GB (59.5%)
  90. 2021-04-30T19:01:44.827+0800 [##############..........] db4.rcmd_1_tag_li_liao 2.00GB/3.32GB (60.3%)
  91. 2021-04-30T19:01:47.097+0800 [##############..........] db4.rcmd_1_tag_li_liao 2.05GB/3.32GB (61.8%)
  92. 2021-04-30T19:01:50.098+0800 [###############.........] db4.rcmd_1_tag_li_liao 2.12GB/3.32GB (63.9%)
  93. 2021-04-30T19:01:53.097+0800 [###############.........] db4.rcmd_1_tag_li_liao 2.19GB/3.32GB (65.9%)
  94. 2021-04-30T19:01:56.097+0800 [################........] db4.rcmd_1_tag_li_liao 2.26GB/3.32GB (67.9%)
  95. 2021-04-30T19:01:59.099+0800 [################........] db4.rcmd_1_tag_li_liao 2.32GB/3.32GB (69.8%)
  96. 2021-04-30T19:02:02.098+0800 [#################.......] db4.rcmd_1_tag_li_liao 2.39GB/3.32GB (72.0%)
  97. 2021-04-30T19:02:05.097+0800 [#################.......] db4.rcmd_1_tag_li_liao 2.47GB/3.32GB (74.4%)
  98. 2021-04-30T19:02:08.097+0800 [##################......] db4.rcmd_1_tag_li_liao 2.52GB/3.32GB (76.0%)
  99. 2021-04-30T19:02:11.097+0800 [##################......] db4.rcmd_1_tag_li_liao 2.59GB/3.32GB (77.8%)
  100. 2021-04-30T19:02:14.097+0800 [###################.....] db4.rcmd_1_tag_li_liao 2.66GB/3.32GB (80.0%)
  101. 2021-04-30T19:02:17.097+0800 [###################.....] db4.rcmd_1_tag_li_liao 2.72GB/3.32GB (81.9%)
  102. 2021-04-30T19:02:20.097+0800 [####################....] db4.rcmd_1_tag_li_liao 2.78GB/3.32GB (83.7%)
  103. 2021-04-30T19:02:23.097+0800 [####################....] db4.rcmd_1_tag_li_liao 2.85GB/3.32GB (85.7%)
  104. 2021-04-30T19:02:26.098+0800 [#####################...] db4.rcmd_1_tag_li_liao 2.94GB/3.32GB (88.4%)
  105. 2021-04-30T19:02:29.097+0800 [#####################...] db4.rcmd_1_tag_li_liao 3.00GB/3.32GB (90.4%)
  106. 2021-04-30T19:02:32.097+0800 [######################..] db4.rcmd_1_tag_li_liao 3.06GB/3.32GB (92.1%)
  107. 2021-04-30T19:02:35.099+0800 [######################..] db4.rcmd_1_tag_li_liao 3.12GB/3.32GB (93.9%)
  108. 2021-04-30T19:02:38.097+0800 [######################..] db4.rcmd_1_tag_li_liao 3.15GB/3.32GB (95.0%)
  109. 2021-04-30T19:02:41.098+0800 [#######################.] db4.rcmd_1_tag_li_liao 3.21GB/3.32GB (96.7%)
  110. 2021-04-30T19:02:44.167+0800 [#######################.] db4.rcmd_1_tag_li_liao 3.26GB/3.32GB (98.0%)
  111. 2021-04-30T19:02:47.097+0800 [#######################.] db4.rcmd_1_tag_li_liao 3.32GB/3.32GB (99.9%)
  112. 2021-04-30T19:02:47.392+0800 [########################] db4.rcmd_1_tag_li_liao 3.32GB/3.32GB (100.0%)
  113. 2021-04-30T19:02:47.393+0800 no indexes to restore
  114. 2021-04-30T19:02:47.393+0800 finished restoring db4.rcmd_1_tag_li_liao (379143 documents, 0 failures)
  115. 2021-04-30T19:02:47.393+0800 restoring users from /mongodb/backup/backup/admin/system.users.bson
  116. 2021-04-30T19:02:50.655+0800 admin.tempusers 2.05KB
  117. 2021-04-30T19:02:50.655+0800 admin.tempusers 2.05KB
  118. 2021-04-30T19:02:51.905+0800 replaying oplog
  119. 2021-04-30T19:02:53.097+0800 oplog 483KB
  120. 2021-04-30T19:02:56.097+0800 oplog 20.2MB
  121. 2021-04-30T19:02:59.100+0800 oplog 36.9MB
  122. 2021-04-30T19:03:02.097+0800 oplog 50.1MB
  123. 2021-04-30T19:03:05.098+0800 oplog 69.0MB
  124. 2021-04-30T19:03:08.097+0800 oplog 90.6MB
  125. 2021-04-30T19:03:11.097+0800 oplog 124MB
  126. 2021-04-30T19:03:14.097+0800 oplog 159MB
  127. 2021-04-30T19:03:17.098+0800 oplog 185MB
  128. 2021-04-30T19:03:20.097+0800 oplog 219MB
  129. 2021-04-30T19:03:23.098+0800 oplog 256MB
  130. 2021-04-30T19:03:26.097+0800 oplog 290MB
  131. 2021-04-30T19:03:29.097+0800 oplog 323MB
  132. 2021-04-30T19:03:32.097+0800 oplog 357MB
  133. 2021-04-30T19:03:35.097+0800 oplog 391MB
  134. 2021-04-30T19:03:38.098+0800 oplog 427MB
  135. 2021-04-30T19:03:41.098+0800 oplog 464MB
  136. 2021-04-30T19:03:44.097+0800 oplog 501MB
  137. 2021-04-30T19:03:47.097+0800 oplog 544MB
  138. 2021-04-30T19:03:50.098+0800 oplog 580MB
  139. 2021-04-30T19:03:53.098+0800 oplog 619MB
  140. 2021-04-30T19:03:56.098+0800 oplog 650MB
  141. 2021-04-30T19:03:59.099+0800 oplog 687MB
  142. 2021-04-30T19:04:02.097+0800 oplog 722MB
  143. 2021-04-30T19:04:05.097+0800 oplog 758MB
  144. 2021-04-30T19:04:08.098+0800 oplog 795MB
  145. 2021-04-30T19:04:11.097+0800 oplog 826MB
  146. 2021-04-30T19:04:14.098+0800 oplog 858MB
  147. 2021-04-30T19:04:17.102+0800 oplog 893MB
  148. 2021-04-30T19:04:20.097+0800 oplog 929MB
  149. 2021-04-30T19:04:23.098+0800 oplog 968MB
  150. 2021-04-30T19:04:26.098+0800 oplog 1001MB
  151. 2021-04-30T19:04:29.097+0800 oplog 1002MB
  152. 2021-04-30T19:04:32.097+0800 oplog 1003MB
  153. 2021-04-30T19:04:35.097+0800 oplog 1004MB
  154. 2021-04-30T19:04:38.097+0800 oplog 1004MB
  155. 2021-04-30T19:04:38.597+0800 applied 130029 oplog entries
  156. 2021-04-30T19:04:38.597+0800 oplog 1005MB
  157. 2021-04-30T19:04:38.614+0800 2766634 document(s) restored successfully. 0 document(s) failed to restore.
  158.  

检查恢复结果

  1. > show dbs
  2. admin 0.000GB
  3. config 0.000GB
  4. db3 0.000GB
  5. db4 3.355GB
  6. db5 0.000GB
  7. local 0.000GB
  8. ycsb 0.060GB
  9. ycsb1 0.041GB
  10. >
  11. >
  12. > use db5
  13. switched to db db5
  14. >
  15. > db.test.count()
  16. 100
  17.  

可以看到我们前期插入的100记录以及恢复成功。

到此这篇关于MongoDB利用oplog恢复数据介绍的文章就介绍到这了,更多相关oplog数据恢复内容请搜索w3xue以前的文章或继续浏览下面的相关文章希望大家以后多多支持w3xue!

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号