当前位置： 首页 > news >正文

Mongodb 通过一致性备份搭建SECONDARY.

news 来源：原创 2024/5/3 10:13:38

该方法应用面比较窄,

适用于 : 一主 一备 一投票节点,数据库较大,oplog 比较小,备库需要修复而且主库不能停机的情况.


该方法仅限于学习测试,线上环境慎用.





集群结构：
opsdba-vbj01-1:27018 ARBITE
opsdba-vbj01-1:27019 PRIMARY
opsdba-vbj01-1:27016 SECONDARY



模拟opsdba-vbj01-1:27016 crash ,使用一致性备份搭建备库.

1.在主库,自建role,赋予restore oplog权限.
use admin
db.runCommand({ createRole: "restoreoplog",
privileges:
        [
        { resource: { anyResource: true }, actions: [ "anyAction" ] }
        ],
roles:
       []
});
db.grantRolesToUser( "root", ["restoreoplog"] );


2.主库一致性备份.
[root@opsdba-vbj01-1 dump]# mongodump -uroot -proot123 --port=27019 --oplog --authenticationDatabase=admin -o all_backup
2016-10-11T15:45:58.019+0800    writing admin.system.users to
2016-10-11T15:45:58.019+0800    done dumping admin.system.users (1 document)
2016-10-11T15:45:58.019+0800    writing admin.system.roles to
2016-10-11T15:45:58.020+0800    done dumping admin.system.roles (1 document)
2016-10-11T15:45:58.020+0800    writing admin.system.version to
2016-10-11T15:45:58.020+0800    done dumping admin.system.version (1 document)
2016-10-11T15:45:58.021+0800    writing test.testData to
2016-10-11T15:45:58.021+0800    writing test.tab to
2016-10-11T15:45:58.190+0800    done dumping test.tab (34056 documents)
2016-10-11T15:46:01.022+0800    [###########.............]  test.testData  451185/909000  (49.6%)
2016-10-11T15:46:04.022+0800    [####################....]  test.testData  774771/909000  (85.2%)
2016-10-11T15:46:05.024+0800    [########################]  test.testData  913877/909000  (100.5%)
2016-10-11T15:46:05.024+0800    done dumping test.testData (913877 documents)
2016-10-11T15:46:05.025+0800    writing captured oplog to
2016-10-11T15:46:05.781+0800            dumped 6470 oplog entries


3.把opsdba-vbj01-1:27016 从SECONDARY 转为单节点空库.

/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf  --shutdown
清空data目录,单节点启动:
修改port = 27015,
注释掉replSet
/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf
 
创建管理员用户:
[root@opsdba-vbj01-1 mongodb1]# mongo --port=27015 admin
use admin
db.createUser(
  {
    user: "root",
    pwd: "root123",
    roles:
    [
      {
        role: "root",
        db: "admin"
      }
    ]
  }
);
自建role,赋予restore oplog权限:
[root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin
use admin
db.runCommand({ createRole: "restoreoplog",
privileges:
        [
        { resource: { anyResource: true }, actions: [ "anyAction" ] }
        ],
roles:
       []
});
db.grantRolesToUser( "root", ["restoreoplog"] );


4.导入
[root@opsdba-vbj01-1 dump]# mongorestore -uroot -proot123 --port=27015 --authenticationDatabase=admin --oplogReplay --dir=all_backup
2016-10-11T15:59:48.505+0800    building a list of dbs and collections to restore from all_backup dir
2016-10-11T15:59:48.507+0800    reading metadata for test.testData from all_backup/test/testData.metadata.json
2016-10-11T15:59:48.508+0800    reading metadata for test.tab from all_backup/test/tab.metadata.json
2016-10-11T15:59:48.556+0800    restoring test.tab from all_backup/test/tab.bson
2016-10-11T15:59:48.604+0800    restoring test.testData from all_backup/test/testData.bson
2016-10-11T15:59:49.147+0800    restoring indexes for collection test.tab from metadata
2016-10-11T15:59:49.147+0800    finished restoring test.tab (34056 documents)
2016-10-11T15:59:51.507+0800    [####....................]  test.testData  18.5 MB/94.1 MB  (19.7%)
2016-10-11T15:59:54.507+0800    [##########..............]  test.testData  39.8 MB/94.1 MB  (42.3%)
2016-10-11T15:59:57.507+0800    [###############.........]  test.testData  59.7 MB/94.1 MB  (63.5%)
2016-10-11T16:00:00.507+0800    [####################....]  test.testData  80.1 MB/94.1 MB  (85.1%)
2016-10-11T16:00:03.288+0800    [########################]  test.testData  94.1 MB/94.1 MB  (100.0%)
2016-10-11T16:00:03.288+0800    restoring indexes for collection test.testData from metadata
2016-10-11T16:00:03.289+0800    finished restoring test.testData (913877 documents)
2016-10-11T16:00:03.289+0800    restoring users from all_backup/admin/system.users.bson
2016-10-11T16:00:03.416+0800    restoring roles from all_backup/admin/system.roles.bson
2016-10-11T16:00:03.466+0800    replaying oplog
2016-10-11T16:00:03.808+0800    done

5.获取最后一个oplog的时间戳
[root@opsdba-vbj01-1 dump]# cd all_backup/
[root@opsdba-vbj01-1 all_backup]# bsondump oplog.bson >oplog.txt
[root@opsdba-vbj01-1 all_backup]# tail -1 oplog.txt
{"ts":{"$timestamp":{"t":1476171965,"i":805}},"t":{"$numberLong":"2"},"h":{"$numberLong":"6906152948185446623"},"v":2,"op":"i","ns":"test.testData","o":{"_id":{"$oid":"57fc98bddfa99af76706f721"},"x":6470.0,"name":"MACLEAN","name1":"MACLEAN","name2":"MACLEAN","name3":"MACLEAN"}}

6.初始化local库的相关表
[root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin
use local
db.runCommand( { create: "oplog.rs", capped: true, size: (1* 1024 * 1024 * 1024) } );

#数据来自oplog.txt
db.oplog.rs.save({"ts" : Timestamp(1476171965, 805),"h" : NumberLong("6906152948185446623")});
db.db.replset.minvalid.save({"ts" : Timestamp(1476171965,805), "t" : NumberLong(2)});

#数据来自主库的数据查询
db.replset.election.save({ "_id" : ObjectId("57fc5ea0cfa6486e03e975d0"), "term" : NumberLong(2), "candidateIndex" : NumberLong(2) });
db.system.replset.save({ "_id" : "myrelset", "version" : 5, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 1, "host" : "opsdba-vbj01-1:27018", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : {  }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 2, "host" : "opsdba-vbj01-1:27019", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : {  }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 3, "host" : "opsdba-vbj01-1:27016", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : {  }, "slaveDelay" : NumberLong(0), "votes" : 1 } ], "settings" : { "chainingAllowed" : true, "heartbeatIntervalMillis" : 2000, "heartbeatTimeoutSecs" : 10, "electionTimeoutMillis" : 10000, "getLastErrorModes" : {  }, "getLastErrorDefaults" : { "w" : 1, "wtimeout" : 0 }, "replicaSetId" : ObjectId("57bfdcdcd40cbe4bf173396a") } });


7.重启
use admin
db.shutdownServer();

修改为原始值.
port=27016
replSet 取消注释
启动
/data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf

8.检测:
比对行数.
db.collection.count()

9.启动日志
2016-10-11T16:42:51.520+0800 I CONTROL  [initandlisten] MongoDB starting : pid=633 port=27016 dbpath=/data/mongodb/mongodb1/data 64-bit host=opsdba-vbj01-1
2016-10-11T16:42:51.520+0800 I CONTROL  [initandlisten] db version v3.2.8
2016-10-11T16:42:51.520+0800 I CONTROL  [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-10-11T16:42:51.520+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten] allocator: tcmalloc
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten] modules: none
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten] build environment:
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten]     distmod: rhel62
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten]     distarch: x86_64
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2016-10-11T16:42:51.521+0800 I CONTROL  [initandlisten] options: { config: "/data/mongodb/mongodb1/conf/mongod.cnf", net: { http: { enabled: false }, maxIncomingConnections: 3000, port: 27016, unixDomainSocket: { pathPrefix: "/data/mongodb/mongodb1/data" } }, operationProfiling: { mode: "slowOp", slowOpThresholdMs: 800 }, processManagement: { fork: true, pidFilePath: "/data/mongodb/mongodb1/data/mongod.pid" }, replication: { replSet: "myrelset" }, security: { clusterAuthMode: "keyFile", keyFile: "/data/mongodb/mongodb1/conf/myrelset.keyfile" }, storage: { dbPath: "/data/mongodb/mongodb1/data", directoryPerDB: true, engine: "wiredTiger", journal: { commitIntervalMs: 300, enabled: true }, mmapv1: { nsSize: 32 }, repairPath: "/data/mongodb/mongodb1/data", syncPeriodSecs: 60.0, wiredTiger: { engineConfig: { cacheSizeGB: 1 } } }, systemLog: { destination: "file", path: "/data/mongodb/mongodb1/log/mongod.log", quiet: true, timeStampFormat: "iso8601-local" } }
2016-10-11T16:42:51.521+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-10-11T16:42:51.944+0800 I STORAGE  [initandlisten] Starting WiredTigerRecordStoreThread local.oplog.rs
2016-10-11T16:42:51.944+0800 I STORAGE  [initandlisten] The size storer reports that the oplog contains 1 records totaling to 45 bytes
2016-10-11T16:42:51.944+0800 I STORAGE  [initandlisten] Scanning the oplog to determine where to place markers for truncation
2016-10-11T16:42:51.982+0800 W STORAGE  [initandlisten] Detected configuration for non-active storage engine mmapv1 when current storage engine is wiredTiger
2016-10-11T16:42:51.982+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-10-11T16:42:51.982+0800 I CONTROL  [initandlisten]
2016-10-11T16:42:51.982+0800 I CONTROL  [initandlisten]
2016-10-11T16:42:51.982+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 65535 files. Number of processes should be at least 32767.5 : 0.5 times number of files.
2016-10-11T16:42:52.030+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/mongodb/mongodb1/data/diagnostic.data'
2016-10-11T16:42:52.030+0800 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-10-11T16:42:52.031+0800 I NETWORK  [initandlisten] waiting for connections on port 27016
2016-10-11T16:42:52.066+0800 I REPL     [ReplicationExecutor] New replica set config in use: { _id: "myrelset", version: 5, protocolVersion: 1, members: [ { _id: 1, host: "opsdba-vbj01-1:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "opsdba-vbj01-1:27019", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 3, host: "opsdba-vbj01-1:27016", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('57bfdcdcd40cbe4bf173396a') } }
2016-10-11T16:42:52.067+0800 I REPL     [ReplicationExecutor] This node is opsdba-vbj01-1:27016 in the config
2016-10-11T16:42:52.067+0800 I REPL     [ReplicationExecutor] transition to STARTUP2
2016-10-11T16:42:52.067+0800 I REPL     [ReplicationExecutor] Starting replication applier threads
2016-10-11T16:42:52.067+0800 I REPL     [ReplicationExecutor] transition to RECOVERING
2016-10-11T16:42:52.072+0800 I REPL     [ReplicationExecutor] transition to SECONDARY
2016-10-11T16:42:52.102+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27018
2016-10-11T16:42:52.102+0800 I ASIO     [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27019
2016-10-11T16:42:52.102+0800 I REPL     [ReplicationExecutor] Member opsdba-vbj01-1:27018 is now in state SECONDARY
2016-10-11T16:42:52.103+0800 I REPL     [ReplicationExecutor] Member opsdba-vbj01-1:27019 is now in state PRIMARY
2016-10-11T16:42:58.069+0800 I REPL     [ReplicationExecutor] syncing from: opsdba-vbj01-1:27018
2016-10-11T16:42:58.086+0800 I REPL     [SyncSourceFeedback] setting syncSourceFeedback to opsdba-vbj01-1:27018
2016-10-11T16:42:58.104+0800 I ASIO     [NetworkInterfaceASIO-BGSync-0] Successfully connected to opsdba-vbj01-1:27018


相关文章：

  • 手把手教使用WebStorm搭建ExtJs5开发环境
  • 国内某公有云 linux云主机开机初始化过程分析和他的镜像制作过程
  • [Todo] C++学习资料进度
  • 词法分析器报告
  • httpclient 认证方式访问http api/resutful api并获取json结果
  • 2015年Java开发岗位面试题归类
  • 文件包含漏洞总结
  • 使用Xshell登录AWS的EC2云服务器和开启EC2上允许root+密码方式登录
  • 一次意外的ORA-12520
  • Linux配置SSH公钥认证与Jenkins远程登录进行自动发布
  • 日期格式 CST
  • Android重写getResources规避用户调整系统字体大小影响Android屏幕适配
  • Java.io.DataInputStream.readInt()
  • 第八课：通道抠图1（毛绒玩具）
  • Spring框架总结
  • “寒冬”下的金三银四跳槽季来了，帮你客观分析一下局面
  • 「面试题」如何实现一个圣杯布局？
  • 「前端」从UglifyJSPlugin强制开启css压缩探究webpack插件运行机制
  • 4月23日世界读书日 网络营销论坛推荐《正在爆发的营销革命》
  • ES6 学习笔记（一）let，const和解构赋值
  • gf框架之分页模块(五) - 自定义分页
  • Java 网络编程（2）：UDP 的使用
  • js ES6 求数组的交集，并集，还有差集
  • JS字符串转数字方法总结
  • Node.js 新计划：使用 V8 snapshot 将启动速度提升 8 倍
  • Redux系列x：源码分析
  • vue的全局变量和全局拦截请求器
  • 大型网站性能监测、分析与优化常见问题QA
  • 关于 Cirru Editor 存储格式
  • 欢迎参加第二届中国游戏开发者大会
  • 简析gRPC client 连接管理
  • 每天一个设计模式之命令模式
  • 前端学习笔记之观察者模式
  • 如何抓住下一波零售风口？看RPA玩转零售自动化
  • 找一份好的前端工作，起点很重要
  • 你学不懂C语言，是因为不懂编写C程序的7个步骤 ...
  • #免费 苹果M系芯片Macbook电脑MacOS使用Bash脚本写入(读写)NTFS硬盘教程
  • ( 10 )MySQL中的外键
  • (07)Hive——窗口函数详解
  • (delphi11最新学习资料) Object Pascal 学习笔记---第8章第5节（封闭类和Final方法）
  • (javascript)再说document.body.scrollTop的使用问题
  • （Java岗）秋招打卡！一本学历拿下美团、阿里、快手、米哈游offer
  • （Redis使用系列） Springboot 使用redis实现接口幂等性拦截 十一
  • （博弈 sg入门）kiki's game -- hdu -- 2147
  • （附源码）springboot“微印象”在线打印预约系统 毕业设计 061642
  • (附源码)计算机毕业设计SSM智慧停车系统
  • （离散数学）逻辑连接词
  • （四）【Jmeter】 JMeter的界面布局与组件概述
  • （四）TensorRT | 基于 GPU 端的 Python 推理
  • （心得）获取一个数二进制序列中所有的偶数位和奇数位， 分别输出二进制序列。
  • .NET : 在VS2008中计算代码度量值
  • .NET Entity FrameWork 总结 ，在项目中用处个人感觉不大。适合初级用用，不涉及到与数据库通信。
  • .net 受管制代码
  • .NET下的多线程编程—1-线程机制概述
  • @Controller和@RestController的区别？