该方法应用面比较窄,
适用于 : 一主 一备 一投票节点,数据库较大,oplog 比较小,备库需要修复而且主库不能停机的情况.
该方法仅限于学习测试,线上环境慎用.
集群结构： opsdba-vbj01-1:27018 ARBITE opsdba-vbj01-1:27019 PRIMARY opsdba-vbj01-1:27016 SECONDARY
模拟opsdba-vbj01-1:27016 crash ,使用一致性备份搭建备库. 1.在主库,自建role,赋予restore oplog权限. use admin db.runCommand({ createRole: "restoreoplog", privileges: [ { resource: { anyResource: true }, actions: [ "anyAction" ] } ], roles: [] }); db.grantRolesToUser( "root", ["restoreoplog"] ); 2.主库一致性备份. [root@opsdba-vbj01-1 dump]# mongodump -uroot -proot123 --port=27019 --oplog --authenticationDatabase=admin -o all_backup 2016-10-11T15:45:58.019+0800 writing admin.system.users to 2016-10-11T15:45:58.019+0800 done dumping admin.system.users (1 document) 2016-10-11T15:45:58.019+0800 writing admin.system.roles to 2016-10-11T15:45:58.020+0800 done dumping admin.system.roles (1 document) 2016-10-11T15:45:58.020+0800 writing admin.system.version to 2016-10-11T15:45:58.020+0800 done dumping admin.system.version (1 document) 2016-10-11T15:45:58.021+0800 writing test.testData to 2016-10-11T15:45:58.021+0800 writing test.tab to 2016-10-11T15:45:58.190+0800 done dumping test.tab (34056 documents) 2016-10-11T15:46:01.022+0800 [###########.............] test.testData 451185/909000 (49.6%) 2016-10-11T15:46:04.022+0800 [####################....] test.testData 774771/909000 (85.2%) 2016-10-11T15:46:05.024+0800 [########################] test.testData 913877/909000 (100.5%) 2016-10-11T15:46:05.024+0800 done dumping test.testData (913877 documents) 2016-10-11T15:46:05.025+0800 writing captured oplog to 2016-10-11T15:46:05.781+0800 dumped 6470 oplog entries 3.把opsdba-vbj01-1:27016 从SECONDARY 转为单节点空库. /data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf --shutdown 清空data目录,单节点启动: 修改port = 27015, 注释掉replSet /data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf 创建管理员用户: [root@opsdba-vbj01-1 mongodb1]# mongo --port=27015 admin use admin db.createUser( { user: "root", pwd: "root123", roles: [ { role: "root", db: "admin" } ] } ); 自建role,赋予restore oplog权限: [root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin use admin db.runCommand({ createRole: "restoreoplog", privileges: [ { resource: { anyResource: true }, actions: [ "anyAction" ] } ], roles: [] }); db.grantRolesToUser( "root", ["restoreoplog"] ); 4.导入 [root@opsdba-vbj01-1 dump]# mongorestore -uroot -proot123 --port=27015 --authenticationDatabase=admin --oplogReplay --dir=all_backup 2016-10-11T15:59:48.505+0800 building a list of dbs and collections to restore from all_backup dir 2016-10-11T15:59:48.507+0800 reading metadata for test.testData from all_backup/test/testData.metadata.json 2016-10-11T15:59:48.508+0800 reading metadata for test.tab from all_backup/test/tab.metadata.json 2016-10-11T15:59:48.556+0800 restoring test.tab from all_backup/test/tab.bson 2016-10-11T15:59:48.604+0800 restoring test.testData from all_backup/test/testData.bson 2016-10-11T15:59:49.147+0800 restoring indexes for collection test.tab from metadata 2016-10-11T15:59:49.147+0800 finished restoring test.tab (34056 documents) 2016-10-11T15:59:51.507+0800 [####....................] test.testData 18.5 MB/94.1 MB (19.7%) 2016-10-11T15:59:54.507+0800 [##########..............] test.testData 39.8 MB/94.1 MB (42.3%) 2016-10-11T15:59:57.507+0800 [###############.........] test.testData 59.7 MB/94.1 MB (63.5%) 2016-10-11T16:00:00.507+0800 [####################....] test.testData 80.1 MB/94.1 MB (85.1%) 2016-10-11T16:00:03.288+0800 [########################] test.testData 94.1 MB/94.1 MB (100.0%) 2016-10-11T16:00:03.288+0800 restoring indexes for collection test.testData from metadata 2016-10-11T16:00:03.289+0800 finished restoring test.testData (913877 documents) 2016-10-11T16:00:03.289+0800 restoring users from all_backup/admin/system.users.bson 2016-10-11T16:00:03.416+0800 restoring roles from all_backup/admin/system.roles.bson 2016-10-11T16:00:03.466+0800 replaying oplog 2016-10-11T16:00:03.808+0800 done 5.获取最后一个oplog的时间戳 [root@opsdba-vbj01-1 dump]# cd all_backup/ [root@opsdba-vbj01-1 all_backup]# bsondump oplog.bson >oplog.txt [root@opsdba-vbj01-1 all_backup]# tail -1 oplog.txt {"ts":{"$timestamp":{"t":1476171965,"i":805}},"t":{"$numberLong":"2"},"h":{"$numberLong":"6906152948185446623"},"v":2,"op":"i","ns":"test.testData","o":{"_id":{"$oid":"57fc98bddfa99af76706f721"},"x":6470.0,"name":"MACLEAN","name1":"MACLEAN","name2":"MACLEAN","name3":"MACLEAN"}} 6.初始化local库的相关表 [root@opsdba-vbj01-1 mongodb1]# mongo -uroot -proot123 --port=27015 admin use local db.runCommand( { create: "oplog.rs", capped: true, size: (1* 1024 * 1024 * 1024) } ); #数据来自oplog.txt db.oplog.rs.save({"ts" : Timestamp(1476171965, 805),"h" : NumberLong("6906152948185446623")}); db.db.replset.minvalid.save({"ts" : Timestamp(1476171965,805), "t" : NumberLong(2)}); #数据来自主库的数据查询 db.replset.election.save({ "_id" : ObjectId("57fc5ea0cfa6486e03e975d0"), "term" : NumberLong(2), "candidateIndex" : NumberLong(2) }); db.system.replset.save({ "_id" : "myrelset", "version" : 5, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 1, "host" : "opsdba-vbj01-1:27018", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 2, "host" : "opsdba-vbj01-1:27019", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 }, { "_id" : 3, "host" : "opsdba-vbj01-1:27016", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 } ], "settings" : { "chainingAllowed" : true, "heartbeatIntervalMillis" : 2000, "heartbeatTimeoutSecs" : 10, "electionTimeoutMillis" : 10000, "getLastErrorModes" : { }, "getLastErrorDefaults" : { "w" : 1, "wtimeout" : 0 }, "replicaSetId" : ObjectId("57bfdcdcd40cbe4bf173396a") } }); 7.重启 use admin db.shutdownServer(); 修改为原始值. port=27016 replSet 取消注释 启动 /data/mongodb/mongodb/bin/mongod -f /data/mongodb/mongodb1/conf/mongod.cnf 8.检测: 比对行数. db.collection.count() 9.启动日志 2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] MongoDB starting : pid=633 port=27016 dbpath=/data/mongodb/mongodb1/data 64-bit host=opsdba-vbj01-1 2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] db version v3.2.8 2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0 2016-10-11T16:42:51.520+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] allocator: tcmalloc 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] modules: none 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] build environment: 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] distmod: rhel62 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] distarch: x86_64 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] target_arch: x86_64 2016-10-11T16:42:51.521+0800 I CONTROL [initandlisten] options: { config: "/data/mongodb/mongodb1/conf/mongod.cnf", net: { http: { enabled: false }, maxIncomingConnections: 3000, port: 27016, unixDomainSocket: { pathPrefix: "/data/mongodb/mongodb1/data" } }, operationProfiling: { mode: "slowOp", slowOpThresholdMs: 800 }, processManagement: { fork: true, pidFilePath: "/data/mongodb/mongodb1/data/mongod.pid" }, replication: { replSet: "myrelset" }, security: { clusterAuthMode: "keyFile", keyFile: "/data/mongodb/mongodb1/conf/myrelset.keyfile" }, storage: { dbPath: "/data/mongodb/mongodb1/data", directoryPerDB: true, engine: "wiredTiger", journal: { commitIntervalMs: 300, enabled: true }, mmapv1: { nsSize: 32 }, repairPath: "/data/mongodb/mongodb1/data", syncPeriodSecs: 60.0, wiredTiger: { engineConfig: { cacheSizeGB: 1 } } }, systemLog: { destination: "file", path: "/data/mongodb/mongodb1/log/mongod.log", quiet: true, timeStampFormat: "iso8601-local" } } 2016-10-11T16:42:51.521+0800 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] Starting WiredTigerRecordStoreThread local.oplog.rs 2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] The size storer reports that the oplog contains 1 records totaling to 45 bytes 2016-10-11T16:42:51.944+0800 I STORAGE [initandlisten] Scanning the oplog to determine where to place markers for truncation 2016-10-11T16:42:51.982+0800 W STORAGE [initandlisten] Detected configuration for non-active storage engine mmapv1 when current storage engine is wiredTiger 2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] 2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] 2016-10-11T16:42:51.982+0800 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 65535 files. Number of processes should be at least 32767.5 : 0.5 times number of files. 2016-10-11T16:42:52.030+0800 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/mongodb/mongodb1/data/diagnostic.data' 2016-10-11T16:42:52.030+0800 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker 2016-10-11T16:42:52.031+0800 I NETWORK [initandlisten] waiting for connections on port 27016 2016-10-11T16:42:52.066+0800 I REPL [ReplicationExecutor] New replica set config in use: { _id: "myrelset", version: 5, protocolVersion: 1, members: [ { _id: 1, host: "opsdba-vbj01-1:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "opsdba-vbj01-1:27019", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 3, host: "opsdba-vbj01-1:27016", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('57bfdcdcd40cbe4bf173396a') } } 2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] This node is opsdba-vbj01-1:27016 in the config 2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] transition to STARTUP2 2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] Starting replication applier threads 2016-10-11T16:42:52.067+0800 I REPL [ReplicationExecutor] transition to RECOVERING 2016-10-11T16:42:52.072+0800 I REPL [ReplicationExecutor] transition to SECONDARY 2016-10-11T16:42:52.102+0800 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27018 2016-10-11T16:42:52.102+0800 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to opsdba-vbj01-1:27019 2016-10-11T16:42:52.102+0800 I REPL [ReplicationExecutor] Member opsdba-vbj01-1:27018 is now in state SECONDARY 2016-10-11T16:42:52.103+0800 I REPL [ReplicationExecutor] Member opsdba-vbj01-1:27019 is now in state PRIMARY 2016-10-11T16:42:58.069+0800 I REPL [ReplicationExecutor] syncing from: opsdba-vbj01-1:27018 2016-10-11T16:42:58.086+0800 I REPL [SyncSourceFeedback] setting syncSourceFeedback to opsdba-vbj01-1:27018 2016-10-11T16:42:58.104+0800 I ASIO [NetworkInterfaceASIO-BGSync-0] Successfully connected to opsdba-vbj01-1:27018
