This forum is a read-only archive. Posting, replying and signing in are disabled.

Node synchronization stuck

shixiaofeia
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 1.251263927s","readonly":false,"time":"2021-06-16T08:06:51.161450Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 248.08133ms","readonly":false,"time":"2021-06-16T08:06:52.409985Z"}
{"file":"txSyncer.go","function":"github.com/algorand/go-algorand/rpcs.(*TxSyncer).syncFromClient","level":"info","line":113,"msg":"TxSyncer.Sync: asking client r-al.algorand-mainnet.network:4160 for missing transactions","time":"2021-06-16T08:06:53.486971Z"}
{"file":"httpTxSync.go","function":"github.com/algorand/go-algorand/rpcs.(*HTTPTxSync).Sync","level":"info","line":119,"msg":"http sync from http://r-mt.algorand-mainnet.network:4160/v1/mainnet-v1.0/txsync","time":"2021-06-16T08:06:53.616588Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 379.518455ms","readonly":false,"time":"2021-06-16T08:06:54.222981Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 124.634592ms","readonly":false,"time":"2021-06-16T08:06:55.348002Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 96.297219ms","readonly":false,"time":"2021-06-16T08:06:58.033363Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 521.291277ms","readonly":false,"time":"2021-06-16T08:06:59.555079Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 157.907313ms","readonly":false,"time":"2021-06-16T08:07:00.713473Z"}
{"callee":"github.com/algorand/go-algorand/node/indexer.(*DB).AddBlock.func1","caller":"github.com/algorand/go-algorand/node/indexer/db.go:104","file":"dbutil.go","function":"github.com/algorand/go-algorand/util/db.(*Accessor).atomic","level":"warning","line":344,"msg":"dbatomic: tx surpassed expected deadline by 586.105922ms","readonly":false,"time":"2021-06-16T08:07:02.299973Z"}
{"file":"bootstrap.go","function":"github.com/algorand/go-algorand/tools/network.ReadFromSRV","level":"info","line":43,"msg":"ReadFromBootstrap: DNS LookupSRV failed when using system resolver: no signature in DNS response for _archive._tcp.mainnet.algorand.network","time":"2021-06-16T08:07:05.756689Z"}
{"event":"Disconnected","file":"wsNetwork.go","function":"github.com/algorand/go-algorand/network.(*WebsocketNetwork).removePeer","level":"info","line":2166,"local":"http:","messageDelay":434541596,"msg":"Peer r-mn.algorand-mainnet.network:4160 disconnected: CliqueResolving","name":"","remote":"r-mn.algorand-mainnet.network:4160","time":"2021-06-16T08:07:05.761424Z"}
{"details":{"Address":"35.203.61.15","HostName":"50f2cb54-5fe6-4c0a-af79-19c009e01024:mtlg-alg-mainrelay-2.prod.purestake.tech","Incoming":false,"InstanceName":"JoNfl9BMlVQVwiDf","Endpoint":"r-mn.algorand-mainnet.network:4160","MessageDelay":434541596,"Reason":"CliqueResolving"},"file":"telemetry.go","function":"github.com/algorand/go-algorand/logging.(*telemetryState).logTelemetry","instanceName":"5hjo7Sx15JT2sQbg","level":"info","line":212,"msg":"/Network/DisconnectPeer","name":"","session":"","time":"2021-06-16T08:07:05.761536Z"}
{"event":"ConnectedOut","file":"wsNetwork.go","function":"github.com/algorand/go-algorand/network.(*WebsocketNetwork).tryConnect","level":"info","line":2078,"local":"http:","msg":"Made outgoing connection to peer r-br.algorand-mainnet.network:4160","name":"","remote":"r-br.algorand-mainnet.network:4160","time":"2021-06-16T08:07:06.144993Z"}
{"details":{"Address":"3.85.202.128","HostName":"eabea902-38b0-42b2-9507-f29084084772:bk5aot2cm1jn68pjjb7g.bdnodes.net","Incoming":false,"InstanceName":"ippUdpI3mexf5i2Z","Endpoint":"r-br.algorand-mainnet.network:4160"},"file":"telemetry.go","function":"github.com/algorand/go-algorand/logging.(*telemetryState).logTelemetry","instanceName":"5hjo7Sx15JT2sQbg","level":"info","line":212,"msg":"/Network/ConnectPeer","name":"","session":"","time":"2021-06-16T08:07:06.145097Z"}

tsachi

@shixiaofeia,

From the above log, it looks like you enabled the algod internal indexer. Is that correct ?
I suspect that this is the reason your node is catching up so slowly.

The internal indexer is not as performant as the standalone indexer, which is using a Postgres compatible database.

In case you don’t need the internal indexer, please make sure to turn it off in the config.json file.

shixiaofeia

I opened Archival, IsIndexerActive and EndpointAddress in the configuration file, because I want to keep all the ledgers for easy query, and I use Postgres for indexer query, if I want to close the internal indexer and keep the Postgres indexer, I What changes should be made to the configuration file?

shixiaofeia

OK, you have modified it as you said, I will continue to look at it synchronously, thank you for your help

tsachi

One more thing that wan’t mentioned above, is that after making changes to the config.json file, you’ll need to restart algod. You can do that by goal node restart -d <data dir>, by doing stop/start or rebooting your computer.

The config.json is loaded into memory only on startup, and is not being monitored/tested/loaded after that time.

shixiaofeia

I encountered this problem after restarting, but fortunately, the node has now caught up with the synchronization, maybe it was only accidental, and it has been unable to catch up before.