We were unable to access the server hosting the explorer when it went down but the API service was fine. All good now

We were unable to access the server hosting the explorer when it went down but the API service was fine. All good now

This is what the node is responding meaning we have seen the transaction, good luck getting it in a block!

Regarding the block headers, yes! You need the nodes but for most needs you either need to query an overlay services or query a node providing an overlay service(because that is not the role of a node unless you develop it as an extra service)

Exactly! Disregarding the role of indexers for a user friendly experience is far from a practical solution. In theory you can be restrict about the SPV, users being accountable for their private keys/transactions, etc. but in action you need to use other third parties if you don’t want to ruin the UX. Best you can do is not to rely on one single service. Anyone thinking otherwise is dreaming and can’t deliver what they promise.

Yes it was late April, and we did 152 millions transactions with a 2000+ sustained TPS(9000 TPS at peak)

Congrats on the upgrades, It looks awesome! Always a pleasure to see how you are pushing things forward👏

I recommend checking out the UTXO manager, there is a high chance that things fall apart at that point(if you haven’t checked already of course). We have seen many dropped transactions due to bad UTXO management(using one indexer to fetch the UTXOs which sometimes are already spent and another to broadcast or as a load balance). If possible, try using Bitails to fetch UTXOs and see how that makes a difference(or doesn’t).

One particular indexer full of unknown transactions(seen by no peers) was WoC. We know that because we were asked to rebroadcast the transactions but the inputs were already spent.

11500 TPS in syncing mode which means with a block with 100000 txs, takes 9 seconds to be indexed and added to the database.

Actually it was the indexers problem in providing a UTXO that the nodes considered spent and rejected the transaction. In this case, the indexer’s mempool kept the tx and updated the UTXO-Set based on accepting the tx and you can see how messy this gets from here.

Some pf those transactions sitting on one single indexer were not being mine simply because they were invalid due to invalid UTXO usage(already spent, parent invalid, etc.)

We broadcasts transactions periodically till they are mined, but yesterday we had some issues regarding this which should be resolved now. That being said, if you are using another indexer as UTXO management, there is always a possibility that this happens since some indexers are not providing a clean and updated UTXO set and your tx gets dropped due to network policies.

Have you noticed some miners not broadcasting all the transactions right after they receive them from clients?(seeing some transactions in a block from other miners which have not been propagated before)

This is because the miner didn’t put any transaction in the blockchain other than its coinbase transaction.

Stopped the test to change up the broadcaster service. Will continue again today or tomorrow.