What is Algorand's binary serialization format?

wayneApr 18, 2019

I am looking at this transaction for example

https://algoexplorer.io/tx/IAHIOLPF4274MPUQV4CRGTPL7QWKIHWTQHEXFX2XIGLTUE2EYTCQ

How would I convert FBPLS3VDAG7FU36BSD2WWGLX4QI223G52HKMK7N65NZKCEEOPY6E5R5MIU to a byte stream for the public key bytes?

I am also curious about the tx id IAHIOLPF4274MPUQV4CRGTPL7QWKIHWTQHEXFX2XIGLTUE2EYTCQ and how that was converted from the sha bytes

As a follow on, I could answer my own question if I could inspect the source for myself. Is there any rough ETA date range for making algorand open source?

✔ Solved by vervious in post #2

verviousBenjamin ChanApr 18, 2019

The first one (FBPLS) is an address. An address wraps an ed25519 public key, which is a 32 byte array. The string representation is computed by appending a 4 byte checksum (computed by taking the last 4 bytes of a SHA512/256 digest of the public key) to the public key, giving us a 36 byte array, and then converting this to a base32 string.

An example in the java sdk https://github.com/algorand/java-algorand-sdk/blob/4cb9920148b85501013e4548f2ec4f8afec18843/src/main/java/com/algorand/algosdk/crypto/Address.java#L89. There are equivalent examples in the go-sdk, and js-sdk.
To convert it back to a raw byte eddsa public key, reverse the above. An example is here: https://github.com/algorand/java-algorand-sdk/blob/4cb9920148b85501013e4548f2ec4f8afec18843/src/main/java/com/algorand/algosdk/crypto/Address.java#L58

The second, the tx-id, is the base32 string encoding (stripping padding) of a SHA512/256 digest of a [msg-pack-encoded raw transaction prepended with the prefix bytes(“TX”)]
So we have a raw transaction - then we encode with canonical msg-pack - then we prepend a few bytes corresponding to “TX” - and then we take the SHA512/256 digest, and then we take the base32 representation of the hashed output.

https://github.com/algorand/java-algorand-sdk/blob/4cb9920148b85501013e4548f2ec4f8afec18843/src/main/java/com/algorand/algosdk/account/Account.java#L152

Regarding open source date someone else will be able to answer better than I. For now the sdk code is all open, at least!

wayneApr 18, 2019

Wow this is cool, I am surprised to see this in the client code though, I will definitely be taking a closer look into the clients. Usually they’re pretty dumb, you generally want them to do as little as possible.

I am surprised to see the base32 representation rather than the more common and compact base58.

verviousBenjamin ChanApr 18, 2019

The sdks give us a way to generate keys and sign transactions without running algod and the overhead of an agreement protocol - which is in some ways pretty nice.

I believe base32 is more readable than base64, and both are an rfc standard compared to base58 (which is quite slow to compute, and not as common outside of the bitcoin/ipfs world). You’re right, we certainly aren’t optimizing for compactness here.

wayneApr 18, 2019

It makes sense to me, it is somewhat unique in my experience. Interestingly, you then are implicitly sort of requiring language support so you need to re-implement all of this in each language you want to use. And there are a lot of languages. It looks to me like there is a nice microservice brewing here that could support any language via REST endpoints. It could even be completely stateless.