Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EIP-4844 progress #27047

Closed
26 tasks done
karalabe opened this issue Apr 4, 2023 · 8 comments
Closed
26 tasks done

EIP-4844 progress #27047

karalabe opened this issue Apr 4, 2023 · 8 comments
Labels

Comments

@karalabe
Copy link
Member

karalabe commented Apr 4, 2023

This is a tracking issue for EIP-4844. Some of the issues might get broken down into very low level piecemeal ones that might help review and/or work on a different larger ones. Ideally we'll end up with a batch of PRs people can take a look at to wrap their head around 4844 and it's progress / coding history.

Note, an initial proof of concept was prepared by Coinbase and Optimism at #26283.

@jtraglia
Copy link
Member

jtraglia commented Apr 4, 2023

For KZG implementations, there are two options for Go clients:

  • Go-KZG-4844
    • Pure Go (with gnark-crypto)
    • Supports parallelization
    • Go 1.18 required
    • CGO not required
  • C-KZG-4844 (Go bindings)
    • Core implementation in C (with blst)
    • All single-threaded
    • Go 1.19 required
    • CGO required

Right now, our recommendation is for Go clients to use Go-KZG-4844. While C-KZG-4844 is expected to be slightly faster when run on single-threaded systems, Go-KZG-4844 also provides parallelized functions that can be faster (though less efficient) on systems with multiple threads. Most clients will use C-KZG-4844 bindings, but that's because there isn't a great alternative for them (right now). From a diversity perspective, using a different implementation would be beneficial. Both implementations will be audited and have been differentially fuzzed against each other.

Also, some information about the audit. There will be a one month audit (starting on April 17th 2023) for both of these implementations (and their bindings) by a reputable security firm. For Go-KZG-4844, code from gnark-crypto that it needs but was not audited has been copied into its project so it gets audited.

Benchmarks for Go-KZG-4844

With parallelization enabled on a system with 20 cores:

Benchmark/BlobToKZGCommitment-20         	     307	   3901151 ns/op
Benchmark/ComputeKZGProof-20             	     254	   4697532 ns/op
Benchmark/ComputeBlobKZGProof-20         	     246	   4861995 ns/op
Benchmark/VerifyKZGProof-20              	     858	   1390373 ns/op
Benchmark/VerifyBlobKZGProof-20          	     585	   2032785 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=1)-20         	     585	   2035749 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=2)-20         	     370	   3227315 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=4)-20         	     243	   4920444 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=8)-20         	     145	   8207486 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=16)-20        	      82	  14729597 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=32)-20        	      43	  27678134 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=64)-20        	      21	  53038206 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=1)-20      	     583	   2039326 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=2)-20      	     538	   2224141 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=4)-20      	     501	   2383239 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=8)-20      	     451	   2656815 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=16)-20     	     366	   3253719 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=32)-20     	     187	   6241685 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=64)-20     	     100	  11882722 ns/op

For a better comparision of efficiency, with GOMAXPROCS=1:

Benchmark/BlobToKZGCommitment         	      26	  43713460 ns/op
Benchmark/ComputeKZGProof             	      25	  45990877 ns/op
Benchmark/ComputeBlobKZGProof         	      25	  46085868 ns/op
Benchmark/VerifyKZGProof              	     859	   1390975 ns/op
Benchmark/VerifyBlobKZGProof          	     585	   2035813 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=1)         	     588	   2035962 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=2)         	     355	   3381252 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=4)         	     225	   5297926 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=8)         	     135	   8825641 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=16)        	      74	  15600914 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=32)        	      39	  29332908 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=64)        	      20	  55860546 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=1)      	     584	   2047389 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=2)      	     286	   4179565 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=4)      	     141	   8363224 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=8)      	      72	  16752780 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=16)     	      34	  33594562 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=32)     	      16	  67546948 ns/op
Benchmark/VerifyBlobKZGProofBatchPar(count=64)     	       8	 134386234 ns/op
Benchmarks for C-KZG-4844 (Go bindings)
Benchmark/BlobToKZGCommitment-20         	      28	  40685543 ns/op
Benchmark/ComputeKZGProof-20             	      27	  42668193 ns/op
Benchmark/ComputeBlobKZGProof-20         	      27	  42853123 ns/op
Benchmark/VerifyKZGProof-20              	    1162	   1020716 ns/op
Benchmark/VerifyBlobKZGProof-20          	     742	   1609853 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=1)-20         	     751	   1596980 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=2)-20         	     452	   2648943 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=4)-20         	     260	   4590053 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=8)-20         	     141	   8483093 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=16)-20        	      73	  16198360 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=32)-20        	      37	  31688225 ns/op
Benchmark/VerifyBlobKZGProofBatch(count=64)-20        	      18	  62790266 ns/op

@sauliusgrigaitis
Copy link
Contributor

There is a third option - rust-kzg. It exposes the same C API as c-kzg-4844 so you can use the same Go bindings that are provided with c-kzg-4844. See here for instructions.

@karalabe
Copy link
Member Author

@jtraglia The trusted setup file seems to be slightly different for the two libs: The Go version requires some Lagrange form of G1 points whereas the C does not. Any particular reason for that / not being able to calculate it internally?

@sauliusgrigaitis
Copy link
Contributor

Conversion is expensive. But I'm in favor that all the libs should use exactly the same trusted setup file.

@karalabe
Copy link
Member Author

The trusted setup file in the Go lib has 4096 G2 points but the C lib rejects that because it wants 65 G2 points. Which is correct?

@sauliusgrigaitis
Copy link
Contributor

65 G2 points is more than enough.

@jtraglia
Copy link
Member

jtraglia commented Apr 24, 2023

Hey @karalabe, good question. For both presets, there should only be 65 G2 points. I suspect Go-KZG-4844 has 4096 because that's how the trusted setup was defined initially; about 4 months ago, this PR changed the count from 4096 to 65. Also, you should use the trusted setups (mainnet and minimal) from the consensus-specs.

@lightclient
Copy link
Member

4844 shipped in March 🎊 https://ethereum.org/en/roadmap/dencun/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants