Merge #159: Sync Upstream

b7ebe64 Test APIs of funcs that need an ecmult_gen ctx with static ctx (Jonas Nick) e82144e Fixup skew before global Z fixup (Peter Dettman) 40b624c Add tests for _gej_cmov (Peter Dettman) 8c13a9b ECDH skews by 0 or 1 (Peter Dettman) 1515099 Simpler and faster ecdh skew fixup (Peter Dettman) 3d7cbaf tests: Fix test whose result is implementation-defined (Tim Ruffing) 77a1975 Use xoshiro256++ PRNG instead of RFC6979 in tests (Pieter Wuille) 5f2efe6 secp256k1_testrand_int(2**N) -> secp256k1_testrand_bits(N) (Pieter Wuille) 3ed0d02 doc: add CHANGELOG template (Jonas Nick) 6f42dc1 doc: add release_process.md (Jonas Nick) 0bd3e42 build: set library version to 0.0.0 explicitly (Jonas Nick) b4b02fd build: change libsecp version from 0.1 to 0.1.0-pre (Jonas Nick) 05e049b ecmult: move `_ecmult_odd_multiples_table_globalz_windowa` (siv2r) b4ac1a1 ci: Run valgrind/memcheck tasks with 2 CPUs (Tim Ruffing) e70acab ci: Use Cirrus "greedy" flag to use idle CPU time when available (Tim Ruffing) d07e301 ci: Update brew on macOS (Tim Ruffing) 22382f0 ci: Test different ecmult window sizes (Tim Ruffing) 26a022a ci: Remove STATICPRECOMPUTATION (Tim Ruffing) 10461d8 precompute_ecmult: Always compute all tables up to default WINDOW_G (Tim Ruffing) 1287786 doc: Add comment to top of field_10x26_impl.h (Elliott Jin) 58da5bd doc: Fix upper bounds + cleanup in field_5x52_impl.h comment (Elliott Jin) 22d25c8 Add another ecmult_multi test (Pieter Wuille) 515e795 Improve checks at top of _fe_negate methods (Peter Dettman) e05da9e Fix c++ build (Pieter Wuille) c45386d Cleanup preprocessor indentation in precompute{,d}_ecmult{,_gen} (Pieter Wuille) 19d96e1 Split off .c file from precomputed_ecmult.h (Pieter Wuille) 1a6691a Split off .c file from precomputed_ecmult_gen.h (Pieter Wuille) bb36331 Simplify precompute_ecmult_print_* (Pieter Wuille) 38cd84a Compute ecmult tables at runtime for tests_exhaustive (Pieter Wuille) e458ec2 Move ecmult table computation code to separate file (Pieter Wuille) fc1bf9f Split ecmult table computation and printing (Pieter Wuille) 31feab0 Rename function secp256k1_ecmult_gen_{create_prec -> compute}_table (Pieter Wuille) 725370c Rename ecmult_gen_prec -> ecmult_gen_compute_table (Pieter Wuille) 075252c Rename ecmult_static_pre_g -> precomputed_ecmult (Pieter Wuille) 7cf47f7 Rename ecmult_gen_static_prec_table -> precomputed_ecmult_gen (Pieter Wuille) f95b810 Rename gen_ecmult_static_pre_g -> precompute_ecmult (Pieter Wuille) bae7768 Rename gen_ecmult_gen_static_prec_table -> precompute_ecmult_gen (Pieter Wuille) 7dfcece build: Remove #undef hack for ASM in the precomputation programs (Tim Ruffing) bb36fe9 ci: Test `make precomp` (Tim Ruffing) d94a37a build: Remove CC_FOR_BUILD stuff (Tim Ruffing) ad63bb4 build: Prebuild and distribute ecmult_gen table (Tim Ruffing) ac49361 prealloc: Get rid of manual memory management for prealloc contexts (Tim Ruffing) 6573c08 ecmult_gen: Tidy precomputed file and save space (Tim Ruffing) 5eba83f ecmult_gen: Precompute tables for all values of ECMULT_GEN_PREC_BITS (Tim Ruffing) fdb33dd refactor: Make PREC_BITS a parameter of ecmult_gen_build_prec_table (Tim Ruffing) a4875e3 refactor: Move default callbacks to util.h (Tim Ruffing) 4c94c55 doc: Remove obsolete hint for valgrind stack size (Tim Ruffing) 5106226 exhaustive_tests: Fix with ecmult_gen table with custom generator (Tim Ruffing) e1a7653 refactor: Make generator a parameter of ecmult_gen_create_prec_table (Tim Ruffing) 9ad09f6 refactor: Rename program that generates static ecmult_gen table (Tim Ruffing) 8ae18f1 refactor: Rename file that contains static ecmult_gen table (Tim Ruffing) 00d2fa1 ecmult_gen: Make code consistent with comment (Tim Ruffing) 3b0c218 ecmult_gen: Simplify ecmult_gen context after making table static (Tim Ruffing) e43ba02 refactor: Decouple table generation and ecmult_gen context (Tim Ruffing) 22dc2c0 ecmult_gen: Move table creation to new file and force static prec (Tim Ruffing) 099bad9 Comment and check a parameter for inf in secp256k1_ecmult_const. (Russell O'Connor) 6c0be85 Verify that secp256k1_ge_set_gej_zinv does not operate on infinity. a->x and a->y should not be used if the infinity flag is set. (Russell O'Connor) 5eb519e ci: reduce TEST_ITERS in memcheck run (Pieter Wuille) e2cf773 Test ecmult functions for all i*2^j for j=0..255 and odd i=1..255. (Pieter Wuille) c0cd7de build: add -no-undefined to libtool LDFLAGS (fanquake) fe32a79 build: pass win32-dll to LT_INIT (fanquake) 7c7ce87 build: Add a check that Valgrind actually supports a host platform (Hennadii Stepanov) 592661c ci: move test environment variable declaration to .cirrus.yml (siv2r) dcbe84b bench: add --help option to bench. (siv2r) 2b7c749 build: replace backtick command substitution with $() (fanquake) 60bf889 ecmult: fix definition of STRAUSS_SCRATCH_OBJECTS (Jonas Nick) 214042a build: don't append valgrind CPPFLAGS if not installed (fanquake) 812ff5c doc: remove use of 0xa0 "no break space" (fanquake) dc9b685 doc: Minor fixes in safegcd_implementation.md (Elliott Jin) 2332975 Fix typos (Dimitris Apostolou) 72de135 ci: Enable -g if we set CFLAGS manually (Tim Ruffing) 16d1322 refactor: Use (int)&(int) in boolean context to avoid compiler warning (MarcoFalke) 3b157c4 doc: Suggest keys.openpgp.org as keyserver in SECURITY.md (Tim Ruffing) 73a7472 doc: Replace apoelstra's GPG key by jonasnick's GPG key (Tim Ruffing) af6abcb Make bench support selecting which benchmarks to run (Pieter Wuille) 9f56bdf Merge bench_schnorrsig into bench (Pieter Wuille) 3208557 Merge bench_recover into bench (Pieter Wuille) 855e18d Merge bench_ecdh into bench (Pieter Wuille) 2a7be67 Combine bench_sign and bench_verify into single bench (Pieter Wuille) 5324f89 Make aux_rnd32==NULL behave identical to 0x0000..00. (Pieter Wuille) 2888640 VERIFY_CHECK precondition for secp256k1_fe_set_int. (Russell O'Connor) d49011f Make _set_fe_int( . , 0 ) set magnitude to 0 (Tim Ruffing) 23e2f66 bench: don't return 1 in have_flag() if argc = 1 (Jonas Nick) 96b1ad2 bench_ecmult: improve clarity of output (Jonas Nick) b4b1306 create csv file from the benchmark output (siv2r) 26a255b Shared benchmark format for command line and CSV outputs (siv2r) 044d956 Fix G.y parity in sage code (Pieter Wuille) b53e0cd Avoid overly-wide multiplications (Peter Dettman) 9be7b0f Avoid computing out-of-bounds pointer. (Tim Ruffing) bc08599 Remove OpenSSL testing support (Pieter Wuille) db4667d Make aux_rand32 arg to secp256k1_schnorrsig_sign const (Pieter Wuille) 189f6bc Fix unused parameter warnings when building without VERIFY (Jonas Nick) d439937 tests: remove `secp256k1_fe_verify` from tests.c and modify `secp256k1_fe_from_storage` to call `secp256k1_fe_verify` (siv2r) Pull request description: [bitcoin-core/secp256k1#986]: tests: remove `secp256k1_fe_verify` from tests.c and modify `_fe_from_storage` to call `_fe_verify` [bitcoin-core/secp256k1#987]: Fix unused parameter warnings when building without VERIFY [bitcoin-core/secp256k1#966]: Make aux_rand32 arg to secp256k1_schnorrsig_sign const [bitcoin-core/secp256k1#983]: [RFC] Remove OpenSSL testing support [bitcoin-core/secp256k1#952]: Avoid computing out-of-bounds pointer. [bitcoin-core/secp256k1#810]: Avoid overly-wide multiplications in 5x52 field mul/sqr [bitcoin-core/secp256k1#996]: Fix G.y parity in sage code [bitcoin-core/secp256k1#989]: Shared benchmark format for command line and CSV outputs [bitcoin-core/secp256k1#999]: bench_ecmult: improve clarity of output [bitcoin-core/secp256k1#943]: VERIFY_CHECK precondition for secp256k1_fe_set_int. [bitcoin-core/secp256k1#1002]: Make aux_rnd32==NULL behave identical to 0x0000..00. [bitcoin-core/secp256k1#991]: Merge all "external" benchmarks into a single bench binary [bitcoin-core/secp256k1#1007]: doc: Replace apoelstra's GPG key by jonasnick's GPG key [bitcoin-core/secp256k1#1009]: refactor: Use (int)&(int) in boolean context to avoid compiler warning [bitcoin-core/secp256k1#1011]: ci: Enable -g if we set CFLAGS manually [bitcoin-core/secp256k1#1012]: Fix typos [bitcoin-core/secp256k1#1010]: doc: Minor fixes in safegcd_implementation.md [bitcoin-core/secp256k1#1020]: doc: remove use of <0xa0> "no break space" [bitcoin-core/secp256k1#1019]: build: don't append valgrind CPPFLAGS if not installed (macOS) [bitcoin-core/secp256k1#1004]: ecmult: fix definition of STRAUSS_SCRATCH_OBJECTS [bitcoin-core/secp256k1#1025]: build: replace backtick command substitution with $() [bitcoin-core/secp256k1#1008]: bench.c: add `--help` option and ci: move env variables [bitcoin-core/secp256k1#1027]: build: Add a check that Valgrind actually supports a host platform [bitcoin-core/secp256k1#1022]: build: Windows DLL additions [bitcoin-core/secp256k1#920]: Test all ecmult functions with many j*2^i combinations [bitcoin-core/secp256k1#942]: Verify that secp256k1_ge_set_gej_zinv does not operate on infinity. [bitcoin-core/secp256k1#988]: Make signing table fully static [bitcoin-core/secp256k1#1042]: Follow-ups to making all tables fully static [bitcoin-core/secp256k1#816]: Improve checks at top of _fe_negate methods [bitcoin-core/secp256k1#1044]: Add another ecmult_multi test [bitcoin-core/secp256k1#1030]: doc: Fix upper bounds + cleanup in field_5x52_impl.h comment [bitcoin-core/secp256k1#1047]: ci: Various improvements [bitcoin-core/secp256k1#1053]: ecmult: move `_ecmult_odd_multiples_table_globalz_windowa` [bitcoin-core/secp256k1#964]: Add release-process.md [bitcoin-core/secp256k1#1052]: Use xoshiro256++ instead of RFC6979 for tests [bitcoin-core/secp256k1#1054]: tests: Fix test whose result is implementation-defined [bitcoin-core/secp256k1#1029]: Simpler and faster ecdh skew fixup This PR can be recreated with `./contrib/sync-upstream.sh range a1102b1`. ACKs for top commit: apoelstra: utACK b7ebe64 real-or-random: ACK b7ebe64 diff looks good. tested on my machine, also on valgrind. Tree-SHA512: 8b01347bbb9ac35cb93df628eaaf2a997fc8182046588bccc48a0623e9595d40cad2d46102a9c62c819ff77069331f344361138fd8ad0afc81bba9c1690bb541
BlockstreamResearch · Jan 5, 2022 · 21e2d65 · 21e2d65
2 parents b220661 + b7ebe64
commit 21e2d65
Show file tree

Hide file tree

Showing 69 changed files with 11,649 additions and 1,928 deletions.
diff --git a/.cirrus.yml b/.cirrus.yml
@@ -4,10 +4,10 @@ env:
   # Specific warnings can be disabled with -Wno-error=foo.
   # -pedantic-errors is not equivalent to -Werror=pedantic and thus not implied by -Werror according to the GCC manual.
   WERROR_CFLAGS: -Werror -pedantic-errors
-  MAKEFLAGS: -j2
+  MAKEFLAGS: -j4
   BUILD: check
   ### secp256k1 config
-  STATICPRECOMPUTATION: yes
+  ECMULTWINDOW: auto
   ECMULTGENPRECISION: auto
   ASM: no
   WIDEMUL: auto
@@ -24,9 +24,9 @@ env:
   MUSIG: no
   ECDSAADAPTOR: no
   ### test options
-  TEST_ITERS:
+  SECP256K1_TEST_ITERS:
   BENCH: yes
-  BENCH_ITERS: 2
+  SECP256K1_BENCH_ITERS: 2
   CTIMETEST: yes
 
 cat_logs_snippet: &CAT_LOGS
@@ -55,14 +55,19 @@ merge_base_script_snippet: &MERGE_BASE
     - git config --global user.name "ci"
     - git merge FETCH_HEAD  # Merge base to detect silent merge conflicts
 
-task:
-  name: "x86_64: Linux (Debian stable)"
+linux_container_snippet: &LINUX_CONTAINER
   container:
     dockerfile: ci/linux-debian.Dockerfile
     # Reduce number of CPUs to be able to do more builds in parallel.
     cpu: 1
+    # Gives us more CPUs for free if they're available.
+    greedy: true
     # More than enough for our scripts.
     memory: 1G
+
+task:
+  name: "x86_64: Linux (Debian stable)"
+  << : *LINUX_CONTAINER
   matrix: &ENV_MATRIX
     - env: {WIDEMUL:  int64,  RECOVERY: yes}
     - env: {WIDEMUL:  int64,                 ECDH: yes, EXPERIMENTAL: yes, SCHNORRSIG: yes, ECDSA_S2C: yes,  RANGEPROOF: yes, WHITELIST: yes, GENERATOR: yes, MUSIG: yes, ECDSAADAPTOR: yes}
@@ -71,12 +76,11 @@ task:
     - env: {WIDEMUL: int128,                 ECDH: yes, EXPERIMENTAL: yes, SCHNORRSIG: yes, ECDSA_S2C: yes, RANGEPROOF: yes, WHITELIST: yes, GENERATOR: yes, MUSIG: yes, ECDSAADAPTOR: yes}
     - env: {WIDEMUL: int128,  ASM: x86_64}
     - env: {                  RECOVERY: yes,            EXPERIMENTAL: yes, SCHNORRSIG: yes, ECDSA_S2C: yes, RANGEPROOF: yes, WHITELIST: yes, GENERATOR: yes, MUSIG: yes, ECDSAADAPTOR: yes}
-    - env: {                  STATICPRECOMPUTATION: no}
     - env: {BUILD: distcheck, WITH_VALGRIND: no, CTIMETEST: no, BENCH: no}
     - env: {CPPFLAGS: -DDETERMINISTIC}
     - env: {CFLAGS: -O0, CTIMETEST: no}
-    - env: { ECMULTGENPRECISION: 2 }
-    - env: { ECMULTGENPRECISION: 8 }
+    - env: { ECMULTGENPRECISION: 2, ECMULTWINDOW: 2 }
+    - env: { ECMULTGENPRECISION: 8, ECMULTWINDOW: 4 }
   matrix:
     - env:
         CC: gcc
@@ -89,10 +93,7 @@ task:
 
 task:
   name: "i686: Linux (Debian stable)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     HOST: i686-linux-gnu
     ECDH: yes
@@ -147,8 +148,9 @@ task:
   ##   - rm /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
   ##
   brew_valgrind_pre_script:
+    - brew update
     - brew config
-    - brew tap --shallow LouisBrunner/valgrind
+    - brew tap LouisBrunner/valgrind
     # Fetch valgrind source but don't build it yet.
     - brew fetch --HEAD LouisBrunner/valgrind/valgrind
   brew_valgrind_cache:
@@ -178,13 +180,10 @@ task:
 
 task:
   name: "s390x (big-endian): Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-s390x
-    TEST_ITERS: 16
+    SECP256K1_TEST_ITERS: 16
     HOST: s390x-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
@@ -207,13 +206,10 @@ task:
 
 task:
   name: "ARM32: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-arm
-    TEST_ITERS: 16
+    SECP256K1_TEST_ITERS: 16
     HOST: arm-linux-gnueabihf
     WITH_VALGRIND: no
     ECDH: yes
@@ -231,13 +227,10 @@ task:
 
 task:
   name: "ARM64: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-aarch64
-    TEST_ITERS: 16
+    SECP256K1_TEST_ITERS: 16
     HOST: aarch64-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
@@ -252,13 +245,10 @@ task:
 
 task:
   name: "ppc64le: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-ppc64le
-    TEST_ITERS: 16
+    SECP256K1_TEST_ITERS: 16
     HOST: powerpc64le-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
@@ -273,13 +263,10 @@ task:
 
 task:
   name: "x86_64 (mingw32-w64): Windows (Debian stable, Wine)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: wine64-stable
-    TEST_ITERS: 16
+    SECP256K1_TEST_ITERS: 16
     HOST: x86_64-w64-mingw32
     WITH_VALGRIND: no
     ECDH: yes
@@ -295,10 +282,7 @@ task:
 # Sanitizers
 task:
   timeout_in: 120m
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 2G
+  << : *LINUX_CONTAINER
   env:
     ECDH: yes
     RECOVERY: yes
@@ -311,30 +295,31 @@ task:
     MUSIG: yes
     ECDSAADAPTOR: yes
     CTIMETEST: no
-    EXTRAFLAGS: "--disable-openssl-tests"
   matrix:
     - name: "Valgrind (memcheck)"
+      container:
+        cpu: 2
       env:
         # The `--error-exitcode` is required to make the test fail if valgrind found errors, otherwise it'll return 0 (https://www.valgrind.org/docs/manual/manual-core.html)
         WRAPPER_CMD: "valgrind --error-exitcode=42"
-        TEST_ITERS: 8
+        SECP256K1_TEST_ITERS: 2
     - name: "UBSan, ASan, LSan"
+      container:
+        memory: 2G
       env:
-        CFLAGS: "-fsanitize=undefined,address"
-        CFLAGS_FOR_BUILD: "-fsanitize=undefined,address"
+        CFLAGS: "-fsanitize=undefined,address -g"
         UBSAN_OPTIONS: "print_stacktrace=1:halt_on_error=1"
         ASAN_OPTIONS: "strict_string_checks=1:detect_stack_use_after_return=1:detect_leaks=1"
         LSAN_OPTIONS: "use_unaligned=1"
-        TEST_ITERS: 32
+        SECP256K1_TEST_ITERS: 32
   # Try to cover many configurations with just a tiny matrix.
   matrix:
     - env:
         ASM: auto
-        STATICPRECOMPUTATION: yes
     - env:
         ASM: no
-        STATICPRECOMPUTATION: no
         ECMULTGENPRECISION: 2
+        ECMULTWINDOW: 2
   matrix:
     - env:
         CC: clang
@@ -348,15 +333,12 @@ task:
 
 task:
   name: "C++ -fpermissive"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     # ./configure correctly errors out when given CC=g++.
     # We hack around this by passing CC=g++ only to make.
     CC: gcc
-    MAKEFLAGS: -j2 CC=g++ CFLAGS=-fpermissive
+    MAKEFLAGS: -j4 CC=g++ CFLAGS=-fpermissive\ -g
     WERROR_CFLAGS:
     EXPERIMENTAL: yes
     ECDH: yes

diff --git a/.gitattributes b/.gitattributes
@@ -1 +1,2 @@
-src/ecmult_static_pre_g.h linguist-generated
+src/precomputed_ecmult.c linguist-generated
+src/precomputed_ecmult_gen.c linguist-generated
diff --git a/.gitignore b/.gitignore
@@ -1,21 +1,17 @@
-bench_inv
-bench_ecdh
+bench
 bench_ecmult
 bench_generator
 bench_rangeproof
-bench_schnorrsig
-bench_sign
-bench_verify
-bench_recover
 bench_internal
 tests
 exhaustive_tests
-gen_context
-gen_ecmult_static_pre_g
+precompute_ecmult_gen
+precompute_ecmult
 valgrind_ctime_test
 *.exe
 *.so
 *.a
+*.csv
 !.gitignore
 
 Makefile
@@ -47,7 +43,6 @@ coverage.*.html
 
 src/libsecp256k1-config.h
 src/libsecp256k1-config.h.in
-src/ecmult_static_context.h
 build-aux/config.guess
 build-aux/config.sub
 build-aux/depcomp