Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python3 startup time approaching 500ms #1847

Open
heapcrash opened this issue Mar 9, 2021 · 1 comment
Open

Python3 startup time approaching 500ms #1847

heapcrash opened this issue Mar 9, 2021 · 1 comment

Comments

@heapcrash
Copy link
Collaborator

heapcrash commented Mar 9, 2021

It looks like we've regressed pretty far on import times for Python3.

$ hyperfine 'python -c "from pwn import *"'
Benchmark #1: python -c "from pwn import *"
  Time (mean ± σ):     455.5 ms ± 181.3 ms    [User: 201.0 ms, System: 219.1 ms]
  Range (min … max):   385.5 ms … 970.9 ms    10 runs

The most offending modules can be seen with the following command. It looks like we can shave off a lot of time by deferring imports of requests, site, urllib3, and ssl. We should investigate more places that we can defer imports until they're actually needed.

python3 -X importtime -c 'from pwn import *' |& awk '{print $5 "|" $7}' | sort -n | tail -20
36699|urllib3
37312|pwnlib.util
38026|pwnlib.constants
38644|pwnlib.elf.corefile
38938|pwnlib.elf
39584|pwnlib.shellcraft
42324|OpenSSL.crypto
43896|OpenSSL
43917|OpenSSL.SSL
47755|requests.compat
47956|requests._internal_utils
52052|requests.utils
63720|urllib3.contrib.pyopenssl
84509|site
170627|pwnlib
177054|requests
353090|pwn.toplevel
366794|pwn
@Arusekk
Copy link
Member

Arusekk commented Mar 9, 2021

Requests could be a proxy object that would import the actual module on demand (pwntools does not even use it anywhere, but exports it in the namespace on a "you would need it anyway" basis). But SSL imports could be deferred too. Also, rpyc is a time consumer, and is only used by object-oriented GDB procedures. See #1753 for some work already done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants