Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import of psutil errors with "import _psutil_mswindows .. DLL load failed" #348

Closed
giampaolo opened this issue May 23, 2014 · 50 comments
Closed

Import of psutil errors with "import _psutil_mswindows .. DLL load failed" #348

giampaolo opened this issue May 23, 2014 · 50 comments
Labels
bug critical imported windows

Comments

@giampaolo
Copy link
Owner

From Drydercs...@gmail.com on January 02, 2013 02:56:28

What steps will reproduce the problem?  
1. Install python-3.3.0 and psutil 0.6.1.win32 on windows-XP, 32bit
2. start python cli
3. import psutil 

What is the expected output?  
succesful import of psutil 

What do you see instead?  
>python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (In
tel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\Python33\lib\site-packages\psutil\__init__.py", line 76, in <module>
    import psutil._psmswindows as _psplatform
  File "c:\Python33\lib\site-packages\psutil\_psmswindows.py", line 16, in <modu
le>
    import _psutil_mswindows
ImportError: DLL load failed: The specified procedure could not be found.
>>> 

What version of psutil are you using? What Python version?  
psutil 0.6.1
python 3.3 

On what operating system? Is it 32bit or 64bit version?  
Windows XP, 32bit, SP-3 

Please provide any additional information below.  
I have installed same python 3.3 and psutil 0.6.1 on a windows 7 systems and it works.

Original issue: http://code.google.com/p/psutil/issues/detail?id=348
@giampaolo
Copy link
Owner Author

From g.rodola on January 04, 2013 08:55:32

I don't have time to look into this now but a quick google search suggests this 
a problem common to other projects as well.

@giampaolo
Copy link
Owner Author

From eryksun on March 19, 2013 01:42:42

I checked _psutil_mswindows.pyd with Dependency Walker. It's looking for the 
following functions in Kernel32.dll:

K32EnumProcesses
K32GetMappedFileNameA
K32GetProcessImageFileNameA
K32GetProcessMemoryInfo

These were added to Kernel32 in NT 6.1+, such as Windows 7. If you're using XP 
(NT 5.1) or Vista (NT 6.0), they're instead in Psapi.dll, without the K32 
prefix. The following blog discusses a workaround to bypass the K32
macros by setting PSAPI_VERSION=1: 
http://blogs.msdn.com/b/vcblog/archive/2009/08/27/windows-sdk-v7-0-v7-0a-incompatibility-workaround.aspx
 For now, in Vista I used Visual Studio 2010 Express to build from source.

@giampaolo
Copy link
Owner Author

From g.rodola on March 19, 2013 03:37:17

Thanks for the insight.
Can someone please try the patch in attachment? I'm still unable to reproduce 
this issue.

Attachment: issue348.patch

@giampaolo
Copy link
Owner Author

From g.rodola on March 19, 2013 03:37:40

Thanks for the insight.
Can someone please try the patch in attachment? I'm still unable to reproduce 
this issue.

Attachment: issue348.patch

@giampaolo
Copy link
Owner Author

From eryksun on March 22, 2013 05:53:02

It's already conditionally defined in Psapi.h:

#ifndef PSAPI_VERSION
#if (NTDDI_VERSION >= NTDDI_WIN7)
#define PSAPI_VERSION 2
#else
#define PSAPI_VERSION 1
#endif
#endif

I can successfully build from source on Vista. The issue is with the version 
distributed in the downloads. You have the option to force PSAPI_VERSION=1 when 
building so as to also support XP and Vista in a single build. This incurs a 
performance penalty because the linker uses Psapi.dll instead of the K32 
functions in kernel32.dll. Instead you could instead distribute separate 
versions for XP/Vista and Windows 7+.

@giampaolo
Copy link
Owner Author

From g.rodola on March 22, 2013 11:16:57

You mean performance penalty when compiling?
Distributing different installers for XP/7+ is a non option since we're already 
providing 8 installers for Windows.

@giampaolo
Copy link
Owner Author

From eryksun on March 22, 2013 15:15:34

The old Psapi.dll still exists in 7+. They've just reimplemented it in 
kernel32.dll for performance. You could test linking to kernel32.dll vs 
psapi.dll, but I doubt there's much difference in Python, in which case it 
makes sense to force PSAPI_VERSION=1 for the build that you distribute. You can 
leave it as an option for someone on Windows 7+ building their own copy.

@giampaolo
Copy link
Owner Author

From g.rodola on March 23, 2013 04:22:31

Fine. Can anybody confirm that the patch I attached earlier fixes this problem?

@giampaolo
Copy link
Owner Author

From eryksun on March 23, 2013 09:17:13

For Windows 7, WINVER == _WIN32_WINNT == 0x0601, and the test would be WINVER 
>= 0x0601. 

But why not modify your macro definition in setup.py to ('_WIN32_WINNT', 
'0x501'), which targets XP and implicitly sets the psapi, or instead add 
('PSAPI_VERSION', 1)?

Windows version macros: http://msdn.microsoft.com/en-us/library/aa383745

@giampaolo
Copy link
Owner Author

From g.rodola on March 26, 2013 05:13:34

> For Windows 7, WINVER == _WIN32_WINNT == 0x0601, and 
> the test would be WINVER >= 0x0601. 

Right, thanks. How about now? Can someone try the patch in attachment?

> But why not modify your macro definition in setup.py to 
> ('_WIN32_WINNT', '0x501')

Because we're relying on _WIN32_WINNT in different parts of _psutil_mswindows.c.

Attachment: issue348.patch

@giampaolo
Copy link
Owner Author

From jlo...@gmail.com on March 26, 2013 07:02:28

What did you need this tested on? XP? I don't have an XP box I can test on at 
the moment, just Win 2003 I think.

@giampaolo
Copy link
Owner Author

From g.rodola on March 26, 2013 07:13:03

The original OP's message was talking about Win XP. I have Win XP but I cannot 
reproduce the issue though so basically what I need is someone who does.

@giampaolo
Copy link
Owner Author

From eryksun on March 26, 2013 09:22:48

Regarding the patch, you call EnumProcesses in process_info.c, so you'd have to 
add it there too. Also, you're only defining the _WIN32_WINNT macro in 
setup.py, so you need to test _WIN32_WINNT instead. The value of WINVER gets 
set to _WIN32_WINNT in the headers included by Windows.h. However, rather than 
define this all over the place, you could just do it once in setup.py.

I still don't grok this approach, though. Currently, if someone builds psutil 
himself with SDK 7 (Python 3.3 needs this version), he won't see this extension 
module import problem. Your setup.py sets _WIN32_WINNT to the build system's 
version of NT, which indirectly chooses the correct PSAPI_VERSION. However, if 
he builds an installer on NT 6.1+ using SDK 7, the extension module won't work 
on NT 5.x/6.0 -- as setup.py currently stands. 

It seems the simplest approach is to target the pre-built extension module to 
the widest audience possible. It's not as if an XP/Vista consumer of the 
pre-built 3.3 extension cares about optimizations for a particular _WIN32_WINNT 
version at *build* time. If the build system is NT 6.1+, it's just liable to 
break things for them. You can add an option to override the default 0x0501 
target for the rare user on Windows that actually builds C extensions from source.

@giampaolo
Copy link
Owner Author

From g.rodola on March 26, 2013 09:29:04

I'm not sure I'm completely following you here. My knowledge about Windows, 
especially when C code is involved is rather limited.
If the problem is clear to you and you can reproduce the issue maybe you can 
provide a patch?
The ideal solution would be to fix the problem on all Windows 
versions/compilers transparently without involving a cmdline option for 
setup.py (if that is what you meant in the first place).

Labels: OpSys-Windows

@giampaolo
Copy link
Owner Author

From makdaddy98102@gmail.com on April 29, 2013 12:05:43

We were able to fix this for the one user who was seeing it by copying the two files

_psutil_mswindows.pyc
_psutil_mswindows.pyd

into dist/library/psutil.

See https://groups.google.com/forum/?fromgroups#!topic/psutil/gQa0z872pJI

@giampaolo
Copy link
Owner Author

From g.rodola on May 02, 2013 07:56:48

That's not the best solution though.
I was hoping you could try the patch above, compile psutil, and tell us if that 
fixes the issue.

@giampaolo
Copy link
Owner Author

From g.rodola on May 03, 2013 08:20:48

Labels: -Priority-Medium Priority-High

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 18, 2013 04:44:16

hi! any news on this one? it's still broken in 1.1.3 on my windows xp box (with 
python 3.3). this is a stopper for me. :-(

@giampaolo
Copy link
Owner Author

From g.rodola on November 18, 2013 05:12:03

I proposed a solution ( issue348 .patch) asking if somebody could try the patch 
and report back but no one did.
I cannot know by myself whether issue348 .patch fixes the issue because I 
cannot reproduce it.

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 18, 2013 05:25:21

i'd be happy to test it but i'm unable to compile it myself... could you 
prepare a package so i can just install and test it?

@giampaolo
Copy link
Owner Author

From g.rodola on November 18, 2013 05:31:48

Ok, I can do that later today when I'll be back home.
What Python version? 32 or 64 bit?

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 18, 2013 05:33:17

great! it's python 3.3 (32bit), win xp sp-3 32bit

@giampaolo
Copy link
Owner Author

From g.rodola on November 18, 2013 10:17:49

I currently cannot compile for Python 3.3. 
Here's an installer for Python 2.7 32-bit (please install that one).

Attachment: psutil-2.0.0.win32-py2.7.exe

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 19, 2013 01:59:46

imports fine (and cpu_percent() works) so i'd call it fixed. thanks!

also, if there's nothing else stopping a new release, please do so (our 
company's product just migrated to python3 and this turns out to be a 
show-stopper). thank you very much.

@giampaolo
Copy link
Owner Author

From g.rodola on November 19, 2013 08:12:14

I will definitively release a version soon, although I'm not sure when (I hope 
someday during this week).

@giampaolo
Copy link
Owner Author

From g.rodola on November 20, 2013 12:33:07

Fixed in version 1.2.0 which I released a moment ago.

Status: Fixed
Labels: Milestone-1.2.0

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 04:03:18

just installed the 1.2.0 version under python 3.3 and the import fails just as 
with the old version. :-( ...so this is unfortunately NOT fixed and seems to be 
not only windows xp related but also python 3 related...

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 04:45:26

btw, should you need a test-monkey i'd be happy to run and test anything you 
give me (compiled). ;-)

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 05:05:42

Shit!
What about Python 2.7? Does it fail too?

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 05:28:22

no, the 1.2.0 version imports fine under python 2.7.

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 05:35:15

That doesn't make any sense to me. They were compiled form the same source code base.

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 05:36:01

Can someone else try 1.2.0 on Windows XP?

Status: ReOpened

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 05:46:33

is it compiled using the same compiler (same version)? i thought python3 is 
linked to a newer version of msvcr or something (i'm really not familiar with 
the details)...

@giampaolo
Copy link
Owner Author

From eryksun on November 21, 2013 06:25:32

Refer to comment #13. Your latest build links to kernel32.dll K32EnumProcesses, 
instead of psapi.dll EnumProcesses. K32EnumProcesses does not exist prior to 
Windows 7. 

I don't agree with patching this separately in every compilation unit. Plus, 
even if it is done that way, it should be testing the _WIN32_WINNT version that 
you define in setup.py. WINVER doesn't get updated to _WIN32_WINNT until after 
you include Windows.h.

IMO, just set _WIN32_WINNT to '0x0501' (XP) in setup.py when building for the 
distribution that needs to support XP/Vista. 
https://code.google.com/p/psutil/source/browse/setup.py?name=release-1.2.0#87 
That gives you the old PSAPI without any further mucking around, and it still 
runs in all versions of Windows. Or just unconditionally define PSAPI_VERSION 
to 1 when building for distribution. Either way, someone can always build a 
version targeting a newer Windows version if they so choose.

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 13:45:28

eryksun: thanks for the help and sorry for not considering your comment #13 but 
that's because I don't have a proper understanding of the problem.
Please take a look at file "348.patch" I'm attaching. Is this what you were 
suggesting?

radekpodgorny: attached are two compiled versions for Python 2.7 and 3.3 which 
were produced after applying the patch in attachment. Could you please try them 
and report back?

Labels: -Milestone-1.2.0 Milestone-1.2.1

Attachment: psutil-1.2.1.win32-py2.7.exe psutil-1.2.1.win32-py3.3.exe 348.patch

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 14:40:29

sorry to say bad news but it's still the same: works under 2.7 and fails under 3.3.

(yes, i've checked 1.2.1 was actually installed for both pythons in control panel)

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 14:43:09

p.s.: i see you're probably compiling with mingw. don't you have a tutorial on 
how to set up the compiling environment you have so can play with it myself? thank you!

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 14:46:20

Take a look at the INSTALL file. 
It contains instructions on how to build using mingw32.

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 21, 2013 15:26:58

1) installed mingw (actually found some old install on my system)
2) cloned hg version on psutil
3) built it as recommended (-c mingw32) (had to tweak distutils not to pass 
-mno-cygwin)
4) installed (setup.py install)
5) succesfully imported in python (3.3)

...so i has to be something related to your build system.

@giampaolo
Copy link
Owner Author

From g.rodola on November 21, 2013 15:41:54

I use Visual Studio in order to compile for Python 3.3, that's why.

@giampaolo
Copy link
Owner Author

From eryksun on November 21, 2013 15:54:34

The extension module in the 1.2.1 build is back to using the kernel32 K32* 
functions that aren't available in XP/Vista. In patch 348 if it's not building 
on XP (Vista is NT 6.0), _WIN32_WINNT is set to the build system Windows 
version returned by get_winver(). But if it's building on XP this isn't even an 
issue, and get_winver() would return '0x0501' anyway.

What I proposed was to unconditionally define ('_WIN32_WINNT', '0x0501') or 
('PSAPI_VERSION', 1). At least do this for the builds on PyPI. 

Using the old PSAPI will work fine on new versions of Windows -- at least for 
as long as Microsoft supports psapi.dll. That should be at least until Vista is 
no longer supported.

@giampaolo
Copy link
Owner Author

From g.rodola on November 22, 2013 12:28:03

I added ('PSAPI_VERSION', 1) in setup.py. 
radekpodgorny here's the new exes.

Attachment: psutil-1.2.1.win32-py2.7.exe psutil-1.2.1.win32-py3.3.exe

@giampaolo
Copy link
Owner Author

From radekpod...@gmail.com on November 23, 2013 02:45:47

works! thank you!

@giampaolo
Copy link
Owner Author

From g.rodola on November 25, 2013 10:46:29

Committed in revision 195e1eaf4a9c .

Status: FixedInHG

@giampaolo
Copy link
Owner Author

From g.rodola on November 26, 2013 00:52:33

Closing.

@giampaolo
Copy link
Owner Author

From g.rodola on November 26, 2013 00:52:51

Status: Fixed

@pac2000
Copy link

pac2000 commented Mar 20, 2016

Sorry to bring this up ... yet again
but this is working on win 2003 ... also tried python 3.4 ... no joy
any chance that this is an easy fix ??? sigh
(and no, I cant compile patches)

windows server 2003 R2
enterprise x64 - SP2

psutil 4.1.0
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:40:30) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.

========= RESTART: Z:\common\scheduled tasks\kill stuff\kill_this.py =========

Traceback (most recent call last):
File "Z:\common\scheduled tasks\kill stuff\kill_this.py", line 1, in
import psutil
File "C:\Python27\lib\site-packages\psutil__init__.py", line 124, in
from . import _pswindows as _psplatform
File "C:\Python27\lib\site-packages\psutil_pswindows.py", line 14, in
from . import _psutil_windows as cext
ImportError: DLL load failed: The specified procedure could not be found.

@bitranox
Copy link

same here - can not load psutil 4.1.0 on Windows XP, SP3, Python 3.4.3 :

Microsoft Windows XP [Version 5.1.2600]
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32

import psutil
Traceback (most recent call last):
File "", line 1, in
File "C:\Python34\lib\site-packages\psutil__init__.py", line 124, in 

from . import _pswindows as _psplatform

File "C:\Python34\lib\site-packages\psutil_pswindows.py", line 14, in
from . import _psutil_windows as cext
ImportError: DLL load failed: The specified procedure could not be found.

And I can not update that particular machine, because it is a gateway for an ancient database.
Please Help !

@nsobiratel nsobiratel mentioned this issue Oct 3, 2016
Closed
@maatthc
Copy link

maatthc commented Oct 17, 2016

Guys,
I was having the same issue with a Windows 2003 Server and I solved it temporarily downgrading the PsUtil version to 2.2.0 - it is now running with Python 3.4 .
The file I used to install it can be found here: https://pypi.python.org/simple/psutil/

Cheers,
Alexandre

@giampaolo giampaolo mentioned this issue Oct 17, 2016
Closed
@giampaolo
Copy link
Owner Author

Duplicate of #810

@twangboy twangboy mentioned this issue Mar 11, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug critical imported windows
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@giampaolo @maatthc @bitranox @pac2000