cython编译python windows,如何在Windows上使用cython编译init.py文件

悠悠 2023-01-24 02:21 21阅读 0赞

![Image 1][]

When I compile an arbitrary \_\_init\_\_.py file on Windows with setup.py build\_ext --inplace command, it has an unresolvable external symbol error (i.e. "LINK : error LNK2001: An unresolvable external symbol PyInit\_\_\_init\_\_").

The local environment:

python3.7,

Cython 0.29.14,

window10 x64,

Microsoft Visual Studio 2017,

ctest/\_\_init\_\_.py

\# cython: language\_level=3

print('\_\_init\_\_')

setup.py

from distutils.core import setup

from Cython.Build import cythonize

def compile\_code(name, filename):

setup(

name=name,

ext\_modules=cythonize(filename),

)

if \_\_name\_\_ == '\_\_main\_\_':

compile\_code('a', 'ctest/\_\_init\_\_.py')

The information printed by the terminal:

Compiling ctest/\_\_init\_\_.py because it changed.

\[1/1\] Cythonizing ctest/\_\_init\_\_.py

running build\_ext

building 'ctest.\_\_init\_\_' extension

creating build

creating build\\temp.win-amd64-3.7

creating build\\temp.win-amd64-3.7\\Release

creating build\\temp.win-amd64-3.7\\Release\\ctest

C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Id:\\py37\\include -Id:\\py37\\incl

ude "-IC:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\include" "-IC:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.6.1\\include\\um" "-IC:\\Pro

gram Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\ucrt" "-IC:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\shared" "-IC:\\Program Files (x86)\\Windows Kits\\10\\includ

e\\10.0.18362.0\\um" "-IC:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\winrt" "-IC:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\cppwinrt" /Tcctest/\_\_init\_\_

.c /Fobuild\\temp.win-amd64-3.7\\Release\\ctest/\_\_init\_\_.obj

\_\_init\_\_.c

C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTU

AC:NO /LIBPATH:d:\\py37\\Libs /LIBPATH:D:\\ENVS\\cpytrantest\\libs /LIBPATH:D:\\ENVS\\cpytrantest\\PCbuild\\amd64 "/LIBPATH:C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC

\\Tools\\MSVC\\14.16.27023\\lib\\x64" "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.6.1\\lib\\um\\x64" "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.18362.0\\ucrt\\x6

4" "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.18362.0\\um\\x64" /EXPORT:PyInit\_\_\_init\_\_ build\\temp.win-amd64-3.7\\Release\\ctest/\_\_init\_\_.obj /OUT:C:\\Users\\76923\\Deskto

p\\cpythonrecord\\ctest\\\_\_init\_\_.cp37-win\_amd64.pyd /IMPLIB:build\\temp.win-amd64-3.7\\Release\\ctest\\\_\_init\_\_.cp37-win\_amd64.lib

LINK : error LNK2001: An unresolvable external symbol PyInit\_\_\_init\_\_

build\\temp.win-amd64-3.7\\Release\\ctest\\\_\_init\_\_.cp37-win\_amd64.lib : fatal error LNK1120: An external command that cannot be parsed

error: command 'C:\\\\Program Files (x86)\\\\Microsoft Visual Studio\\\\2017\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.16.27023\\\\bin\\\\HostX86\\\\x64\\\\link.exe' failed with exit status 1120

解决方案

Maybe this behaviour might be viewed as a small bug in distutils-package (as pointed out by @DavidW there is this open issue: https://bugs.python.org/issue35893). However, it also shows, that cythonizing/compiling \_\_init\_\_.py isn't very popular and uses some undocumented implementation details which might change in the future, so it could be wiser to refrain from meddling with \_\_init\_\_.py.

But if you must...

When a package is imported explicitly, e.g.

import ctest

or implicitly, e.g.

import ctest.something

The FileFinder will see that a package, and not a module, is imported and will try to load ctest/\_\_init\_\_.py instead of ctest.py (which most likely doesn't exists):

\# Check if the module is the name of a directory (and thus a package).

if cache\_module in cache:

base\_path = \_path\_join(self.path, tail\_module)

for suffix, loader\_class in self.\_loaders:

init\_filename = '\_\_init\_\_' + suffix

full\_path = \_path\_join(base\_path, init\_filename)

if \_path\_isfile(full\_path):

return self.\_get\_spec(loader\_class, fullname, full\_path, \[base\_path\], target)

Used suffix, loader\_class are for loading \_\_init\_\_.so, \_\_init\_\_.py and \_\_init\_\_.pyc in this order (see also this SO-post). This means, \_\_init\_\_.so will be loaded instead of \_\_init\_\_.py if we manage to create one.

While \_\_init\_\_.py is executed, The property \_\_name\_\_ is the name of the package, i.e. ctest in your case, and not \_\_init\_\_ as one might think. Thus, the name of the init-function, Python-interpreter will call when loading the extension \_\_init\_\_.so is PyInit\_ctest in your case (and not PyInit\_\_\_init\_\_ as one might think).

The above explains, why it all works on Linux out-of-the-box. What about Windows?

The loader can only use symbols from a so/dll which aren't hidden. Per default all symbols built with gcc are visible, but not for VisualStudio on Windows - where all symbols are hidden per default (see e.g. this SO-post).

However, the init-function of a C-extension must be visible (and only the init-function) so it can be called with help of the loader - the solution is to export this symbol (i.e. PyInit\_ctest) while linking, in your case it is the wrong /EXPORT:PyInit\_\_\_init\_\_-option for the linker.

The problem can be found in distutils, or more precise in build\_ext-class:

def get\_export\_symbols(self, ext):

"""Return the list of symbols that a shared extension has to

export. This either uses 'ext.export\_symbols' or, if it's not

provided, "PyInit\_" + module\_name. Only relevant on Windows, where

the .pyd file (DLL) must export the module "PyInit\_" function.

"""

initfunc\_name = "PyInit\_" + ext.name.split('.')\[-1\]

if initfunc\_name not in ext.export\_symbols:

ext.export\_symbols.append(initfunc\_name)

return ext.export\_symbols

Here, sadly ext.name has \_\_init\_\_ in it.

From here, one possible solution is easy : to override get\_export\_symbols, i.e. to add the following to your setup.py-file (read on for a even simpler version):

...

from distutils.command.build\_ext import build\_ext

def get\_export\_symbols\_fixed(self, ext):

names = ext.name.split('.')

if names\[-1\] != "\_\_init\_\_":

initfunc\_name = "PyInit\_" + names\[-1\]

else:

\# take name of the package if it is an \_\_init\_\_-file

initfunc\_name = "PyInit\_" + names\[-2\]

if initfunc\_name not in ext.export\_symbols:

ext.export\_symbols.append(initfunc\_name)

return ext.export\_symbols