Install environment for Windows #2753
Replies: 7 comments
-
@shliang0603, we have not tried, but if you succeed, please publish a tutorial! |
Beta Was this translation helpful? Give feedback.
-
Is there no Windows-compatible MPI that allows for the mpicxx -show (or any mpicxx command) to be ran? I have been running into a multitude of issues while trying to install horovod on my Windows 10 OS. Currently, I have MSMPI (Microsoft's MPI application) but it seems there are some compatibility issues when installing horovod (through the pip install horovod command) from the command prompt (the only way, I believe, to do so) |
Beta Was this translation helpful? Give feedback.
-
I successfully built and ran Horovod with PyTorch on Windows with IntelMPI. You will have to set HOROVOD_MPICXX_SHOW to 'mpicxx.bat -show' and comment out a first couple lines in mpicc.bat file. There would be more problems down the line, though :) |
Beta Was this translation helpful? Give feedback.
-
Can you explain better what does it mean "comment out a first couple lines in mpicc.bat file." which ones? Thanks |
Beta Was this translation helpful? Give feedback.
-
Well, my understanding is that horovod using "mpicxx -show" command to find out where the MPI include and lib folders are. If you look inside mpicc.bat that comes from IntelMPI, you will see that the first thing it does is talks about itself:
Horovod cannot process those lines, so they need to be commented out, so that the output is only like
I only got horovod running on CPU on Windows. I haven't figured out the way to run it on GPU yet.. |
Beta Was this translation helpful? Give feedback.
-
Thanks for your reply, I can't install anyway but It's a step towards salvation 😂😂 (maybe I will try only for CPU). |
Beta Was this translation helpful? Give feedback.
-
I have attempted to use IntelMPI instead of OpenMPI on windows however I am still getting the same error. However when I try to do pip install horovod, I still get this error: During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Does the horovor support installing in windows 10 environment ?
I Install in my windows 10 PC, but return the error for following:
C:\Users\93176>pip install -i https://pypi.tuna.tsinghua.edu.cn/simple horovod
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting horovod
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/17/1a/3e836e3f659d3b331afcb679f84c46c8d489c2b8c21b1d85a99ac31d3480/horovod-0.15.2.tar.gz (119kB)
100% |████████████████████████████████| 122kB 1.1MB/s
Requirement already satisfied: cffi>=1.4.0 in d:\anaconda_install\lib\site-packages (from horovod) (1.11.5)
Requirement already satisfied: pycparser in d:\anaconda_install\lib\site-packages (from cffi>=1.4.0->horovod) (2.18)
Building wheels for collected packages: horovod
Running setup.py bdist_wheel for horovod ... error
Complete output from command d:\anaconda_install\python.exe -u -c "import setuptools, tokenize;file='C:\Users\93176\AppData\Local\Temp\pip-install-3dsbnrvp\horovod\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d C:\Users\93176\AppData\Local\Temp\pip-wheel-ho87zqee --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\horovod
copying horovod_init_.py -> build\lib.win-amd64-3.6\horovod
creating build\lib.win-amd64-3.6\horovod\common
copying horovod\common_init_.py -> build\lib.win-amd64-3.6\horovod\common
creating build\lib.win-amd64-3.6\horovod\keras
copying horovod\keras\callbacks.py -> build\lib.win-amd64-3.6\horovod\keras
copying horovod\keras_init_.py -> build\lib.win-amd64-3.6\horovod\keras
creating build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow\compression.py -> build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow\mpi_ops.py -> build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow_init_.py -> build\lib.win-amd64-3.6\horovod\tensorflow
creating build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch\compression.py -> build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch\mpi_ops.py -> build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch_init_.py -> build\lib.win-amd64-3.6\horovod\torch
creating build\lib.win-amd64-3.6\horovod_keras
copying horovod_keras\callbacks.py -> build\lib.win-amd64-3.6\horovod_keras
copying horovod_keras_init_.py -> build\lib.win-amd64-3.6\horovod_keras
creating build\lib.win-amd64-3.6\horovod\tensorflow\keras
copying horovod\tensorflow\keras\callbacks.py -> build\lib.win-amd64-3.6\horovod\tensorflow\keras
copying horovod\tensorflow\keras_init_.py -> build\lib.win-amd64-3.6\horovod\tensorflow\keras
creating build\lib.win-amd64-3.6\horovod\torch\mpi_lib
copying horovod\torch\mpi_lib_init_.py -> build\lib.win-amd64-3.6\horovod\torch\mpi_lib
creating build\lib.win-amd64-3.6\horovod\torch\mpi_lib_impl
copying horovod\torch\mpi_lib_impl_init_.py -> build\lib.win-amd64-3.6\horovod\torch\mpi_lib_impl
running build_ext
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe -std=c++11 -fPIC -O2 -Wall -mf16c -mavx /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Id:\anaconda_install\include -Id:\anaconda_install\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\include "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tpbuild\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.cc /Fobuild\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.obj
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
cl: 命令行 warning D9002 :忽略未知选项“-fPIC”
cl: 命令行 warning D9002 :忽略未知选项“-mf16c”
cl: 命令行 warning D9002 :忽略未知选项“-mavx”
test_cpp_flags.cc
C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\malloc.h(45): warning C4820: “_heapinfo”:“4”字节填充添加在 数据成员“_heapinfo::_useflag”后
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\include\vcruntime_exception.h(25): warning C4820: “__std_exception_data”:“7”字节填充添加在 数据成员“__std_exception_data::_DoFree”后
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\anaconda_install\libs /LIBPATH:d:\anaconda_install\PCbuild\amd64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\lib\x64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\lib\x64 "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\um\x64" build\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.obj /OUT:build\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.dll
正在生成代码
已完成代码的生成
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Id:\anaconda_install\include -Id:\anaconda_install\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\include "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tpbuild\temp.win-amd64-3.6\Release\test_compile\test_link_flags.cc /Fobuild\temp.win-amd64-3.6\Release\test_compile\test_link_flags.obj
test_link_flags.cc
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\link.exe -Wl,--version-script=horovod.lds /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\anaconda_install\libs /LIBPATH:d:\anaconda_install\PCbuild\amd64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\lib\x64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\lib\x64 "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\um\x64" build\temp.win-amd64-3.6\Release\test_compile\test_link_flags.obj /OUT:build\temp.win-amd64-3.6\Release\test_compile\test_link_flags.dll
Microsoft (R) Incremental Linker Version 14.15.26729.0
Copyright (C) Microsoft Corporation. All rights reserved.
LINK : warning LNK4044: 无法识别的选项“/Wl,--version-script=horovod.lds”;已忽略
正在生成代码
已完成代码的生成
error: mpicxx -show failed (see error below), is MPI in $PATH?
Note: If your version of MPI has a custom command to show compilation flags, please specify it with the HOROVOD_MPICXX_SHOW environment variable.
Traceback (most recent call last):
File "C:\Users\93176\AppData\Local\Temp\pip-install-3dsbnrvp\horovod\setup.py", line 228, in get_mpi_flags
shlex.split(show_command), universal_newlines=True).strip()
File "d:\anaconda_install\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "d:\anaconda_install\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "d:\anaconda_install\lib\subprocess.py", line 709, in init
restore_signals, start_new_session)
File "d:\anaconda_install\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。
Failed building wheel for horovod
Running setup.py clean for horovod
Failed to build horovod
Installing collected packages: horovod
Running setup.py install for horovod ... error
Complete output from command d:\anaconda_install\python.exe -u -c "import setuptools, tokenize;file='C:\Users\93176\AppData\Local\Temp\pip-install-3dsbnrvp\horovod\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\93176\AppData\Local\Temp\pip-record-q6e4qgfq\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\horovod
copying horovod_init_.py -> build\lib.win-amd64-3.6\horovod
creating build\lib.win-amd64-3.6\horovod\common
copying horovod\common_init_.py -> build\lib.win-amd64-3.6\horovod\common
creating build\lib.win-amd64-3.6\horovod\keras
copying horovod\keras\callbacks.py -> build\lib.win-amd64-3.6\horovod\keras
copying horovod\keras_init_.py -> build\lib.win-amd64-3.6\horovod\keras
creating build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow\compression.py -> build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow\mpi_ops.py -> build\lib.win-amd64-3.6\horovod\tensorflow
copying horovod\tensorflow_init_.py -> build\lib.win-amd64-3.6\horovod\tensorflow
creating build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch\compression.py -> build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch\mpi_ops.py -> build\lib.win-amd64-3.6\horovod\torch
copying horovod\torch_init_.py -> build\lib.win-amd64-3.6\horovod\torch
creating build\lib.win-amd64-3.6\horovod_keras
copying horovod_keras\callbacks.py -> build\lib.win-amd64-3.6\horovod_keras
copying horovod_keras_init_.py -> build\lib.win-amd64-3.6\horovod_keras
creating build\lib.win-amd64-3.6\horovod\tensorflow\keras
copying horovod\tensorflow\keras\callbacks.py -> build\lib.win-amd64-3.6\horovod\tensorflow\keras
copying horovod\tensorflow\keras_init_.py -> build\lib.win-amd64-3.6\horovod\tensorflow\keras
creating build\lib.win-amd64-3.6\horovod\torch\mpi_lib
copying horovod\torch\mpi_lib_init_.py -> build\lib.win-amd64-3.6\horovod\torch\mpi_lib
creating build\lib.win-amd64-3.6\horovod\torch\mpi_lib_impl
copying horovod\torch\mpi_lib_impl_init_.py -> build\lib.win-amd64-3.6\horovod\torch\mpi_lib_impl
running build_ext
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe -std=c++11 -fPIC -O2 -Wall -mf16c -mavx /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Id:\anaconda_install\include -Id:\anaconda_install\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\include "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tpbuild\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.cc /Fobuild\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.obj
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
cl: 命令行 warning D9002 :忽略未知选项“-fPIC”
cl: 命令行 warning D9002 :忽略未知选项“-mf16c”
cl: 命令行 warning D9002 :忽略未知选项“-mavx”
test_cpp_flags.cc
C:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt\malloc.h(45): warning C4820: “_heapinfo”:“4”字节填充添加在 数据成员“_heapinfo::_useflag”后
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\include\vcruntime_exception.h(25): warning C4820: “__std_exception_data”:“7”字节填充添加在 数据成员“__std_exception_data::_DoFree”后
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\anaconda_install\libs /LIBPATH:d:\anaconda_install\PCbuild\amd64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\lib\x64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\lib\x64 "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\um\x64" build\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.obj /OUT:build\temp.win-amd64-3.6\Release\test_compile\test_cpp_flags.dll
正在生成代码
已完成代码的生成
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Id:\anaconda_install\include -Id:\anaconda_install\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\include -ID:\VS2017_install\VC\Tools\MSVC\14.15.26726\include "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tpbuild\temp.win-amd64-3.6\Release\test_compile\test_link_flags.cc /Fobuild\temp.win-amd64-3.6\Release\test_compile\test_link_flags.obj
test_link_flags.cc
D:\VS2017_install\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\link.exe -Wl,--version-script=horovod.lds /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:d:\anaconda_install\libs /LIBPATH:d:\anaconda_install\PCbuild\amd64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\ATLMFC\lib\x64 /LIBPATH:D:\VS2017_install\VC\Tools\MSVC\14.15.26726\lib\x64 "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17134.0\um\x64" build\temp.win-amd64-3.6\Release\test_compile\test_link_flags.obj /OUT:build\temp.win-amd64-3.6\Release\test_compile\test_link_flags.dll
Microsoft (R) Incremental Linker Version 14.15.26729.0
Copyright (C) Microsoft Corporation. All rights reserved.
Command "d:\anaconda_install\python.exe -u -c "import setuptools, tokenize;file='C:\Users\93176\AppData\Local\Temp\pip-install-3dsbnrvp\horovod\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\93176\AppData\Local\Temp\pip-record-q6e4qgfq\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\93176\AppData\Local\Temp\pip-install-3dsbnrvp\horovod
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Beta Was this translation helpful? Give feedback.
All reactions