这是indexloc提供的服务，不要输入任何密码
Skip to content

[HOTFIX] Fix the Ubuntu 20.04 build issues and update root level CMakeLists.txt #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Oct 27, 2021
Merged

[HOTFIX] Fix the Ubuntu 20.04 build issues and update root level CMakeLists.txt #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
75cdabb
dependency fixes wip
LeonhardFS Oct 26, 2021
394c3eb
forcing stdc++ on gcc
LeonhardFS Oct 26, 2021
7b36eeb
enforcing C++14
LeonhardFS Oct 26, 2021
bdd5238
fix?
LeonhardFS Oct 26, 2021
c414ea8
reqs change
LeonhardFS Oct 26, 2021
1f3132f
cmake cleanup
LeonhardFS Oct 26, 2021
ec3bfd4
message fix
LeonhardFS Oct 26, 2021
e7aa8a8
using ucm for cmake options
LeonhardFS Oct 26, 2021
f0d11e1
disabling GNU gold
LeonhardFS Oct 27, 2021
4e92a1c
remove glibcxx
LeonhardFS Oct 27, 2021
64bc71b
cmake style lto
LeonhardFS Oct 27, 2021
83b8067
cmake cleanup, deactivating gcc's buggy LTO
LeonhardFS Oct 27, 2021
7927dc8
adding additional check to setup.py, restricting build to shared obje…
LeonhardFS Oct 27, 2021
cfef4c0
better detection of runtime
LeonhardFS Oct 27, 2021
aab48ba
fix for shell
LeonhardFS Oct 27, 2021
f8aa203
removed versions from setup, this will allow jedi to hopefully work a…
LeonhardFS Oct 27, 2021
9210dc4
cleanup
LeonhardFS Oct 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,27 +48,27 @@ To build Tuplex, you need several other packages first which can be easily insta
```
brew install llvm@9 boost boost-python3 aws-sdk-cpp pcre2 antlr4-cpp-runtime googletest gflags yaml-cpp celero protobuf libmagic
python3 -m pip install cloudpickle numpy
python3 setup.py install
python3 setup.py install --user
```

#### Ubuntu build from source
To faciliate installing the dependencies for Ubuntu, we do provide two scripts (`scripts/ubuntu1804/install_reqs.sh` for Ubuntu 18.04, or `scripts/ubuntu2004/install_reqs.sh` for Ubuntu 20.04). To create an up to date version of Tuplex, simply run
```
./scripts/ubuntu1804/install_reqs.sh
python3 -m pip install cloudpickle numpy
python3 setup.py install
python3 setup.py install --user
```

#### Customizing the build

Besides building a pip package, cmake can be also directly invoked. To compile the package via cmake
Besides building a pip package, especially for development it may be more useful to invoke cmake directly. To create a development version of Tuplex and work with it like a regular cmake project, go to the folder `tuplex` and then use the standard workflow to compile the package via cmake (and not the top-level setup.py file):
```
mkdir build
cd build
cmake ..
make -j$(nproc)
```
The python package corresponding to Tuplex can be then found in `build/dist/python` with C++ test executables based on googletest in `build/dist/bin`.
The python package corresponding to Tuplex can be then found in `build/dist/python` with C++ test executables based on googletest in `build/dist/bin`. If you'd like to use a cmake-compatible IDE like CLion or VSCode you can simply open the `tuplex/` folder and import the `CMakeLists.txt` contained there.

To customize the cmake build, the following options are available to be passed via `-D<option>=<value>`:

Expand Down
2 changes: 1 addition & 1 deletion scripts/ubuntu1804/install_reqs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ apt-get install -y build-essential autoconf automake libtool software-properties
uuid-dev git python3.7 python3.7-dev python3-pip libffi-dev \
doxygen doxygen-doc doxygen-latex doxygen-gui graphviz \
gcc-7 g++-7 libgflags-dev libncurses-dev \
awscli openjdk-8-jdk libyaml-dev libmagic-dev
awscli openjdk-8-jdk libyaml-dev libmagic-dev ninja-build

# LLVM 9 packages (prob not all of them needed, but here for complete install)
wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh &&
Expand Down
20 changes: 18 additions & 2 deletions scripts/ubuntu2004/install_reqs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,19 @@ apt-get install -y build-essential software-properties-common wget libedit-dev l
uuid-dev git libffi-dev libmagic-dev \
doxygen doxygen-doc doxygen-latex doxygen-gui graphviz \
libgflags-dev libncurses-dev \
awscli openjdk-8-jdk libyaml-dev
awscli openjdk-8-jdk libyaml-dev ninja-build gcc-10 g++-10 autoconf libtool m4

# use GCC 10, as Tuplex doesn't work with GCC 9
update-alternatives --remove-all gcc
update-alternatives --remove-all g++
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10

# update to make sure everything is compiled using gcc-10
ldconfig
export CC=gcc-10
export CXX=g++-10


# LLVM 9 packages (prob not all of them needed, but here for complete install)
wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh &&
./llvm.sh 9 && rm -rf llvm.sh
Expand Down Expand Up @@ -91,7 +97,7 @@ mkdir -p /tmp/celero-v2.6.0 &&
git checkout tags/v2.6.0 &&
mkdir build &&
cd build &&
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt -DBUILD_SHARED_LIBS=OFF -DCMAKE_CXX_FLAGS=$CXX_FLAGS .. &&
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt -DBUILD_SHARED_LIBS=OFF -DCMAKE_CXX_FLAGS="-fPIC" .. &&
make -j 32 &&
make install &&
cd &&
Expand Down Expand Up @@ -151,6 +157,16 @@ popd
# install python packages for tuplex (needs cloudpickle to compile, numpy to run certain tests)
pip3 install cloudpickle numpy

# protobuf 3.12
cd /tmp &&
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.12.0/protobuf-cpp-3.12.0.tar.gz &&
tar xf protobuf-cpp-3.12.0.tar.gz &&
pushd protobuf-3.12.0 &&
./autogen.sh && ./configure "CFLAGS=-fPIC" "CXXFLAGS=-fPIC" &&
make -j4 && make install && ldconfig &&
pushd


# setup bash aliases
echo "alias antlr='java -jar /opt/lib/antlr-4.8-complete.jar'" >>"$HOME/.bashrc"
echo "alias grun='java org.antlr.v4.gui.TestRig'" >>"$HOME/.bashrc"
Expand Down
25 changes: 17 additions & 8 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,17 @@
'six>=1.11.0',
'wcwidth>=0.1.7',
'astor>=0.7.1',
'jedi>=0.13.2',
'prompt_toolkit',
'jedi',
'cloudpickle>=0.6.1',
'PyYAML>=3.13'
]

def ninja_installed():
# check whether ninja is on the path
from distutils.spawn import find_executable
return find_executable('ninja') is not None

def find_files(pattern, path):
result = []
for root, dirs, files in os.walk(path):
Expand All @@ -55,7 +61,6 @@ def find_files(pattern, path):
"win-arm64": "ARM64",
}


# A CMakeExtension needs a sourcedir instead of a file list.
# The name must be the _single_ output extension from the CMake build.
# If you need multiple extensions, see scikit-build.
Expand All @@ -64,7 +69,6 @@ def __init__(self, name, sourcedir=""):
Extension.__init__(self, name, sources=[])
self.sourcedir = os.path.abspath(sourcedir)


class CMakeBuild(build_ext):

def build_extension(self, ext):
Expand Down Expand Up @@ -180,8 +184,10 @@ def find_pkg_path(lines):
# Users can override the generator with CMAKE_GENERATOR in CMake
# 3.15+.
if not cmake_generator:
cmake_args += ["-GNinja"]

# yet, check if Ninja exists...
if ninja_installed():
cmake_args += ["-GNinja"]
else:

# Single config generators are handled "normally"
Expand Down Expand Up @@ -215,10 +221,13 @@ def find_pkg_path(lines):
if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp)

if os.environ.get('CIBUILDWHEEL', '0') == '1':
# on cibuildwheel b.c. manylinux2014 does not have python shared objects, build
# only tuplex target (the python shared object)
build_args += ['--target', 'tuplex']
## on cibuildwheel b.c. manylinux2014 does not have python shared objects, build
## only tuplex target (the python shared object)
#if os.environ.get('CIBUILDWHEEL', '0') == '1':

# because the goal of setup.py is to only build the package, build only target tuplex.
# changed from before.
build_args += ['--target', 'tuplex']

# hack: only run for first invocation!
if ext_filename == 'tuplex_runtime':
Expand Down
Loading