[2021-03-26 13:52:23.577] [global] [error] could not find backend, defaulting to local
[2021-03-26 13:52:23.577] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-26 13:52:23.577] [local ee] [info] initializing LLVM backend
[2021-03-26 13:52:23.577] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-26 13:52:23.577] [LLVM] [info] compiling code for skylake-avx512
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.578] [local execution engine] [info] started local executor E/1 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.578] [E/1] [info] starting detached process queue
[2021-03-26 13:52:23.578] [local execution engine] [info] started local executor E/2 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.578] [E/2] [info] starting detached process queue
[2021-03-26 13:52:23.578] [E/1] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.578] [E/2] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.578] [local execution engine] [info] started local executor E/3 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.578] [E/3] [info] starting detached process queue
[2021-03-26 13:52:23.578] [E/3] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/4 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/4] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/5 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/4] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/5] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/6 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/6] [info] starting detached process queue
[2021-03-26 13:52:23.579] [E/5] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/7 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/6] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/7] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/8 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/7] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/8] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/9 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/8] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/10 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/9] [info] starting detached process queue
[2021-03-26 13:52:23.579] [E/10] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/11 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/9] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/10] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/12 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/11] [info] starting detached process queue
[2021-03-26 13:52:23.579] [E/12] [info] starting detached process queue
[2021-03-26 13:52:23.579] [E/11] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/13 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/12] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/13] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/14 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/13] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/14] [info] starting detached process queue
[2021-03-26 13:52:23.579] [local execution engine] [info] started local executor E/15 (15.00 GB, 32.00 MB default partition size)
[2021-03-26 13:52:23.579] [E/15] [info] starting detached process queue
[2021-03-26 13:52:23.579] [memory] [info] allocated bitmap managed memory region (15.00 GB, 32.00 MB block size)
[2021-03-26 13:52:23.579] [local execution engine] [info] started driver (15.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-26 13:52:23.579] [E/14] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [E/15] [info] initialized runtime memory (4.00 MB)
[2021-03-26 13:52:23.579] [fileinputoperator] [info] found 1 file (1.10 GB) to process.
[2021-03-26 13:52:23.580] [global] [info] sampled file:///data/311/311_preprocessed.csv on 256.00 KB
[2021-03-26 13:52:23.748] [logical planner] [info] logical optimization took 0.009160ms
output type of specialized pipeline is: (Option[str])
is this the most outer stage?: 0
need to upgrade output type to (Option[str])
[2021-03-26 13:52:23.748] [codegen] [info] generating pipeline for (Option[str]) -> (Option[str]) (2 operators pipelined)
[2021-03-26 13:52:23.748] [codegen] [info] generating function fix_zip_codes for (Option[str]) -> Option[str]
[2021-03-26 13:52:23.769] [codegen] [info] generating pipeline for (Option[str]) -> (str) (0 operators pipelined)
[2021-03-26 13:52:23.769] [global] [info] Stage1 depends on: Stage0 
[2021-03-26 13:52:23.769] [global] [info] Stage0 depends on: 
[2021-03-26 13:52:23.770] [global] [info] provided option to explicitly merge bad rows in order back, however rows will be hashed. Disabling option. To silence this warning, set
tuplex.optimizer.mergeExceptionsInOrder=false
[2021-03-26 13:52:23.790] [global] [info] Optimization via LLVM passes took 0.020179 ms
[2021-03-26 13:52:23.798] [global] [info] Compiled code paths for stage 0 in 0.01 ms
[2021-03-26 13:52:23.798] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.0290633s
[2021-03-26 13:52:23.798] [local ee] [info] split /data/311/311_preprocessed.csv into 17 tasks
[2021-03-26 13:52:25.092] [driver] [info] [Task Finished] Transform to in-memory hash table in 1.293099s (0 normal rows, 0 exceptions, 186 buckets)
[2021-03-26 13:52:25.102] [E/10] [info] [Task Finished] Transform to in-memory hash table in 1.303518s (0 normal rows, 0 exceptions, 183 buckets)
[2021-03-26 13:52:25.103] [E/7] [info] [Task Finished] Transform to in-memory hash table in 1.304158s (0 normal rows, 0 exceptions, 178 buckets)
[2021-03-26 13:52:25.103] [E/14] [info] [Task Finished] Transform to in-memory hash table in 1.304624s (0 normal rows, 0 exceptions, 191 buckets)
[2021-03-26 13:52:25.104] [E/5] [info] [Task Finished] Transform to in-memory hash table in 1.305276s (0 normal rows, 0 exceptions, 179 buckets)
[2021-03-26 13:52:25.106] [E/4] [info] [Task Finished] Transform to in-memory hash table in 1.307714s (0 normal rows, 0 exceptions, 188 buckets)
[2021-03-26 13:52:25.107] [E/15] [info] [Task Finished] Transform to in-memory hash table in 1.308669s (0 normal rows, 0 exceptions, 183 buckets)
[2021-03-26 13:52:25.108] [E/8] [info] [Task Finished] Transform to in-memory hash table in 1.309819s (0 normal rows, 0 exceptions, 181 buckets)
[2021-03-26 13:52:25.109] [E/12] [info] [Task Finished] Transform to in-memory hash table in 1.310748s (0 normal rows, 0 exceptions, 185 buckets)
[2021-03-26 13:52:25.111] [E/3] [info] [Task Finished] Transform to in-memory hash table in 1.311986s (0 normal rows, 0 exceptions, 183 buckets)
[2021-03-26 13:52:25.120] [E/2] [info] [Task Finished] Transform to in-memory hash table in 1.321268s (0 normal rows, 0 exceptions, 180 buckets)
[2021-03-26 13:52:25.120] [E/1] [info] [Task Finished] Transform to in-memory hash table in 1.321956s (0 normal rows, 0 exceptions, 184 buckets)
[2021-03-26 13:52:25.131] [E/6] [info] [Task Finished] Transform to in-memory hash table in 1.332884s (0 normal rows, 0 exceptions, 186 buckets)
[2021-03-26 13:52:25.146] [E/11] [info] [Task Finished] Transform to in-memory hash table in 1.347663s (0 normal rows, 0 exceptions, 186 buckets)
[2021-03-26 13:52:25.446] [E/9] [info] [Task Finished] Transform to in-memory hash table in 1.647572s (0 normal rows, 0 exceptions, 184 buckets)
[2021-03-26 13:52:25.508] [E/13] [info] [Task Finished] Transform to in-memory hash table in 1.708987s (0 normal rows, 0 exceptions, 188 buckets)
[2021-03-26 13:52:27.262] [driver] [info] [Task Finished] Transform to in-memory hash table in 2.170813s (0 normal rows, 0 exceptions, 188 buckets)
[2021-03-26 13:52:27.262] [global] [info] [Transform Stage] Stage 0 completed 17 load&transform tasks in 3.46409s
[2021-03-26 13:52:27.263] [global] [info] [Transform Stage] Stage 0 total wall clock time: 23.9108s, 197628000 input rows, time to process 1 row via fast path: 0.000120989ms
[2021-03-26 13:52:27.263] [global] [info] [Transform Stage] Stage 0 completed 17 sink tasks in 0.000451037s
[2021-03-26 13:52:27.263] [global] [info] [Transform Stage] Stage 0 took 3.49364s
[2021-03-26 13:52:27.277] [global] [info] Optimization via LLVM passes took 0.013099 ms
[2021-03-26 13:52:27.282] [global] [info] Compiled code paths for stage 1 in 0.01 ms
[2021-03-26 13:52:27.282] [global] [info] [Transform Stage] Stage 1 compiled to x86 in 0.018825s
[2021-03-26 13:52:27.282] [E/1] [info] [Task Finished] Transform to mem in 0.000040s (246 normal rows, 0 exceptions)
[2021-03-26 13:52:27.282] [global] [info] [Transform Stage] Stage 1 completed 1 load&transform tasks in 8.6093e-05s
[2021-03-26 13:52:27.282] [global] [info] [Transform Stage] Stage 1 total wall clock time: 3.971e-05s, 246 input rows, time to process 1 row via fast path: 0.000161423ms

[2021-03-26 13:52:27.282] [global] [info] Writing 246 rows as output to file (1.44 KB)
[2021-03-26 13:52:27.282] [E/1] [info] [Task Finished] write to file in 0.000069s (246 rows, 1.44 KB)
[2021-03-26 13:52:27.282] [global] [info] writing output took 0.000257s
[2021-03-26 13:52:27.282] [global] [info] [Transform Stage] Stage 1 completed 1 sink tasks in 0.000269981s
[2021-03-26 13:52:27.282] [global] [info] [Transform Stage] Stage 1 took 0.0192029s
[2021-03-26 13:52:27.282] [global] [info] Query Execution took 3.54721s. (planning: 0.0342751s, execution: 3.51293s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
Tuplex startup time: 0.2749028205871582
{"startupTime": 0.2749028205871582, "jobTime": 3.705420732498169}
[2021-03-26 13:52:27.309] [local execution engine] [info] Stopped local executor E/1
[2021-03-26 13:52:27.310] [local execution engine] [info] Stopped local executor E/2
[2021-03-26 13:52:27.310] [local execution engine] [info] Stopped local executor E/3
[2021-03-26 13:52:27.310] [local execution engine] [info] Stopped local executor E/4
[2021-03-26 13:52:27.311] [local execution engine] [info] Stopped local executor E/5
[2021-03-26 13:52:27.311] [local execution engine] [info] Stopped local executor E/6
[2021-03-26 13:52:27.311] [local execution engine] [info] Stopped local executor E/7
[2021-03-26 13:52:27.312] [local execution engine] [info] Stopped local executor E/8
[2021-03-26 13:52:27.312] [local execution engine] [info] Stopped local executor E/9
[2021-03-26 13:52:27.312] [local execution engine] [info] Stopped local executor E/10
[2021-03-26 13:52:27.313] [local execution engine] [info] Stopped local executor E/11
[2021-03-26 13:52:27.313] [local execution engine] [info] Stopped local executor E/12
[2021-03-26 13:52:27.313] [local execution engine] [info] Stopped local executor E/13
[2021-03-26 13:52:27.313] [local execution engine] [info] Stopped local executor E/14
[2021-03-26 13:52:27.314] [local execution engine] [info] Stopped local executor E/15
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
