[2021-03-24 07:25:12.355] [global] [error] could not find backend, defaulting to local
[2021-03-24 07:25:12.356] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-24 07:25:12.356] [local ee] [info] initializing LLVM backend
[2021-03-24 07:25:12.356] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-24 07:25:12.356] [LLVM] [info] compiling code for skylake-avx512
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/1 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/1] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/2 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/2] [info] starting detached process queue
[2021-03-24 07:25:12.357] [E/1] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/2] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/3 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/3] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/4 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/3] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/4] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/5 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/5] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/6 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/4] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/5] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/6] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/7 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/6] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/8 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/7] [info] starting detached process queue
[2021-03-24 07:25:12.357] [E/8] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/9 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/7] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/9] [info] starting detached process queue
[2021-03-24 07:25:12.357] [E/8] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/10 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/9] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [E/10] [info] starting detached process queue
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/11 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/11] [info] starting detached process queue
[2021-03-24 07:25:12.357] [E/10] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.357] [local execution engine] [info] started local executor E/12 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.357] [E/11] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.358] [E/12] [info] starting detached process queue
[2021-03-24 07:25:12.358] [local execution engine] [info] started local executor E/13 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.358] [E/13] [info] starting detached process queue
[2021-03-24 07:25:12.358] [local execution engine] [info] started local executor E/14 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.358] [E/12] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.358] [E/13] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.358] [local execution engine] [info] started local executor E/15 (6.00 GB, 32.00 MB default partition size)
[2021-03-24 07:25:12.358] [E/14] [info] starting detached process queue
[2021-03-24 07:25:12.358] [E/15] [info] starting detached process queue
[2021-03-24 07:25:12.358] [memory] [info] allocated bitmap managed memory region (10.00 GB, 32.00 MB block size)
[2021-03-24 07:25:12.358] [E/14] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.358] [local execution engine] [info] started driver (10.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-24 07:25:12.358] [E/15] [info] initialized runtime memory (4.00 MB)
[2021-03-24 07:25:12.358] [fileinputoperator] [info] found 1 file (9.31 GB) to process.
[2021-03-24 07:25:12.358] [global] [info] sampled file:///data/zillow/Z1_preprocessed/zillow_Z1_10G.csv on 256.00 KB
[2021-03-24 07:25:12.497] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.513] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.521] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.545] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.565] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.579] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.611] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-24 07:25:12.627] [global] [info] performing static typing for UDF in operator map
[2021-03-24 07:25:12.629] [global] [info] performing static typing for UDF in operator map
[2021-03-24 07:25:12.632] [global] [info] performing static typing for UDF in operator map
[2021-03-24 07:25:12.634] [global] [info] performing static typing for UDF in operator map
[2021-03-24 07:25:12.639] [logical planner] [info] logical optimization took 0.009415ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-24 07:25:12.639] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (12 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,i64,i64,str,str,i64)
[2021-03-24 07:25:12.640] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-24 07:25:12.641] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-24 07:25:12.641] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-24 07:25:12.642] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-24 07:25:12.642] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-24 07:25:12.643] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-24 07:25:12.643] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> i64
[2021-03-24 07:25:12.644] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64)) -> i64
[2021-03-24 07:25:12.645] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64,i64)) -> str
[2021-03-24 07:25:12.646] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64,i64,str)) -> i64
[2021-03-24 07:25:12.647] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,i64,i64,str)) -> boolean
[2021-03-24 07:25:12.648] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,i64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,i64,i64,str,str,i64)
[2021-03-24 07:25:12.674] [global] [info] Stage0 depends on: 
[2021-03-24 07:25:12.816] [global] [info] Optimization via LLVM passes took 0.135425 ms
[2021-03-24 07:25:12.877] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-24 07:25:12.877] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.203765s
[2021-03-24 07:25:12.878] [local ee] [info] split /data/zillow/Z1_preprocessed/zillow_Z1_10G.csv into 149 tasks
[2021-03-24 07:25:13.104] [E/14] [info] [Task Finished] Transform to mem in 0.226254s (5882 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [E/8] [info] [Task Finished] Transform to mem in 0.226923s (5663 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [E/4] [info] [Task Finished] Transform to mem in 0.227525s (5772 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [E/2] [info] [Task Finished] Transform to mem in 0.227531s (5779 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [E/3] [info] [Task Finished] Transform to mem in 0.227708s (5710 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [E/9] [info] [Task Finished] Transform to mem in 0.227760s (5883 normal rows, 0 exceptions)
[2021-03-24 07:25:13.105] [driver] [info] [Task Finished] Transform to mem in 0.227857s (5770 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/1] [info] [Task Finished] Transform to mem in 0.227991s (5864 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/5] [info] [Task Finished] Transform to mem in 0.227999s (5748 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/6] [info] [Task Finished] Transform to mem in 0.228056s (5758 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/15] [info] [Task Finished] Transform to mem in 0.228150s (5860 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/12] [info] [Task Finished] Transform to mem in 0.228162s (5744 normal rows, 0 exceptions)
[2021-03-24 07:25:13.106] [E/11] [info] [Task Finished] Transform to mem in 0.228333s (5802 normal rows, 0 exceptions)
[2021-03-24 07:25:13.107] [E/10] [info] [Task Finished] Transform to mem in 0.228921s (5796 normal rows, 0 exceptions)
[2021-03-24 07:25:13.107] [E/13] [info] [Task Finished] Transform to mem in 0.229650s (5706 normal rows, 0 exceptions)
[2021-03-24 07:25:13.108] [E/7] [info] [Task Finished] Transform to mem in 0.230572s (5799 normal rows, 0 exceptions)
[2021-03-24 07:25:13.328] [E/14] [info] [Task Finished] Transform to mem in 0.224530s (6027 normal rows, 0 exceptions)
[2021-03-24 07:25:13.329] [E/8] [info] [Task Finished] Transform to mem in 0.223970s (5766 normal rows, 0 exceptions)
[2021-03-24 07:25:13.329] [E/12] [info] [Task Finished] Transform to mem in 0.223610s (5665 normal rows, 0 exceptions)
[2021-03-24 07:25:13.330] [E/2] [info] [Task Finished] Transform to mem in 0.224423s (5709 normal rows, 0 exceptions)
[2021-03-24 07:25:13.330] [E/9] [info] [Task Finished] Transform to mem in 0.224508s (5821 normal rows, 0 exceptions)
[2021-03-24 07:25:13.330] [E/6] [info] [Task Finished] Transform to mem in 0.224272s (5651 normal rows, 0 exceptions)
[2021-03-24 07:25:13.330] [E/15] [info] [Task Finished] Transform to mem in 0.224188s (5772 normal rows, 0 exceptions)
[2021-03-24 07:25:13.330] [driver] [info] [Task Finished] Transform to mem in 0.224914s (5752 normal rows, 0 exceptions)
[2021-03-24 07:25:13.331] [E/5] [info] [Task Finished] Transform to mem in 0.225060s (5882 normal rows, 0 exceptions)
[2021-03-24 07:25:13.331] [E/11] [info] [Task Finished] Transform to mem in 0.225206s (5716 normal rows, 0 exceptions)
[2021-03-24 07:25:13.331] [E/1] [info] [Task Finished] Transform to mem in 0.225780s (5761 normal rows, 0 exceptions)
[2021-03-24 07:25:13.332] [E/10] [info] [Task Finished] Transform to mem in 0.225161s (5723 normal rows, 0 exceptions)
[2021-03-24 07:25:13.332] [E/13] [info] [Task Finished] Transform to mem in 0.224571s (5792 normal rows, 0 exceptions)
[2021-03-24 07:25:13.333] [E/3] [info] [Task Finished] Transform to mem in 0.227582s (5678 normal rows, 0 exceptions)
[2021-03-24 07:25:13.334] [E/7] [info] [Task Finished] Transform to mem in 0.225460s (5781 normal rows, 0 exceptions)
[2021-03-24 07:25:13.335] [E/4] [info] [Task Finished] Transform to mem in 0.229423s (5706 normal rows, 0 exceptions)
[2021-03-24 07:25:13.552] [E/14] [info] [Task Finished] Transform to mem in 0.223590s (5622 normal rows, 0 exceptions)
[2021-03-24 07:25:13.552] [E/8] [info] [Task Finished] Transform to mem in 0.223838s (5781 normal rows, 0 exceptions)
[2021-03-24 07:25:13.553] [E/12] [info] [Task Finished] Transform to mem in 0.223947s (5842 normal rows, 0 exceptions)
[2021-03-24 07:25:13.554] [E/15] [info] [Task Finished] Transform to mem in 0.224087s (5696 normal rows, 0 exceptions)
[2021-03-24 07:25:13.554] [E/9] [info] [Task Finished] Transform to mem in 0.224264s (5860 normal rows, 0 exceptions)
[2021-03-24 07:25:13.555] [E/6] [info] [Task Finished] Transform to mem in 0.224886s (5749 normal rows, 0 exceptions)
[2021-03-24 07:25:13.555] [E/2] [info] [Task Finished] Transform to mem in 0.225584s (5926 normal rows, 0 exceptions)
[2021-03-24 07:25:13.555] [driver] [info] [Task Finished] Transform to mem in 0.225027s (5821 normal rows, 0 exceptions)
[2021-03-24 07:25:13.556] [E/5] [info] [Task Finished] Transform to mem in 0.225356s (5831 normal rows, 0 exceptions)
[2021-03-24 07:25:13.556] [E/1] [info] [Task Finished] Transform to mem in 0.224668s (5721 normal rows, 0 exceptions)
[2021-03-24 07:25:13.557] [E/11] [info] [Task Finished] Transform to mem in 0.225500s (5915 normal rows, 0 exceptions)
[2021-03-24 07:25:13.558] [E/10] [info] [Task Finished] Transform to mem in 0.225991s (5839 normal rows, 0 exceptions)
[2021-03-24 07:25:13.559] [E/13] [info] [Task Finished] Transform to mem in 0.226628s (5750 normal rows, 0 exceptions)
[2021-03-24 07:25:13.561] [E/3] [info] [Task Finished] Transform to mem in 0.227845s (5878 normal rows, 0 exceptions)
[2021-03-24 07:25:13.561] [E/7] [info] [Task Finished] Transform to mem in 0.227506s (5861 normal rows, 0 exceptions)
[2021-03-24 07:25:13.564] [E/4] [info] [Task Finished] Transform to mem in 0.229700s (5810 normal rows, 0 exceptions)
[2021-03-24 07:25:13.776] [E/14] [info] [Task Finished] Transform to mem in 0.223961s (5662 normal rows, 0 exceptions)
[2021-03-24 07:25:13.777] [E/8] [info] [Task Finished] Transform to mem in 0.224243s (5777 normal rows, 0 exceptions)
[2021-03-24 07:25:13.777] [E/12] [info] [Task Finished] Transform to mem in 0.224058s (5657 normal rows, 0 exceptions)
[2021-03-24 07:25:13.778] [E/15] [info] [Task Finished] Transform to mem in 0.224043s (5731 normal rows, 0 exceptions)
[2021-03-24 07:25:13.778] [E/9] [info] [Task Finished] Transform to mem in 0.224061s (5694 normal rows, 0 exceptions)
[2021-03-24 07:25:13.780] [E/6] [info] [Task Finished] Transform to mem in 0.224727s (5813 normal rows, 0 exceptions)
[2021-03-24 07:25:13.780] [driver] [info] [Task Finished] Transform to mem in 0.224744s (5850 normal rows, 0 exceptions)
[2021-03-24 07:25:13.781] [E/1] [info] [Task Finished] Transform to mem in 0.224668s (5795 normal rows, 0 exceptions)
[2021-03-24 07:25:13.781] [E/2] [info] [Task Finished] Transform to mem in 0.225895s (5893 normal rows, 0 exceptions)
[2021-03-24 07:25:13.781] [E/5] [info] [Task Finished] Transform to mem in 0.225300s (5742 normal rows, 0 exceptions)
[2021-03-24 07:25:13.782] [E/11] [info] [Task Finished] Transform to mem in 0.225300s (5714 normal rows, 0 exceptions)
[2021-03-24 07:25:13.783] [E/13] [info] [Task Finished] Transform to mem in 0.224324s (5750 normal rows, 0 exceptions)
[2021-03-24 07:25:13.784] [E/10] [info] [Task Finished] Transform to mem in 0.226567s (5709 normal rows, 0 exceptions)
[2021-03-24 07:25:13.786] [E/7] [info] [Task Finished] Transform to mem in 0.225127s (5759 normal rows, 0 exceptions)
[2021-03-24 07:25:13.788] [E/3] [info] [Task Finished] Transform to mem in 0.227198s (5750 normal rows, 0 exceptions)
[2021-03-24 07:25:13.792] [E/4] [info] [Task Finished] Transform to mem in 0.227784s (5734 normal rows, 0 exceptions)
[2021-03-24 07:25:14.000] [E/14] [info] [Task Finished] Transform to mem in 0.224149s (5881 normal rows, 0 exceptions)
[2021-03-24 07:25:14.001] [E/8] [info] [Task Finished] Transform to mem in 0.224278s (5807 normal rows, 0 exceptions)
[2021-03-24 07:25:14.001] [E/12] [info] [Task Finished] Transform to mem in 0.223947s (5797 normal rows, 0 exceptions)
[2021-03-24 07:25:14.002] [E/15] [info] [Task Finished] Transform to mem in 0.223969s (5786 normal rows, 0 exceptions)
[2021-03-24 07:25:14.003] [E/9] [info] [Task Finished] Transform to mem in 0.224291s (5826 normal rows, 0 exceptions)
[2021-03-24 07:25:14.004] [E/6] [info] [Task Finished] Transform to mem in 0.224612s (5722 normal rows, 0 exceptions)
[2021-03-24 07:25:14.005] [driver] [info] [Task Finished] Transform to mem in 0.224729s (5746 normal rows, 0 exceptions)
[2021-03-24 07:25:14.006] [E/1] [info] [Task Finished] Transform to mem in 0.225051s (5775 normal rows, 0 exceptions)
[2021-03-24 07:25:14.007] [E/2] [info] [Task Finished] Transform to mem in 0.225528s (5766 normal rows, 0 exceptions)
[2021-03-24 07:25:14.007] [E/5] [info] [Task Finished] Transform to mem in 0.225889s (5877 normal rows, 0 exceptions)
[2021-03-24 07:25:14.007] [E/11] [info] [Task Finished] Transform to mem in 0.225429s (5677 normal rows, 0 exceptions)
[2021-03-24 07:25:14.010] [E/13] [info] [Task Finished] Transform to mem in 0.226717s (5846 normal rows, 0 exceptions)
[2021-03-24 07:25:14.010] [E/10] [info] [Task Finished] Transform to mem in 0.225681s (5851 normal rows, 0 exceptions)
[2021-03-24 07:25:14.013] [E/7] [info] [Task Finished] Transform to mem in 0.226604s (5820 normal rows, 0 exceptions)
[2021-03-24 07:25:14.015] [E/3] [info] [Task Finished] Transform to mem in 0.227335s (5726 normal rows, 0 exceptions)
[2021-03-24 07:25:14.020] [E/4] [info] [Task Finished] Transform to mem in 0.227701s (5684 normal rows, 0 exceptions)
[2021-03-24 07:25:14.224] [E/14] [info] [Task Finished] Transform to mem in 0.223793s (5744 normal rows, 0 exceptions)
[2021-03-24 07:25:14.225] [E/8] [info] [Task Finished] Transform to mem in 0.224078s (5606 normal rows, 0 exceptions)
[2021-03-24 07:25:14.225] [E/12] [info] [Task Finished] Transform to mem in 0.224011s (5707 normal rows, 0 exceptions)
[2021-03-24 07:25:14.226] [E/15] [info] [Task Finished] Transform to mem in 0.224228s (5831 normal rows, 0 exceptions)
[2021-03-24 07:25:14.227] [E/9] [info] [Task Finished] Transform to mem in 0.224480s (5742 normal rows, 0 exceptions)
[2021-03-24 07:25:14.229] [E/6] [info] [Task Finished] Transform to mem in 0.224432s (5546 normal rows, 0 exceptions)
[2021-03-24 07:25:14.229] [driver] [info] [Task Finished] Transform to mem in 0.224399s (5744 normal rows, 0 exceptions)
[2021-03-24 07:25:14.231] [E/1] [info] [Task Finished] Transform to mem in 0.225025s (5879 normal rows, 0 exceptions)
[2021-03-24 07:25:14.232] [E/2] [info] [Task Finished] Transform to mem in 0.224950s (5865 normal rows, 0 exceptions)
[2021-03-24 07:25:14.233] [E/11] [info] [Task Finished] Transform to mem in 0.225177s (5632 normal rows, 0 exceptions)
[2021-03-24 07:25:14.233] [E/5] [info] [Task Finished] Transform to mem in 0.225761s (5776 normal rows, 0 exceptions)
[2021-03-24 07:25:14.234] [E/13] [info] [Task Finished] Transform to mem in 0.224516s (5766 normal rows, 0 exceptions)
[2021-03-24 07:25:14.236] [E/10] [info] [Task Finished] Transform to mem in 0.226063s (5851 normal rows, 0 exceptions)
[2021-03-24 07:25:14.238] [E/7] [info] [Task Finished] Transform to mem in 0.225204s (5842 normal rows, 0 exceptions)
[2021-03-24 07:25:14.243] [E/3] [info] [Task Finished] Transform to mem in 0.227141s (5844 normal rows, 0 exceptions)
[2021-03-24 07:25:14.248] [E/4] [info] [Task Finished] Transform to mem in 0.228064s (5911 normal rows, 0 exceptions)
[2021-03-24 07:25:14.448] [E/14] [info] [Task Finished] Transform to mem in 0.224265s (5907 normal rows, 0 exceptions)
[2021-03-24 07:25:14.449] [E/8] [info] [Task Finished] Transform to mem in 0.224267s (5866 normal rows, 0 exceptions)
[2021-03-24 07:25:14.450] [E/12] [info] [Task Finished] Transform to mem in 0.224190s (5743 normal rows, 0 exceptions)
[2021-03-24 07:25:14.450] [E/15] [info] [Task Finished] Transform to mem in 0.223810s (5720 normal rows, 0 exceptions)
[2021-03-24 07:25:14.452] [E/9] [info] [Task Finished] Transform to mem in 0.224427s (5685 normal rows, 0 exceptions)
[2021-03-24 07:25:14.454] [E/6] [info] [Task Finished] Transform to mem in 0.224998s (5883 normal rows, 0 exceptions)
[2021-03-24 07:25:14.454] [driver] [info] [Task Finished] Transform to mem in 0.224641s (5706 normal rows, 0 exceptions)
[2021-03-24 07:25:14.458] [E/1] [info] [Task Finished] Transform to mem in 0.226617s (5929 normal rows, 0 exceptions)
[2021-03-24 07:25:14.458] [E/2] [info] [Task Finished] Transform to mem in 0.226047s (5872 normal rows, 0 exceptions)
[2021-03-24 07:25:14.458] [E/11] [info] [Task Finished] Transform to mem in 0.225644s (5847 normal rows, 0 exceptions)
[2021-03-24 07:25:14.459] [E/5] [info] [Task Finished] Transform to mem in 0.225751s (5816 normal rows, 0 exceptions)
[2021-03-24 07:25:14.461] [E/13] [info] [Task Finished] Transform to mem in 0.226367s (5649 normal rows, 0 exceptions)
[2021-03-24 07:25:14.462] [E/10] [info] [Task Finished] Transform to mem in 0.225943s (5873 normal rows, 0 exceptions)
[2021-03-24 07:25:14.465] [E/7] [info] [Task Finished] Transform to mem in 0.227042s (5686 normal rows, 0 exceptions)
[2021-03-24 07:25:14.470] [E/3] [info] [Task Finished] Transform to mem in 0.227792s (5806 normal rows, 0 exceptions)
[2021-03-24 07:25:14.476] [E/4] [info] [Task Finished] Transform to mem in 0.228183s (5765 normal rows, 0 exceptions)
[2021-03-24 07:25:14.673] [E/14] [info] [Task Finished] Transform to mem in 0.224499s (5770 normal rows, 0 exceptions)
[2021-03-24 07:25:14.675] [E/12] [info] [Task Finished] Transform to mem in 0.225281s (5951 normal rows, 0 exceptions)
[2021-03-24 07:25:14.675] [E/15] [info] [Task Finished] Transform to mem in 0.224992s (5759 normal rows, 0 exceptions)
[2021-03-24 07:25:14.676] [E/8] [info] [Task Finished] Transform to mem in 0.226265s (5643 normal rows, 0 exceptions)
[2021-03-24 07:25:14.677] [E/9] [info] [Task Finished] Transform to mem in 0.225644s (5698 normal rows, 0 exceptions)
[2021-03-24 07:25:14.679] [E/6] [info] [Task Finished] Transform to mem in 0.225336s (5955 normal rows, 0 exceptions)
[2021-03-24 07:25:14.679] [driver] [info] [Task Finished] Transform to mem in 0.225317s (5768 normal rows, 0 exceptions)
[2021-03-24 07:25:14.684] [E/1] [info] [Task Finished] Transform to mem in 0.226602s (5772 normal rows, 0 exceptions)
[2021-03-24 07:25:14.685] [E/11] [info] [Task Finished] Transform to mem in 0.226976s (5799 normal rows, 0 exceptions)
[2021-03-24 07:25:14.686] [E/5] [info] [Task Finished] Transform to mem in 0.226755s (5862 normal rows, 0 exceptions)
[2021-03-24 07:25:14.686] [E/13] [info] [Task Finished] Transform to mem in 0.225927s (5856 normal rows, 0 exceptions)
[2021-03-24 07:25:14.687] [E/2] [info] [Task Finished] Transform to mem in 0.229298s (5802 normal rows, 0 exceptions)
[2021-03-24 07:25:14.689] [E/10] [info] [Task Finished] Transform to mem in 0.226756s (5685 normal rows, 0 exceptions)
[2021-03-24 07:25:14.692] [E/7] [info] [Task Finished] Transform to mem in 0.226349s (5751 normal rows, 0 exceptions)
[2021-03-24 07:25:14.699] [E/3] [info] [Task Finished] Transform to mem in 0.228278s (5891 normal rows, 0 exceptions)
[2021-03-24 07:25:14.705] [E/4] [info] [Task Finished] Transform to mem in 0.228635s (5852 normal rows, 0 exceptions)
[2021-03-24 07:25:14.898] [E/14] [info] [Task Finished] Transform to mem in 0.225084s (5722 normal rows, 0 exceptions)
[2021-03-24 07:25:14.899] [E/12] [info] [Task Finished] Transform to mem in 0.224492s (5645 normal rows, 0 exceptions)
[2021-03-24 07:25:14.899] [E/8] [info] [Task Finished] Transform to mem in 0.223868s (5813 normal rows, 0 exceptions)
[2021-03-24 07:25:14.901] [E/15] [info] [Task Finished] Transform to mem in 0.225377s (5788 normal rows, 0 exceptions)
[2021-03-24 07:25:14.902] [E/9] [info] [Task Finished] Transform to mem in 0.225019s (5674 normal rows, 0 exceptions)
[2021-03-24 07:25:14.905] [E/6] [info] [Task Finished] Transform to mem in 0.225644s (5853 normal rows, 0 exceptions)
[2021-03-24 07:25:14.905] [driver] [info] [Task Finished] Transform to mem in 0.225684s (5848 normal rows, 0 exceptions)
[2021-03-24 07:25:14.910] [E/1] [info] [Task Finished] Transform to mem in 0.225483s (5811 normal rows, 0 exceptions)
[2021-03-24 07:25:14.911] [E/11] [info] [Task Finished] Transform to mem in 0.225677s (5732 normal rows, 0 exceptions)
[2021-03-24 07:25:14.912] [E/5] [info] [Task Finished] Transform to mem in 0.226117s (5801 normal rows, 0 exceptions)
[2021-03-24 07:25:14.912] [E/2] [info] [Task Finished] Transform to mem in 0.225378s (5868 normal rows, 0 exceptions)
[2021-03-24 07:25:14.914] [E/13] [info] [Task Finished] Transform to mem in 0.227057s (5827 normal rows, 0 exceptions)
[2021-03-24 07:25:14.915] [E/10] [info] [Task Finished] Transform to mem in 0.226450s (5794 normal rows, 0 exceptions)
[2021-03-24 07:25:14.919] [E/7] [info] [Task Finished] Transform to mem in 0.227850s (5817 normal rows, 0 exceptions)
[2021-03-24 07:25:14.927] [E/3] [info] [Task Finished] Transform to mem in 0.227864s (5696 normal rows, 0 exceptions)
[2021-03-24 07:25:14.933] [E/4] [info] [Task Finished] Transform to mem in 0.228223s (5732 normal rows, 0 exceptions)
[2021-03-24 07:25:15.121] [E/14] [info] [Task Finished] Transform to mem in 0.223450s (5629 normal rows, 0 exceptions)
[2021-03-24 07:25:15.124] [E/12] [info] [Task Finished] Transform to mem in 0.224932s (5789 normal rows, 0 exceptions)
[2021-03-24 07:25:15.124] [E/8] [info] [Task Finished] Transform to mem in 0.224909s (5740 normal rows, 0 exceptions)
[2021-03-24 07:25:15.126] [E/15] [info] [Task Finished] Transform to mem in 0.224934s (5841 normal rows, 0 exceptions)
[2021-03-24 07:25:15.130] [E/9] [info] [Task Finished] Transform to mem in 0.227901s (5857 normal rows, 0 exceptions)
[2021-03-24 07:25:15.130] [global] [info] [Transform Stage] Stage 0 completed 149 load&transform tasks in 2.25281s
[2021-03-24 07:25:15.130] [global] [info] [Transform Stage] Stage 0 total wall clock time: 33.644s, 48740298 input rows, time to process 1 row via fast path: 0.00069027ms

[2021-03-24 07:25:15.130] [global] [info] Writing 861387 rows as output to file (116.34 MB)
[2021-03-24 07:25:15.144] [E/10] [info] [Task Finished] write to file in 0.013409s (51895 rows, 7.01 MB)
[2021-03-24 07:25:15.145] [E/4] [info] [Task Finished] write to file in 0.014168s (57421 rows, 7.75 MB)
[2021-03-24 07:25:15.145] [E/12] [info] [Task Finished] write to file in 0.014338s (57946 rows, 7.83 MB)
[2021-03-24 07:25:15.145] [driver] [info] [Task Finished] write to file in 0.014364s (57887 rows, 7.82 MB)
[2021-03-24 07:25:15.145] [E/14] [info] [Task Finished] write to file in 0.014429s (57812 rows, 7.81 MB)
[2021-03-24 07:25:15.145] [E/9] [info] [Task Finished] write to file in 0.014472s (57440 rows, 7.76 MB)
[2021-03-24 07:25:15.145] [E/11] [info] [Task Finished] write to file in 0.014486s (58082 rows, 7.84 MB)
[2021-03-24 07:25:15.145] [E/8] [info] [Task Finished] write to file in 0.014492s (58015 rows, 7.84 MB)
[2021-03-24 07:25:15.145] [E/5] [info] [Task Finished] write to file in 0.014506s (57870 rows, 7.82 MB)
[2021-03-24 07:25:15.145] [E/3] [info] [Task Finished] write to file in 0.014526s (57598 rows, 7.78 MB)
[2021-03-24 07:25:15.145] [E/13] [info] [Task Finished] write to file in 0.014545s (57768 rows, 7.80 MB)
[2021-03-24 07:25:15.145] [E/7] [info] [Task Finished] write to file in 0.014571s (58044 rows, 7.84 MB)
[2021-03-24 07:25:15.145] [E/6] [info] [Task Finished] write to file in 0.014614s (57865 rows, 7.82 MB)
[2021-03-24 07:25:15.145] [E/2] [info] [Task Finished] write to file in 0.014628s (58008 rows, 7.83 MB)
[2021-03-24 07:25:15.148] [E/1] [info] [Task Finished] write to file in 0.017127s (57736 rows, 7.80 MB)
[2021-03-24 07:25:15.148] [global] [info] writing output took 0.017461s
[2021-03-24 07:25:15.148] [global] [info] [Transform Stage] Stage 0 completed 149 sink tasks in 0.0175377s
[2021-03-24 07:25:15.148] [global] [info] [Transform Stage] Stage 0 took 2.47416s
[2021-03-24 07:25:15.148] [global] [info] Query Execution took 2.52061s. (planning: 0.0464225s, execution: 2.47419s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z1_preprocessed/zillow_Z1_10G.csv']
Tuplex startup time: 0.006124734878540039
Tuplex job time: 2.790982246398926 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 15, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '10G', 'tuplex.env.hostname': '4e1bb76d17c5', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.00941532,"llvm_optimization_time_s":0.135425,"llvm_compilation_time_s":0.0612169,"total_compilation_time_s":0.203762,"sampling_time_s":0.113592,"stages":[{"stage_no":0,"fast_path_wall_time_s":33.644,"fast_path_time_s":2.25283,"fast_path_per_row_time_ns":690.27,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.006124734878540039, "jobTime": 2.790982246398926}
[2021-03-24 07:25:15.172] [local execution engine] [info] Stopped local executor E/1
[2021-03-24 07:25:15.173] [local execution engine] [info] Stopped local executor E/2
[2021-03-24 07:25:15.174] [local execution engine] [info] Stopped local executor E/3
[2021-03-24 07:25:15.175] [local execution engine] [info] Stopped local executor E/4
[2021-03-24 07:25:15.175] [local execution engine] [info] Stopped local executor E/5
[2021-03-24 07:25:15.176] [local execution engine] [info] Stopped local executor E/6
[2021-03-24 07:25:15.177] [local execution engine] [info] Stopped local executor E/7
[2021-03-24 07:25:15.178] [local execution engine] [info] Stopped local executor E/8
[2021-03-24 07:25:15.178] [local execution engine] [info] Stopped local executor E/9
[2021-03-24 07:25:15.179] [local execution engine] [info] Stopped local executor E/10
[2021-03-24 07:25:15.180] [local execution engine] [info] Stopped local executor E/11
[2021-03-24 07:25:15.180] [local execution engine] [info] Stopped local executor E/12
[2021-03-24 07:25:15.181] [local execution engine] [info] Stopped local executor E/13
[2021-03-24 07:25:15.182] [local execution engine] [info] Stopped local executor E/14
[2021-03-24 07:25:15.183] [local execution engine] [info] Stopped local executor E/15
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
