[2021-03-31 18:13:35.791] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:13:35.791] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:13:35.791] [local ee] [info] initializing LLVM backend
[2021-03-31 18:13:35.791] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:13:35.791] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/1 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/1] [info] starting detached process queue
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/2 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/2] [info] starting detached process queue
[2021-03-31 18:13:35.792] [E/1] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.792] [E/2] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/3 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/3] [info] starting detached process queue
[2021-03-31 18:13:35.792] [E/3] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/4 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/4] [info] starting detached process queue
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/5 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/4] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.792] [E/5] [info] starting detached process queue
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/6 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/6] [info] starting detached process queue
[2021-03-31 18:13:35.792] [E/5] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.792] [local execution engine] [info] started local executor E/7 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.792] [E/6] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/7] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/8 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/7] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/8] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/9 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/8] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/9] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/10 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/10] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/11 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/9] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/10] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/11] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/12 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/12] [info] starting detached process queue
[2021-03-31 18:13:35.793] [E/11] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/13 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/12] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/14 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/13] [info] starting detached process queue
[2021-03-31 18:13:35.793] [E/14] [info] starting detached process queue
[2021-03-31 18:13:35.793] [local execution engine] [info] started local executor E/15 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:35.793] [E/13] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/14] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [E/15] [info] starting detached process queue
[2021-03-31 18:13:35.793] [memory] [info] allocated bitmap managed memory region (10.00 GB, 32.00 MB block size)
[2021-03-31 18:13:35.793] [local execution engine] [info] started driver (10.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:13:35.793] [E/15] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:35.793] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:13:35.793] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:13:35.927] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:35.943] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:35.951] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:35.978] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:35.998] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:36.012] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:36.044] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:36.061] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:36.063] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:36.066] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:36.068] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:36.084] [logical planner] [info] logical optimization took 0.020330ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:13:36.084] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:13:36.085] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:13:36.085] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:13:36.086] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:13:36.086] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:13:36.087] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:13:36.088] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:13:36.089] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:13:36.089] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:13:36.090] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:13:36.091] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:13:36.092] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:13:36.093] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:13:36.093] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:13:36.118] [global] [info] Stage0 depends on: 
[2021-03-31 18:13:36.263] [global] [info] Optimization via LLVM passes took 0.136896 ms
[2021-03-31 18:13:36.324] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:13:36.325] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.206067s
[2021-03-31 18:13:36.325] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:13:36.580] [E/10] [info] [Task Finished] Transform to mem in 0.255381s (13920 normal rows, 0 exceptions)
[2021-03-31 18:13:36.580] [E/11] [info] [Task Finished] Transform to mem in 0.255444s (13953 normal rows, 0 exceptions)
[2021-03-31 18:13:36.580] [E/14] [info] [Task Finished] Transform to mem in 0.255564s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:36.580] [E/13] [info] [Task Finished] Transform to mem in 0.255638s (13988 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [driver] [info] [Task Finished] Transform to mem in 0.255734s (14217 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/9] [info] [Task Finished] Transform to mem in 0.255920s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/15] [info] [Task Finished] Transform to mem in 0.256116s (14020 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/8] [info] [Task Finished] Transform to mem in 0.256157s (14242 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/7] [info] [Task Finished] Transform to mem in 0.256192s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/4] [info] [Task Finished] Transform to mem in 0.256300s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/2] [info] [Task Finished] Transform to mem in 0.256482s (14162 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/1] [info] [Task Finished] Transform to mem in 0.256494s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/3] [info] [Task Finished] Transform to mem in 0.256496s (14177 normal rows, 0 exceptions)
[2021-03-31 18:13:36.581] [E/5] [info] [Task Finished] Transform to mem in 0.256660s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:36.582] [E/12] [info] [Task Finished] Transform to mem in 0.256761s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:36.596] [E/6] [info] [Task Finished] Transform to mem in 0.271110s (14207 normal rows, 0 exceptions)
[2021-03-31 18:13:36.833] [E/11] [info] [Task Finished] Transform to mem in 0.252515s (14059 normal rows, 0 exceptions)
[2021-03-31 18:13:36.833] [E/10] [info] [Task Finished] Transform to mem in 0.252868s (14241 normal rows, 0 exceptions)
[2021-03-31 18:13:36.833] [E/13] [info] [Task Finished] Transform to mem in 0.252830s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/9] [info] [Task Finished] Transform to mem in 0.252741s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/8] [info] [Task Finished] Transform to mem in 0.252554s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/4] [info] [Task Finished] Transform to mem in 0.252627s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/7] [info] [Task Finished] Transform to mem in 0.252764s (14097 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/1] [info] [Task Finished] Transform to mem in 0.252648s (14100 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/2] [info] [Task Finished] Transform to mem in 0.252786s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/15] [info] [Task Finished] Transform to mem in 0.253345s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/12] [info] [Task Finished] Transform to mem in 0.252780s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/3] [info] [Task Finished] Transform to mem in 0.253067s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:36.834] [E/5] [info] [Task Finished] Transform to mem in 0.252991s (14099 normal rows, 0 exceptions)
[2021-03-31 18:13:36.837] [driver] [info] [Task Finished] Transform to mem in 0.256477s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:36.839] [E/14] [info] [Task Finished] Transform to mem in 0.258273s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:36.864] [E/6] [info] [Task Finished] Transform to mem in 0.267946s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:37.089] [E/10] [info] [Task Finished] Transform to mem in 0.256095s (14083 normal rows, 0 exceptions)
[2021-03-31 18:13:37.089] [E/9] [info] [Task Finished] Transform to mem in 0.255935s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:37.090] [E/8] [info] [Task Finished] Transform to mem in 0.255987s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:37.090] [E/1] [info] [Task Finished] Transform to mem in 0.255581s (13961 normal rows, 0 exceptions)
[2021-03-31 18:13:37.090] [E/13] [info] [Task Finished] Transform to mem in 0.256736s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:37.090] [E/12] [info] [Task Finished] Transform to mem in 0.255929s (13915 normal rows, 0 exceptions)
[2021-03-31 18:13:37.091] [E/4] [info] [Task Finished] Transform to mem in 0.257015s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:37.091] [E/15] [info] [Task Finished] Transform to mem in 0.256488s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:37.091] [E/2] [info] [Task Finished] Transform to mem in 0.256768s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:37.091] [E/5] [info] [Task Finished] Transform to mem in 0.256740s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:37.091] [E/3] [info] [Task Finished] Transform to mem in 0.256935s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:37.093] [E/11] [info] [Task Finished] Transform to mem in 0.259982s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:37.094] [E/7] [info] [Task Finished] Transform to mem in 0.260342s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:37.097] [driver] [info] [Task Finished] Transform to mem in 0.259465s (13892 normal rows, 0 exceptions)
[2021-03-31 18:13:37.101] [E/14] [info] [Task Finished] Transform to mem in 0.261895s (14245 normal rows, 0 exceptions)
[2021-03-31 18:13:37.135] [E/6] [info] [Task Finished] Transform to mem in 0.271278s (14260 normal rows, 0 exceptions)
[2021-03-31 18:13:37.345] [E/10] [info] [Task Finished] Transform to mem in 0.255501s (13978 normal rows, 0 exceptions)
[2021-03-31 18:13:37.345] [E/1] [info] [Task Finished] Transform to mem in 0.255803s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:37.346] [E/8] [info] [Task Finished] Transform to mem in 0.256354s (14234 normal rows, 0 exceptions)
[2021-03-31 18:13:37.346] [E/9] [info] [Task Finished] Transform to mem in 0.256692s (14181 normal rows, 0 exceptions)
[2021-03-31 18:13:37.347] [E/13] [info] [Task Finished] Transform to mem in 0.256459s (14142 normal rows, 0 exceptions)
[2021-03-31 18:13:37.347] [E/4] [info] [Task Finished] Transform to mem in 0.255829s (13982 normal rows, 0 exceptions)
[2021-03-31 18:13:37.347] [E/12] [info] [Task Finished] Transform to mem in 0.256398s (14280 normal rows, 0 exceptions)
[2021-03-31 18:13:37.347] [E/2] [info] [Task Finished] Transform to mem in 0.256402s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:37.347] [E/15] [info] [Task Finished] Transform to mem in 0.256579s (14173 normal rows, 0 exceptions)
[2021-03-31 18:13:37.348] [E/5] [info] [Task Finished] Transform to mem in 0.256290s (14054 normal rows, 0 exceptions)
[2021-03-31 18:13:37.348] [E/3] [info] [Task Finished] Transform to mem in 0.256595s (14129 normal rows, 0 exceptions)
[2021-03-31 18:13:37.354] [E/11] [info] [Task Finished] Transform to mem in 0.261169s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:37.355] [E/7] [info] [Task Finished] Transform to mem in 0.261256s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:37.357] [driver] [info] [Task Finished] Transform to mem in 0.260327s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:37.361] [E/14] [info] [Task Finished] Transform to mem in 0.260014s (14208 normal rows, 0 exceptions)
[2021-03-31 18:13:37.406] [E/6] [info] [Task Finished] Transform to mem in 0.270941s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:37.597] [E/10] [info] [Task Finished] Transform to mem in 0.252619s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:37.598] [E/1] [info] [Task Finished] Transform to mem in 0.252847s (14232 normal rows, 0 exceptions)
[2021-03-31 18:13:37.599] [E/8] [info] [Task Finished] Transform to mem in 0.252582s (14088 normal rows, 0 exceptions)
[2021-03-31 18:13:37.599] [E/9] [info] [Task Finished] Transform to mem in 0.252619s (14106 normal rows, 0 exceptions)
[2021-03-31 18:13:37.599] [E/4] [info] [Task Finished] Transform to mem in 0.252482s (14113 normal rows, 0 exceptions)
[2021-03-31 18:13:37.600] [E/12] [info] [Task Finished] Transform to mem in 0.252812s (14075 normal rows, 0 exceptions)
[2021-03-31 18:13:37.600] [E/13] [info] [Task Finished] Transform to mem in 0.253364s (14211 normal rows, 0 exceptions)
[2021-03-31 18:13:37.600] [E/15] [info] [Task Finished] Transform to mem in 0.252994s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:37.601] [E/5] [info] [Task Finished] Transform to mem in 0.253016s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:37.601] [E/2] [info] [Task Finished] Transform to mem in 0.253379s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:37.602] [E/3] [info] [Task Finished] Transform to mem in 0.253621s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:37.611] [E/11] [info] [Task Finished] Transform to mem in 0.256813s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:37.613] [E/7] [info] [Task Finished] Transform to mem in 0.257567s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:37.614] [driver] [info] [Task Finished] Transform to mem in 0.256746s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:37.617] [E/14] [info] [Task Finished] Transform to mem in 0.256291s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:37.673] [E/6] [info] [Task Finished] Transform to mem in 0.267286s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:37.851] [E/10] [info] [Task Finished] Transform to mem in 0.253458s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:37.851] [E/8] [info] [Task Finished] Transform to mem in 0.252368s (13920 normal rows, 0 exceptions)
[2021-03-31 18:13:37.851] [E/1] [info] [Task Finished] Transform to mem in 0.252963s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:37.852] [E/12] [info] [Task Finished] Transform to mem in 0.252264s (13912 normal rows, 0 exceptions)
[2021-03-31 18:13:37.852] [E/4] [info] [Task Finished] Transform to mem in 0.253083s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:37.852] [E/9] [info] [Task Finished] Transform to mem in 0.253459s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:37.853] [E/13] [info] [Task Finished] Transform to mem in 0.253320s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:37.853] [E/15] [info] [Task Finished] Transform to mem in 0.252935s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:37.853] [E/5] [info] [Task Finished] Transform to mem in 0.252819s (13988 normal rows, 0 exceptions)
[2021-03-31 18:13:37.854] [E/2] [info] [Task Finished] Transform to mem in 0.253256s (14171 normal rows, 0 exceptions)
[2021-03-31 18:13:37.855] [E/3] [info] [Task Finished] Transform to mem in 0.253017s (14234 normal rows, 0 exceptions)
[2021-03-31 18:13:37.867] [E/11] [info] [Task Finished] Transform to mem in 0.256445s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:37.870] [driver] [info] [Task Finished] Transform to mem in 0.256469s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:37.872] [E/7] [info] [Task Finished] Transform to mem in 0.259082s (14176 normal rows, 0 exceptions)
[2021-03-31 18:13:37.873] [E/14] [info] [Task Finished] Transform to mem in 0.256023s (14059 normal rows, 0 exceptions)
[2021-03-31 18:13:37.942] [E/6] [info] [Task Finished] Transform to mem in 0.268063s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:38.107] [E/10] [info] [Task Finished] Transform to mem in 0.256320s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:38.108] [E/1] [info] [Task Finished] Transform to mem in 0.256634s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:38.108] [E/12] [info] [Task Finished] Transform to mem in 0.256553s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:38.108] [E/4] [info] [Task Finished] Transform to mem in 0.256263s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:38.109] [E/9] [info] [Task Finished] Transform to mem in 0.256304s (14091 normal rows, 0 exceptions)
[2021-03-31 18:13:38.109] [E/8] [info] [Task Finished] Transform to mem in 0.257896s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:38.110] [E/5] [info] [Task Finished] Transform to mem in 0.256879s (14113 normal rows, 0 exceptions)
[2021-03-31 18:13:38.110] [E/15] [info] [Task Finished] Transform to mem in 0.256931s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:38.111] [E/13] [info] [Task Finished] Transform to mem in 0.257649s (14218 normal rows, 0 exceptions)
[2021-03-31 18:13:38.111] [E/2] [info] [Task Finished] Transform to mem in 0.256927s (14197 normal rows, 0 exceptions)
[2021-03-31 18:13:38.111] [E/3] [info] [Task Finished] Transform to mem in 0.256688s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:38.128] [E/11] [info] [Task Finished] Transform to mem in 0.260628s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:38.131] [driver] [info] [Task Finished] Transform to mem in 0.260399s (14197 normal rows, 0 exceptions)
[2021-03-31 18:13:38.133] [E/14] [info] [Task Finished] Transform to mem in 0.259861s (14083 normal rows, 0 exceptions)
[2021-03-31 18:13:38.133] [E/7] [info] [Task Finished] Transform to mem in 0.260884s (14115 normal rows, 0 exceptions)
[2021-03-31 18:13:38.213] [E/6] [info] [Task Finished] Transform to mem in 0.271816s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:38.360] [E/10] [info] [Task Finished] Transform to mem in 0.252601s (14144 normal rows, 0 exceptions)
[2021-03-31 18:13:38.361] [E/1] [info] [Task Finished] Transform to mem in 0.252608s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:38.361] [E/9] [info] [Task Finished] Transform to mem in 0.252661s (14010 normal rows, 0 exceptions)
[2021-03-31 18:13:38.361] [E/4] [info] [Task Finished] Transform to mem in 0.252828s (14159 normal rows, 0 exceptions)
[2021-03-31 18:13:38.362] [E/12] [info] [Task Finished] Transform to mem in 0.253586s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:38.362] [E/8] [info] [Task Finished] Transform to mem in 0.253391s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:38.363] [E/15] [info] [Task Finished] Transform to mem in 0.252737s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:38.363] [E/5] [info] [Task Finished] Transform to mem in 0.253139s (14178 normal rows, 0 exceptions)
[2021-03-31 18:13:38.364] [E/3] [info] [Task Finished] Transform to mem in 0.252434s (13901 normal rows, 0 exceptions)
[2021-03-31 18:13:38.364] [E/2] [info] [Task Finished] Transform to mem in 0.253043s (14236 normal rows, 0 exceptions)
[2021-03-31 18:13:38.364] [E/13] [info] [Task Finished] Transform to mem in 0.253412s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:38.386] [E/11] [info] [Task Finished] Transform to mem in 0.257855s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:38.387] [driver] [info] [Task Finished] Transform to mem in 0.256616s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:38.389] [E/14] [info] [Task Finished] Transform to mem in 0.256315s (13953 normal rows, 0 exceptions)
[2021-03-31 18:13:38.393] [E/7] [info] [Task Finished] Transform to mem in 0.259547s (14207 normal rows, 0 exceptions)
[2021-03-31 18:13:38.483] [E/6] [info] [Task Finished] Transform to mem in 0.269249s (14245 normal rows, 0 exceptions)
[2021-03-31 18:13:38.612] [E/10] [info] [Task Finished] Transform to mem in 0.252531s (13999 normal rows, 0 exceptions)
[2021-03-31 18:13:38.614] [E/1] [info] [Task Finished] Transform to mem in 0.253190s (14158 normal rows, 0 exceptions)
[2021-03-31 18:13:38.614] [E/4] [info] [Task Finished] Transform to mem in 0.252583s (14020 normal rows, 0 exceptions)
[2021-03-31 18:13:38.614] [E/9] [info] [Task Finished] Transform to mem in 0.253075s (14242 normal rows, 0 exceptions)
[2021-03-31 18:13:38.615] [E/8] [info] [Task Finished] Transform to mem in 0.253086s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:38.615] [E/12] [info] [Task Finished] Transform to mem in 0.253401s (14173 normal rows, 0 exceptions)
[2021-03-31 18:13:38.616] [E/15] [info] [Task Finished] Transform to mem in 0.252979s (14054 normal rows, 0 exceptions)
[2021-03-31 18:13:38.617] [E/5] [info] [Task Finished] Transform to mem in 0.253393s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:38.617] [E/3] [info] [Task Finished] Transform to mem in 0.253305s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:38.617] [E/2] [info] [Task Finished] Transform to mem in 0.253245s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:38.617] [E/13] [info] [Task Finished] Transform to mem in 0.253049s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:38.643] [E/11] [info] [Task Finished] Transform to mem in 0.257429s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:38.644] [driver] [info] [Task Finished] Transform to mem in 0.256661s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:38.646] [E/14] [info] [Task Finished] Transform to mem in 0.256660s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:38.651] [E/7] [info] [Task Finished] Transform to mem in 0.258733s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:38.751] [E/6] [info] [Task Finished] Transform to mem in 0.267901s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:38.865] [E/10] [info] [Task Finished] Transform to mem in 0.252852s (14100 normal rows, 0 exceptions)
[2021-03-31 18:13:38.867] [E/4] [info] [Task Finished] Transform to mem in 0.252880s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:38.867] [E/1] [info] [Task Finished] Transform to mem in 0.253141s (14217 normal rows, 0 exceptions)
[2021-03-31 18:13:38.868] [E/9] [info] [Task Finished] Transform to mem in 0.253193s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:38.869] [E/12] [info] [Task Finished] Transform to mem in 0.253141s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:38.869] [E/8] [info] [Task Finished] Transform to mem in 0.253733s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:38.869] [E/15] [info] [Task Finished] Transform to mem in 0.253329s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:38.871] [E/2] [info] [Task Finished] Transform to mem in 0.253488s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:38.871] [E/3] [info] [Task Finished] Transform to mem in 0.253681s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:38.871] [E/5] [info] [Task Finished] Transform to mem in 0.254165s (14232 normal rows, 0 exceptions)
[2021-03-31 18:13:38.871] [E/13] [info] [Task Finished] Transform to mem in 0.253978s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:38.900] [driver] [info] [Task Finished] Transform to mem in 0.256032s (13961 normal rows, 0 exceptions)
[2021-03-31 18:13:38.900] [E/11] [info] [Task Finished] Transform to mem in 0.257099s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:38.903] [E/14] [info] [Task Finished] Transform to mem in 0.256878s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:38.909] [E/7] [info] [Task Finished] Transform to mem in 0.257369s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:39.063] [E/6] [info] [Task Finished] Transform to mem in 0.312871s (16320 normal rows, 0 exceptions)
[2021-03-31 18:13:39.063] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 2.73892s
[2021-03-31 18:13:39.063] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.0332s, 54108450 input rows, time to process 1 row via fast path: 0.000758351ms

[2021-03-31 18:13:39.107] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:13:39.132] [E/13] [info] [Task Finished] write to file in 0.024924s (87192 rows, 13.27 MB)
[2021-03-31 18:13:39.147] [E/9] [info] [Task Finished] write to file in 0.040459s (155704 rows, 23.70 MB)
[2021-03-31 18:13:39.148] [E/7] [info] [Task Finished] write to file in 0.041079s (155626 rows, 23.69 MB)
[2021-03-31 18:13:39.148] [E/3] [info] [Task Finished] write to file in 0.041274s (155476 rows, 23.67 MB)
[2021-03-31 18:13:39.148] [E/11] [info] [Task Finished] write to file in 0.041356s (155417 rows, 23.66 MB)
[2021-03-31 18:13:39.148] [E/8] [info] [Task Finished] write to file in 0.041397s (155541 rows, 23.68 MB)
[2021-03-31 18:13:39.148] [E/12] [info] [Task Finished] write to file in 0.041701s (155395 rows, 23.66 MB)
[2021-03-31 18:13:39.149] [driver] [info] [Task Finished] write to file in 0.041863s (155541 rows, 23.68 MB)
[2021-03-31 18:13:39.149] [E/10] [info] [Task Finished] write to file in 0.041935s (155530 rows, 23.67 MB)
[2021-03-31 18:13:39.149] [E/2] [info] [Task Finished] write to file in 0.041946s (155417 rows, 23.66 MB)
[2021-03-31 18:13:39.149] [E/15] [info] [Task Finished] write to file in 0.042007s (155476 rows, 23.67 MB)
[2021-03-31 18:13:39.149] [E/14] [info] [Task Finished] write to file in 0.042013s (155606 rows, 23.69 MB)
[2021-03-31 18:13:39.149] [E/6] [info] [Task Finished] write to file in 0.042129s (155415 rows, 23.66 MB)
[2021-03-31 18:13:39.149] [E/4] [info] [Task Finished] write to file in 0.042220s (155530 rows, 23.67 MB)
[2021-03-31 18:13:39.156] [E/1] [info] [Task Finished] write to file in 0.049352s (155704 rows, 23.70 MB)
[2021-03-31 18:13:39.156] [global] [info] writing output took 0.092626s
[2021-03-31 18:13:39.156] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.0927054s
[2021-03-31 18:13:39.156] [global] [info] [Transform Stage] Stage 0 took 3.03774s
[2021-03-31 18:13:39.156] [global] [info] Query Execution took 3.0952s. (planning: 0.0574244s, execution: 3.03777s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.006071567535400391
Tuplex job time: 3.3643012046813965 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 15, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '10G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0203297,"llvm_optimization_time_s":0.136896,"llvm_compilation_time_s":0.0616756,"total_compilation_time_s":0.206065,"sampling_time_s":0.109099,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.0332,"fast_path_time_s":2.73894,"fast_path_per_row_time_ns":758.351,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.006071567535400391, "jobTime": 3.3643012046813965}
[2021-03-31 18:13:39.181] [local execution engine] [info] Stopped local executor E/1
[2021-03-31 18:13:39.182] [local execution engine] [info] Stopped local executor E/2
[2021-03-31 18:13:39.183] [local execution engine] [info] Stopped local executor E/3
[2021-03-31 18:13:39.184] [local execution engine] [info] Stopped local executor E/4
[2021-03-31 18:13:39.186] [local execution engine] [info] Stopped local executor E/5
[2021-03-31 18:13:39.187] [local execution engine] [info] Stopped local executor E/6
[2021-03-31 18:13:39.188] [local execution engine] [info] Stopped local executor E/7
[2021-03-31 18:13:39.189] [local execution engine] [info] Stopped local executor E/8
[2021-03-31 18:13:39.190] [local execution engine] [info] Stopped local executor E/9
[2021-03-31 18:13:39.192] [local execution engine] [info] Stopped local executor E/10
[2021-03-31 18:13:39.193] [local execution engine] [info] Stopped local executor E/11
[2021-03-31 18:13:39.194] [local execution engine] [info] Stopped local executor E/12
[2021-03-31 18:13:39.195] [local execution engine] [info] Stopped local executor E/13
[2021-03-31 18:13:39.196] [local execution engine] [info] Stopped local executor E/14
[2021-03-31 18:13:39.197] [local execution engine] [info] Stopped local executor E/15
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
