[2021-03-23 12:35:20.013] [global] [error] could not find backend, defaulting to local
[2021-03-23 12:35:20.013] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-23 12:35:20.013] [local ee] [info] initializing LLVM backend
[2021-03-23 12:35:20.013] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-23 12:35:20.013] [LLVM] [info] compiling code for skylake-avx512
[2021-03-23 12:35:20.014] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-23 12:35:20.014] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-23 12:35:20.014] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-23 12:35:20.015] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-23 12:35:20.150] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.165] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.173] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.200] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.220] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.233] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.264] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:35:20.280] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:35:20.283] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:35:20.285] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:35:20.288] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:35:20.303] [logical planner] [info] logical optimization took 0.019939ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-23 12:35:20.303] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:35:20.304] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-23 12:35:20.305] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-23 12:35:20.305] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-23 12:35:20.306] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-23 12:35:20.306] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-23 12:35:20.307] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-23 12:35:20.308] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-23 12:35:20.309] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-23 12:35:20.309] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-23 12:35:20.310] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-23 12:35:20.311] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:35:20.312] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:35:20.313] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:35:20.338] [global] [info] Stage0 depends on: 
[2021-03-23 12:35:20.483] [global] [info] Optimization via LLVM passes took 0.137374 ms
[2021-03-23 12:35:20.545] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-23 12:35:20.545] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.206699s
[2021-03-23 12:35:20.545] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-23 12:35:20.806] [driver] [info] [Task Finished] Transform to mem in 0.260361s (14226 normal rows, 0 exceptions)
[2021-03-23 12:35:21.064] [driver] [info] [Task Finished] Transform to mem in 0.257714s (14206 normal rows, 0 exceptions)
[2021-03-23 12:35:21.321] [driver] [info] [Task Finished] Transform to mem in 0.257888s (13970 normal rows, 0 exceptions)
[2021-03-23 12:35:21.580] [driver] [info] [Task Finished] Transform to mem in 0.258988s (14253 normal rows, 0 exceptions)
[2021-03-23 12:35:21.838] [driver] [info] [Task Finished] Transform to mem in 0.257873s (14217 normal rows, 0 exceptions)
[2021-03-23 12:35:22.096] [driver] [info] [Task Finished] Transform to mem in 0.257232s (13920 normal rows, 0 exceptions)
[2021-03-23 12:35:22.354] [driver] [info] [Task Finished] Transform to mem in 0.258178s (14267 normal rows, 0 exceptions)
[2021-03-23 12:35:22.612] [driver] [info] [Task Finished] Transform to mem in 0.258331s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:22.869] [driver] [info] [Task Finished] Transform to mem in 0.256903s (13953 normal rows, 0 exceptions)
[2021-03-23 12:35:23.127] [driver] [info] [Task Finished] Transform to mem in 0.257832s (14207 normal rows, 0 exceptions)
[2021-03-23 12:35:23.384] [driver] [info] [Task Finished] Transform to mem in 0.257037s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:23.640] [driver] [info] [Task Finished] Transform to mem in 0.256526s (13988 normal rows, 0 exceptions)
[2021-03-23 12:35:23.898] [driver] [info] [Task Finished] Transform to mem in 0.257422s (14177 normal rows, 0 exceptions)
[2021-03-23 12:35:24.156] [driver] [info] [Task Finished] Transform to mem in 0.257621s (14242 normal rows, 0 exceptions)
[2021-03-23 12:35:24.412] [driver] [info] [Task Finished] Transform to mem in 0.256930s (14020 normal rows, 0 exceptions)
[2021-03-23 12:35:24.670] [driver] [info] [Task Finished] Transform to mem in 0.257443s (14162 normal rows, 0 exceptions)
[2021-03-23 12:35:24.927] [driver] [info] [Task Finished] Transform to mem in 0.257267s (14241 normal rows, 0 exceptions)
[2021-03-23 12:35:25.184] [driver] [info] [Task Finished] Transform to mem in 0.256863s (14059 normal rows, 0 exceptions)
[2021-03-23 12:35:25.442] [driver] [info] [Task Finished] Transform to mem in 0.257489s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:25.699] [driver] [info] [Task Finished] Transform to mem in 0.257163s (14238 normal rows, 0 exceptions)
[2021-03-23 12:35:25.956] [driver] [info] [Task Finished] Transform to mem in 0.257213s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:26.214] [driver] [info] [Task Finished] Transform to mem in 0.257559s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:26.471] [driver] [info] [Task Finished] Transform to mem in 0.256990s (14221 normal rows, 0 exceptions)
[2021-03-23 12:35:26.727] [driver] [info] [Task Finished] Transform to mem in 0.256743s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:26.984] [driver] [info] [Task Finished] Transform to mem in 0.256991s (14097 normal rows, 0 exceptions)
[2021-03-23 12:35:27.242] [driver] [info] [Task Finished] Transform to mem in 0.258045s (14212 normal rows, 0 exceptions)
[2021-03-23 12:35:27.499] [driver] [info] [Task Finished] Transform to mem in 0.256252s (14108 normal rows, 0 exceptions)
[2021-03-23 12:35:27.756] [driver] [info] [Task Finished] Transform to mem in 0.256882s (14100 normal rows, 0 exceptions)
[2021-03-23 12:35:28.013] [driver] [info] [Task Finished] Transform to mem in 0.257023s (14212 normal rows, 0 exceptions)
[2021-03-23 12:35:28.270] [driver] [info] [Task Finished] Transform to mem in 0.257009s (14099 normal rows, 0 exceptions)
[2021-03-23 12:35:28.527] [driver] [info] [Task Finished] Transform to mem in 0.257153s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:28.784] [driver] [info] [Task Finished] Transform to mem in 0.257295s (14223 normal rows, 0 exceptions)
[2021-03-23 12:35:29.041] [driver] [info] [Task Finished] Transform to mem in 0.256709s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:29.298] [driver] [info] [Task Finished] Transform to mem in 0.256631s (14083 normal rows, 0 exceptions)
[2021-03-23 12:35:29.555] [driver] [info] [Task Finished] Transform to mem in 0.257485s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:29.812] [driver] [info] [Task Finished] Transform to mem in 0.256874s (14170 normal rows, 0 exceptions)
[2021-03-23 12:35:30.069] [driver] [info] [Task Finished] Transform to mem in 0.257092s (14002 normal rows, 0 exceptions)
[2021-03-23 12:35:30.327] [driver] [info] [Task Finished] Transform to mem in 0.258156s (14226 normal rows, 0 exceptions)
[2021-03-23 12:35:30.584] [driver] [info] [Task Finished] Transform to mem in 0.256624s (14206 normal rows, 0 exceptions)
[2021-03-23 12:35:30.840] [driver] [info] [Task Finished] Transform to mem in 0.256531s (13961 normal rows, 0 exceptions)
[2021-03-23 12:35:31.098] [driver] [info] [Task Finished] Transform to mem in 0.257572s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:31.355] [driver] [info] [Task Finished] Transform to mem in 0.256886s (14233 normal rows, 0 exceptions)
[2021-03-23 12:35:31.612] [driver] [info] [Task Finished] Transform to mem in 0.257101s (13915 normal rows, 0 exceptions)
[2021-03-23 12:35:31.870] [driver] [info] [Task Finished] Transform to mem in 0.257522s (14253 normal rows, 0 exceptions)
[2021-03-23 12:35:32.127] [driver] [info] [Task Finished] Transform to mem in 0.257737s (14267 normal rows, 0 exceptions)
[2021-03-23 12:35:32.385] [driver] [info] [Task Finished] Transform to mem in 0.257262s (13892 normal rows, 0 exceptions)
[2021-03-23 12:35:32.643] [driver] [info] [Task Finished] Transform to mem in 0.258046s (14245 normal rows, 0 exceptions)
[2021-03-23 12:35:32.900] [driver] [info] [Task Finished] Transform to mem in 0.257478s (14260 normal rows, 0 exceptions)
[2021-03-23 12:35:33.157] [driver] [info] [Task Finished] Transform to mem in 0.256563s (13978 normal rows, 0 exceptions)
[2021-03-23 12:35:33.414] [driver] [info] [Task Finished] Transform to mem in 0.257541s (14181 normal rows, 0 exceptions)
[2021-03-23 12:35:33.672] [driver] [info] [Task Finished] Transform to mem in 0.257490s (14234 normal rows, 0 exceptions)
[2021-03-23 12:35:33.928] [driver] [info] [Task Finished] Transform to mem in 0.256181s (14015 normal rows, 0 exceptions)
[2021-03-23 12:35:34.186] [driver] [info] [Task Finished] Transform to mem in 0.257896s (14142 normal rows, 0 exceptions)
[2021-03-23 12:35:34.443] [driver] [info] [Task Finished] Transform to mem in 0.257376s (14280 normal rows, 0 exceptions)
[2021-03-23 12:35:34.700] [driver] [info] [Task Finished] Transform to mem in 0.256860s (13982 normal rows, 0 exceptions)
[2021-03-23 12:35:34.957] [driver] [info] [Task Finished] Transform to mem in 0.257150s (14173 normal rows, 0 exceptions)
[2021-03-23 12:35:35.215] [driver] [info] [Task Finished] Transform to mem in 0.257833s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:35.472] [driver] [info] [Task Finished] Transform to mem in 0.256481s (14054 normal rows, 0 exceptions)
[2021-03-23 12:35:35.728] [driver] [info] [Task Finished] Transform to mem in 0.256719s (14129 normal rows, 0 exceptions)
[2021-03-23 12:35:35.986] [driver] [info] [Task Finished] Transform to mem in 0.257175s (14223 normal rows, 0 exceptions)
[2021-03-23 12:35:36.242] [driver] [info] [Task Finished] Transform to mem in 0.256639s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:36.499] [driver] [info] [Task Finished] Transform to mem in 0.256807s (14108 normal rows, 0 exceptions)
[2021-03-23 12:35:36.756] [driver] [info] [Task Finished] Transform to mem in 0.256790s (14208 normal rows, 0 exceptions)
[2021-03-23 12:35:37.013] [driver] [info] [Task Finished] Transform to mem in 0.256731s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:37.270] [driver] [info] [Task Finished] Transform to mem in 0.257131s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:37.527] [driver] [info] [Task Finished] Transform to mem in 0.257218s (14232 normal rows, 0 exceptions)
[2021-03-23 12:35:37.784] [driver] [info] [Task Finished] Transform to mem in 0.256919s (14088 normal rows, 0 exceptions)
[2021-03-23 12:35:38.041] [driver] [info] [Task Finished] Transform to mem in 0.256863s (14106 normal rows, 0 exceptions)
[2021-03-23 12:35:38.298] [driver] [info] [Task Finished] Transform to mem in 0.257212s (14211 normal rows, 0 exceptions)
[2021-03-23 12:35:38.555] [driver] [info] [Task Finished] Transform to mem in 0.256754s (14113 normal rows, 0 exceptions)
[2021-03-23 12:35:38.812] [driver] [info] [Task Finished] Transform to mem in 0.257220s (14075 normal rows, 0 exceptions)
[2021-03-23 12:35:39.069] [driver] [info] [Task Finished] Transform to mem in 0.257412s (14223 normal rows, 0 exceptions)
[2021-03-23 12:35:39.327] [driver] [info] [Task Finished] Transform to mem in 0.257045s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:39.584] [driver] [info] [Task Finished] Transform to mem in 0.257591s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:39.842] [driver] [info] [Task Finished] Transform to mem in 0.257744s (14226 normal rows, 0 exceptions)
[2021-03-23 12:35:40.099] [driver] [info] [Task Finished] Transform to mem in 0.257221s (14170 normal rows, 0 exceptions)
[2021-03-23 12:35:40.356] [driver] [info] [Task Finished] Transform to mem in 0.257184s (14002 normal rows, 0 exceptions)
[2021-03-23 12:35:40.614] [driver] [info] [Task Finished] Transform to mem in 0.257909s (14226 normal rows, 0 exceptions)
[2021-03-23 12:35:40.872] [driver] [info] [Task Finished] Transform to mem in 0.257430s (14206 normal rows, 0 exceptions)
[2021-03-23 12:35:41.128] [driver] [info] [Task Finished] Transform to mem in 0.256746s (13970 normal rows, 0 exceptions)
[2021-03-23 12:35:41.386] [driver] [info] [Task Finished] Transform to mem in 0.257995s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:41.644] [driver] [info] [Task Finished] Transform to mem in 0.257378s (14233 normal rows, 0 exceptions)
[2021-03-23 12:35:41.901] [driver] [info] [Task Finished] Transform to mem in 0.256856s (13920 normal rows, 0 exceptions)
[2021-03-23 12:35:42.159] [driver] [info] [Task Finished] Transform to mem in 0.257957s (14267 normal rows, 0 exceptions)
[2021-03-23 12:35:42.416] [driver] [info] [Task Finished] Transform to mem in 0.257318s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:42.673] [driver] [info] [Task Finished] Transform to mem in 0.256578s (13912 normal rows, 0 exceptions)
[2021-03-23 12:35:42.930] [driver] [info] [Task Finished] Transform to mem in 0.257772s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:43.188] [driver] [info] [Task Finished] Transform to mem in 0.257156s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:43.444] [driver] [info] [Task Finished] Transform to mem in 0.256699s (13988 normal rows, 0 exceptions)
[2021-03-23 12:35:43.702] [driver] [info] [Task Finished] Transform to mem in 0.257282s (14171 normal rows, 0 exceptions)
[2021-03-23 12:35:43.959] [driver] [info] [Task Finished] Transform to mem in 0.257087s (14234 normal rows, 0 exceptions)
[2021-03-23 12:35:44.216] [driver] [info] [Task Finished] Transform to mem in 0.256803s (14015 normal rows, 0 exceptions)
[2021-03-23 12:35:44.474] [driver] [info] [Task Finished] Transform to mem in 0.257955s (14176 normal rows, 0 exceptions)
[2021-03-23 12:35:44.731] [driver] [info] [Task Finished] Transform to mem in 0.257340s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:44.988] [driver] [info] [Task Finished] Transform to mem in 0.257442s (14059 normal rows, 0 exceptions)
[2021-03-23 12:35:45.246] [driver] [info] [Task Finished] Transform to mem in 0.257613s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:45.503] [driver] [info] [Task Finished] Transform to mem in 0.257211s (14238 normal rows, 0 exceptions)
[2021-03-23 12:35:45.760] [driver] [info] [Task Finished] Transform to mem in 0.257199s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:46.018] [driver] [info] [Task Finished] Transform to mem in 0.257415s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:46.275] [driver] [info] [Task Finished] Transform to mem in 0.257292s (14221 normal rows, 0 exceptions)
[2021-03-23 12:35:46.533] [driver] [info] [Task Finished] Transform to mem in 0.257498s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:46.790] [driver] [info] [Task Finished] Transform to mem in 0.257296s (14091 normal rows, 0 exceptions)
[2021-03-23 12:35:47.047] [driver] [info] [Task Finished] Transform to mem in 0.257375s (14218 normal rows, 0 exceptions)
[2021-03-23 12:35:47.304] [driver] [info] [Task Finished] Transform to mem in 0.256916s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:47.561] [driver] [info] [Task Finished] Transform to mem in 0.256652s (14113 normal rows, 0 exceptions)
[2021-03-23 12:35:47.818] [driver] [info] [Task Finished] Transform to mem in 0.256989s (14197 normal rows, 0 exceptions)
[2021-03-23 12:35:48.076] [driver] [info] [Task Finished] Transform to mem in 0.257506s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:48.333] [driver] [info] [Task Finished] Transform to mem in 0.257212s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:48.590] [driver] [info] [Task Finished] Transform to mem in 0.257490s (14197 normal rows, 0 exceptions)
[2021-03-23 12:35:48.848] [driver] [info] [Task Finished] Transform to mem in 0.257477s (14115 normal rows, 0 exceptions)
[2021-03-23 12:35:49.106] [driver] [info] [Task Finished] Transform to mem in 0.257769s (14083 normal rows, 0 exceptions)
[2021-03-23 12:35:49.364] [driver] [info] [Task Finished] Transform to mem in 0.257987s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:49.622] [driver] [info] [Task Finished] Transform to mem in 0.258167s (14144 normal rows, 0 exceptions)
[2021-03-23 12:35:49.880] [driver] [info] [Task Finished] Transform to mem in 0.257821s (14015 normal rows, 0 exceptions)
[2021-03-23 12:35:50.139] [driver] [info] [Task Finished] Transform to mem in 0.259068s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:50.398] [driver] [info] [Task Finished] Transform to mem in 0.259219s (14159 normal rows, 0 exceptions)
[2021-03-23 12:35:50.656] [driver] [info] [Task Finished] Transform to mem in 0.258005s (14010 normal rows, 0 exceptions)
[2021-03-23 12:35:50.915] [driver] [info] [Task Finished] Transform to mem in 0.259052s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:51.173] [driver] [info] [Task Finished] Transform to mem in 0.258265s (14178 normal rows, 0 exceptions)
[2021-03-23 12:35:51.431] [driver] [info] [Task Finished] Transform to mem in 0.258153s (13970 normal rows, 0 exceptions)
[2021-03-23 12:35:51.690] [driver] [info] [Task Finished] Transform to mem in 0.258727s (14253 normal rows, 0 exceptions)
[2021-03-23 12:35:51.948] [driver] [info] [Task Finished] Transform to mem in 0.257796s (14236 normal rows, 0 exceptions)
[2021-03-23 12:35:52.205] [driver] [info] [Task Finished] Transform to mem in 0.257361s (13901 normal rows, 0 exceptions)
[2021-03-23 12:35:52.464] [driver] [info] [Task Finished] Transform to mem in 0.258487s (14267 normal rows, 0 exceptions)
[2021-03-23 12:35:52.722] [driver] [info] [Task Finished] Transform to mem in 0.258463s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:52.980] [driver] [info] [Task Finished] Transform to mem in 0.257269s (13953 normal rows, 0 exceptions)
[2021-03-23 12:35:53.238] [driver] [info] [Task Finished] Transform to mem in 0.258094s (14207 normal rows, 0 exceptions)
[2021-03-23 12:35:53.495] [driver] [info] [Task Finished] Transform to mem in 0.257559s (14245 normal rows, 0 exceptions)
[2021-03-23 12:35:53.753] [driver] [info] [Task Finished] Transform to mem in 0.257700s (13999 normal rows, 0 exceptions)
[2021-03-23 12:35:54.012] [driver] [info] [Task Finished] Transform to mem in 0.258704s (14158 normal rows, 0 exceptions)
[2021-03-23 12:35:54.269] [driver] [info] [Task Finished] Transform to mem in 0.257262s (14242 normal rows, 0 exceptions)
[2021-03-23 12:35:54.526] [driver] [info] [Task Finished] Transform to mem in 0.257322s (14020 normal rows, 0 exceptions)
[2021-03-23 12:35:54.785] [driver] [info] [Task Finished] Transform to mem in 0.258619s (14173 normal rows, 0 exceptions)
[2021-03-23 12:35:55.043] [driver] [info] [Task Finished] Transform to mem in 0.258226s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:55.301] [driver] [info] [Task Finished] Transform to mem in 0.257838s (14054 normal rows, 0 exceptions)
[2021-03-23 12:35:55.559] [driver] [info] [Task Finished] Transform to mem in 0.258275s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:55.818] [driver] [info] [Task Finished] Transform to mem in 0.258262s (14238 normal rows, 0 exceptions)
[2021-03-23 12:35:56.076] [driver] [info] [Task Finished] Transform to mem in 0.258007s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:56.333] [driver] [info] [Task Finished] Transform to mem in 0.257773s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:56.591] [driver] [info] [Task Finished] Transform to mem in 0.257754s (14221 normal rows, 0 exceptions)
[2021-03-23 12:35:56.849] [driver] [info] [Task Finished] Transform to mem in 0.257773s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:57.108] [driver] [info] [Task Finished] Transform to mem in 0.258672s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:57.366] [driver] [info] [Task Finished] Transform to mem in 0.258616s (14212 normal rows, 0 exceptions)
[2021-03-23 12:35:57.624] [driver] [info] [Task Finished] Transform to mem in 0.257218s (14108 normal rows, 0 exceptions)
[2021-03-23 12:35:57.881] [driver] [info] [Task Finished] Transform to mem in 0.257885s (14100 normal rows, 0 exceptions)
[2021-03-23 12:35:58.140] [driver] [info] [Task Finished] Transform to mem in 0.258412s (14217 normal rows, 0 exceptions)
[2021-03-23 12:35:58.397] [driver] [info] [Task Finished] Transform to mem in 0.257431s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:58.656] [driver] [info] [Task Finished] Transform to mem in 0.258537s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:58.914] [driver] [info] [Task Finished] Transform to mem in 0.258282s (14223 normal rows, 0 exceptions)
[2021-03-23 12:35:59.172] [driver] [info] [Task Finished] Transform to mem in 0.257942s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:59.430] [driver] [info] [Task Finished] Transform to mem in 0.258113s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:59.689] [driver] [info] [Task Finished] Transform to mem in 0.258785s (14232 normal rows, 0 exceptions)
[2021-03-23 12:35:59.948] [driver] [info] [Task Finished] Transform to mem in 0.258387s (14170 normal rows, 0 exceptions)
[2021-03-23 12:36:00.205] [driver] [info] [Task Finished] Transform to mem in 0.257871s (14002 normal rows, 0 exceptions)
[2021-03-23 12:36:00.464] [driver] [info] [Task Finished] Transform to mem in 0.258719s (14226 normal rows, 0 exceptions)
[2021-03-23 12:36:00.723] [driver] [info] [Task Finished] Transform to mem in 0.258658s (14206 normal rows, 0 exceptions)
[2021-03-23 12:36:00.981] [driver] [info] [Task Finished] Transform to mem in 0.257734s (13961 normal rows, 0 exceptions)
[2021-03-23 12:36:01.239] [driver] [info] [Task Finished] Transform to mem in 0.258340s (14246 normal rows, 0 exceptions)
[2021-03-23 12:36:01.497] [driver] [info] [Task Finished] Transform to mem in 0.258309s (14233 normal rows, 0 exceptions)
[2021-03-23 12:36:01.799] [driver] [info] [Task Finished] Transform to mem in 0.301662s (16320 normal rows, 0 exceptions)
[2021-03-23 12:36:01.799] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 41.2538s
[2021-03-23 12:36:01.799] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.2509s, 54108450 input rows, time to process 1 row via fast path: 0.000762374ms

[2021-03-23 12:36:01.840] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-23 12:36:02.228] [driver] [info] [Task Finished] write to file in 0.388117s (2264570 rows, 344.74 MB)
[2021-03-23 12:36:02.228] [global] [info] writing output took 0.429081s
[2021-03-23 12:36:02.228] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.429167s
[2021-03-23 12:36:02.228] [global] [info] [Transform Stage] Stage 0 took 41.8897s
[2021-03-23 12:36:02.228] [global] [info] Query Execution took 41.9475s. (planning: 0.0577277s, execution: 41.8897s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.005026102066040039
Tuplex job time: 42.21517014503479 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '4e1bb76d17c5', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '4MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0199394,"llvm_optimization_time_s":0.137374,"llvm_compilation_time_s":0.0619763,"total_compilation_time_s":0.206697,"sampling_time_s":0.111509,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.2509,"fast_path_time_s":41.2538,"fast_path_per_row_time_ns":762.374,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.005026102066040039, "jobTime": 42.21517014503479}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
