[2021-03-31 18:28:14.795] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:28:14.796] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:28:14.796] [local ee] [info] initializing LLVM backend
[2021-03-31 18:28:14.796] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:28:14.796] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:28:14.797] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:28:14.797] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:28:14.797] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:28:14.797] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:28:14.934] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:14.949] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:14.957] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:14.983] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:15.003] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:15.015] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:15.046] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:28:15.062] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:28:15.065] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:28:15.067] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:28:15.069] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:28:15.085] [logical planner] [info] logical optimization took 0.019568ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:28:15.085] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:28:15.085] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:28:15.086] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:28:15.087] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:28:15.087] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:28:15.088] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:28:15.088] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:28:15.089] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:28:15.090] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:28:15.091] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:28:15.092] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:28:15.093] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:28:15.094] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:28:15.094] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:28:15.120] [global] [info] Stage0 depends on: 
[2021-03-31 18:28:15.263] [global] [info] Optimization via LLVM passes took 0.136185 ms
[2021-03-31 18:28:15.325] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:28:15.325] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.205035s
[2021-03-31 18:28:15.325] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:28:15.584] [driver] [info] [Task Finished] Transform to mem in 0.259311s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:15.841] [driver] [info] [Task Finished] Transform to mem in 0.256773s (14206 normal rows, 0 exceptions)
[2021-03-31 18:28:16.097] [driver] [info] [Task Finished] Transform to mem in 0.256251s (13970 normal rows, 0 exceptions)
[2021-03-31 18:28:16.355] [driver] [info] [Task Finished] Transform to mem in 0.257299s (14253 normal rows, 0 exceptions)
[2021-03-31 18:28:16.611] [driver] [info] [Task Finished] Transform to mem in 0.256800s (14217 normal rows, 0 exceptions)
[2021-03-31 18:28:16.867] [driver] [info] [Task Finished] Transform to mem in 0.256041s (13920 normal rows, 0 exceptions)
[2021-03-31 18:28:17.125] [driver] [info] [Task Finished] Transform to mem in 0.257044s (14267 normal rows, 0 exceptions)
[2021-03-31 18:28:17.384] [driver] [info] [Task Finished] Transform to mem in 0.259835s (14248 normal rows, 0 exceptions)
[2021-03-31 18:28:17.640] [driver] [info] [Task Finished] Transform to mem in 0.255727s (13953 normal rows, 0 exceptions)
[2021-03-31 18:28:17.900] [driver] [info] [Task Finished] Transform to mem in 0.259903s (14207 normal rows, 0 exceptions)
[2021-03-31 18:28:18.164] [driver] [info] [Task Finished] Transform to mem in 0.264269s (14237 normal rows, 0 exceptions)
[2021-03-31 18:28:18.420] [driver] [info] [Task Finished] Transform to mem in 0.255964s (13988 normal rows, 0 exceptions)
[2021-03-31 18:28:18.678] [driver] [info] [Task Finished] Transform to mem in 0.257394s (14177 normal rows, 0 exceptions)
[2021-03-31 18:28:18.934] [driver] [info] [Task Finished] Transform to mem in 0.256679s (14242 normal rows, 0 exceptions)
[2021-03-31 18:28:19.190] [driver] [info] [Task Finished] Transform to mem in 0.255939s (14020 normal rows, 0 exceptions)
[2021-03-31 18:28:19.447] [driver] [info] [Task Finished] Transform to mem in 0.256559s (14162 normal rows, 0 exceptions)
[2021-03-31 18:28:19.704] [driver] [info] [Task Finished] Transform to mem in 0.256676s (14241 normal rows, 0 exceptions)
[2021-03-31 18:28:19.960] [driver] [info] [Task Finished] Transform to mem in 0.256087s (14059 normal rows, 0 exceptions)
[2021-03-31 18:28:20.217] [driver] [info] [Task Finished] Transform to mem in 0.256767s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:20.473] [driver] [info] [Task Finished] Transform to mem in 0.256644s (14238 normal rows, 0 exceptions)
[2021-03-31 18:28:20.729] [driver] [info] [Task Finished] Transform to mem in 0.255991s (14081 normal rows, 0 exceptions)
[2021-03-31 18:28:20.986] [driver] [info] [Task Finished] Transform to mem in 0.256487s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:21.245] [driver] [info] [Task Finished] Transform to mem in 0.259782s (14221 normal rows, 0 exceptions)
[2021-03-31 18:28:21.505] [driver] [info] [Task Finished] Transform to mem in 0.259646s (14086 normal rows, 0 exceptions)
[2021-03-31 18:28:21.762] [driver] [info] [Task Finished] Transform to mem in 0.256674s (14097 normal rows, 0 exceptions)
[2021-03-31 18:28:22.019] [driver] [info] [Task Finished] Transform to mem in 0.256788s (14212 normal rows, 0 exceptions)
[2021-03-31 18:28:22.278] [driver] [info] [Task Finished] Transform to mem in 0.259598s (14108 normal rows, 0 exceptions)
[2021-03-31 18:28:22.535] [driver] [info] [Task Finished] Transform to mem in 0.256418s (14100 normal rows, 0 exceptions)
[2021-03-31 18:28:22.791] [driver] [info] [Task Finished] Transform to mem in 0.256556s (14212 normal rows, 0 exceptions)
[2021-03-31 18:28:23.047] [driver] [info] [Task Finished] Transform to mem in 0.255980s (14099 normal rows, 0 exceptions)
[2021-03-31 18:28:23.304] [driver] [info] [Task Finished] Transform to mem in 0.256508s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:23.560] [driver] [info] [Task Finished] Transform to mem in 0.256631s (14223 normal rows, 0 exceptions)
[2021-03-31 18:28:23.817] [driver] [info] [Task Finished] Transform to mem in 0.256247s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:24.073] [driver] [info] [Task Finished] Transform to mem in 0.256520s (14083 normal rows, 0 exceptions)
[2021-03-31 18:28:24.330] [driver] [info] [Task Finished] Transform to mem in 0.256387s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:24.586] [driver] [info] [Task Finished] Transform to mem in 0.256452s (14170 normal rows, 0 exceptions)
[2021-03-31 18:28:24.846] [driver] [info] [Task Finished] Transform to mem in 0.259714s (14002 normal rows, 0 exceptions)
[2021-03-31 18:28:25.106] [driver] [info] [Task Finished] Transform to mem in 0.260448s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:25.363] [driver] [info] [Task Finished] Transform to mem in 0.256291s (14206 normal rows, 0 exceptions)
[2021-03-31 18:28:25.619] [driver] [info] [Task Finished] Transform to mem in 0.255954s (13961 normal rows, 0 exceptions)
[2021-03-31 18:28:25.879] [driver] [info] [Task Finished] Transform to mem in 0.259953s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:26.135] [driver] [info] [Task Finished] Transform to mem in 0.256658s (14233 normal rows, 0 exceptions)
[2021-03-31 18:28:26.391] [driver] [info] [Task Finished] Transform to mem in 0.255859s (13915 normal rows, 0 exceptions)
[2021-03-31 18:28:26.648] [driver] [info] [Task Finished] Transform to mem in 0.256989s (14253 normal rows, 0 exceptions)
[2021-03-31 18:28:26.905] [driver] [info] [Task Finished] Transform to mem in 0.256803s (14267 normal rows, 0 exceptions)
[2021-03-31 18:28:27.161] [driver] [info] [Task Finished] Transform to mem in 0.256507s (13892 normal rows, 0 exceptions)
[2021-03-31 18:28:27.418] [driver] [info] [Task Finished] Transform to mem in 0.256949s (14245 normal rows, 0 exceptions)
[2021-03-31 18:28:27.675] [driver] [info] [Task Finished] Transform to mem in 0.256947s (14260 normal rows, 0 exceptions)
[2021-03-31 18:28:27.931] [driver] [info] [Task Finished] Transform to mem in 0.255705s (13978 normal rows, 0 exceptions)
[2021-03-31 18:28:28.188] [driver] [info] [Task Finished] Transform to mem in 0.256741s (14181 normal rows, 0 exceptions)
[2021-03-31 18:28:28.448] [driver] [info] [Task Finished] Transform to mem in 0.259807s (14234 normal rows, 0 exceptions)
[2021-03-31 18:28:28.703] [driver] [info] [Task Finished] Transform to mem in 0.255629s (14015 normal rows, 0 exceptions)
[2021-03-31 18:28:28.964] [driver] [info] [Task Finished] Transform to mem in 0.260375s (14142 normal rows, 0 exceptions)
[2021-03-31 18:28:29.221] [driver] [info] [Task Finished] Transform to mem in 0.256884s (14280 normal rows, 0 exceptions)
[2021-03-31 18:28:29.476] [driver] [info] [Task Finished] Transform to mem in 0.255717s (13982 normal rows, 0 exceptions)
[2021-03-31 18:28:29.737] [driver] [info] [Task Finished] Transform to mem in 0.260280s (14173 normal rows, 0 exceptions)
[2021-03-31 18:28:29.993] [driver] [info] [Task Finished] Transform to mem in 0.256642s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:30.249] [driver] [info] [Task Finished] Transform to mem in 0.256223s (14054 normal rows, 0 exceptions)
[2021-03-31 18:28:30.506] [driver] [info] [Task Finished] Transform to mem in 0.256556s (14129 normal rows, 0 exceptions)
[2021-03-31 18:28:30.762] [driver] [info] [Task Finished] Transform to mem in 0.256399s (14223 normal rows, 0 exceptions)
[2021-03-31 18:28:31.019] [driver] [info] [Task Finished] Transform to mem in 0.256224s (14081 normal rows, 0 exceptions)
[2021-03-31 18:28:31.275] [driver] [info] [Task Finished] Transform to mem in 0.256651s (14108 normal rows, 0 exceptions)
[2021-03-31 18:28:31.532] [driver] [info] [Task Finished] Transform to mem in 0.256507s (14208 normal rows, 0 exceptions)
[2021-03-31 18:28:31.788] [driver] [info] [Task Finished] Transform to mem in 0.256258s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:32.045] [driver] [info] [Task Finished] Transform to mem in 0.256446s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:32.304] [driver] [info] [Task Finished] Transform to mem in 0.259663s (14232 normal rows, 0 exceptions)
[2021-03-31 18:28:32.564] [driver] [info] [Task Finished] Transform to mem in 0.259571s (14088 normal rows, 0 exceptions)
[2021-03-31 18:28:32.820] [driver] [info] [Task Finished] Transform to mem in 0.256131s (14106 normal rows, 0 exceptions)
[2021-03-31 18:28:33.077] [driver] [info] [Task Finished] Transform to mem in 0.256770s (14211 normal rows, 0 exceptions)
[2021-03-31 18:28:33.336] [driver] [info] [Task Finished] Transform to mem in 0.259331s (14113 normal rows, 0 exceptions)
[2021-03-31 18:28:33.593] [driver] [info] [Task Finished] Transform to mem in 0.256347s (14075 normal rows, 0 exceptions)
[2021-03-31 18:28:33.849] [driver] [info] [Task Finished] Transform to mem in 0.256511s (14223 normal rows, 0 exceptions)
[2021-03-31 18:28:34.105] [driver] [info] [Task Finished] Transform to mem in 0.256321s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:34.362] [driver] [info] [Task Finished] Transform to mem in 0.256694s (14086 normal rows, 0 exceptions)
[2021-03-31 18:28:34.619] [driver] [info] [Task Finished] Transform to mem in 0.256525s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:34.875] [driver] [info] [Task Finished] Transform to mem in 0.256207s (14170 normal rows, 0 exceptions)
[2021-03-31 18:28:35.131] [driver] [info] [Task Finished] Transform to mem in 0.255816s (14002 normal rows, 0 exceptions)
[2021-03-31 18:28:35.387] [driver] [info] [Task Finished] Transform to mem in 0.256653s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:35.644] [driver] [info] [Task Finished] Transform to mem in 0.256364s (14206 normal rows, 0 exceptions)
[2021-03-31 18:28:35.903] [driver] [info] [Task Finished] Transform to mem in 0.258939s (13970 normal rows, 0 exceptions)
[2021-03-31 18:28:36.163] [driver] [info] [Task Finished] Transform to mem in 0.260068s (14237 normal rows, 0 exceptions)
[2021-03-31 18:28:36.419] [driver] [info] [Task Finished] Transform to mem in 0.256299s (14233 normal rows, 0 exceptions)
[2021-03-31 18:28:36.675] [driver] [info] [Task Finished] Transform to mem in 0.256012s (13920 normal rows, 0 exceptions)
[2021-03-31 18:28:36.935] [driver] [info] [Task Finished] Transform to mem in 0.260089s (14267 normal rows, 0 exceptions)
[2021-03-31 18:28:37.192] [driver] [info] [Task Finished] Transform to mem in 0.256435s (14248 normal rows, 0 exceptions)
[2021-03-31 18:28:37.448] [driver] [info] [Task Finished] Transform to mem in 0.255802s (13912 normal rows, 0 exceptions)
[2021-03-31 18:28:37.705] [driver] [info] [Task Finished] Transform to mem in 0.257016s (14248 normal rows, 0 exceptions)
[2021-03-31 18:28:37.961] [driver] [info] [Task Finished] Transform to mem in 0.256830s (14237 normal rows, 0 exceptions)
[2021-03-31 18:28:38.217] [driver] [info] [Task Finished] Transform to mem in 0.255924s (13988 normal rows, 0 exceptions)
[2021-03-31 18:28:38.474] [driver] [info] [Task Finished] Transform to mem in 0.256835s (14171 normal rows, 0 exceptions)
[2021-03-31 18:28:38.731] [driver] [info] [Task Finished] Transform to mem in 0.256366s (14234 normal rows, 0 exceptions)
[2021-03-31 18:28:38.987] [driver] [info] [Task Finished] Transform to mem in 0.255967s (14015 normal rows, 0 exceptions)
[2021-03-31 18:28:39.243] [driver] [info] [Task Finished] Transform to mem in 0.256478s (14176 normal rows, 0 exceptions)
[2021-03-31 18:28:39.503] [driver] [info] [Task Finished] Transform to mem in 0.260010s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:39.760] [driver] [info] [Task Finished] Transform to mem in 0.256680s (14059 normal rows, 0 exceptions)
[2021-03-31 18:28:40.020] [driver] [info] [Task Finished] Transform to mem in 0.259844s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:40.276] [driver] [info] [Task Finished] Transform to mem in 0.256427s (14238 normal rows, 0 exceptions)
[2021-03-31 18:28:40.533] [driver] [info] [Task Finished] Transform to mem in 0.256798s (14081 normal rows, 0 exceptions)
[2021-03-31 18:28:40.790] [driver] [info] [Task Finished] Transform to mem in 0.256648s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:41.046] [driver] [info] [Task Finished] Transform to mem in 0.256750s (14221 normal rows, 0 exceptions)
[2021-03-31 18:28:41.303] [driver] [info] [Task Finished] Transform to mem in 0.256624s (14086 normal rows, 0 exceptions)
[2021-03-31 18:28:41.559] [driver] [info] [Task Finished] Transform to mem in 0.256453s (14091 normal rows, 0 exceptions)
[2021-03-31 18:28:41.816] [driver] [info] [Task Finished] Transform to mem in 0.256775s (14218 normal rows, 0 exceptions)
[2021-03-31 18:28:42.073] [driver] [info] [Task Finished] Transform to mem in 0.256512s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:42.329] [driver] [info] [Task Finished] Transform to mem in 0.256246s (14113 normal rows, 0 exceptions)
[2021-03-31 18:28:42.586] [driver] [info] [Task Finished] Transform to mem in 0.256557s (14197 normal rows, 0 exceptions)
[2021-03-31 18:28:42.842] [driver] [info] [Task Finished] Transform to mem in 0.256365s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:43.098] [driver] [info] [Task Finished] Transform to mem in 0.255817s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:43.358] [driver] [info] [Task Finished] Transform to mem in 0.259710s (14197 normal rows, 0 exceptions)
[2021-03-31 18:28:43.617] [driver] [info] [Task Finished] Transform to mem in 0.259435s (14115 normal rows, 0 exceptions)
[2021-03-31 18:28:43.873] [driver] [info] [Task Finished] Transform to mem in 0.256417s (14083 normal rows, 0 exceptions)
[2021-03-31 18:28:44.130] [driver] [info] [Task Finished] Transform to mem in 0.256628s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:44.386] [driver] [info] [Task Finished] Transform to mem in 0.256434s (14144 normal rows, 0 exceptions)
[2021-03-31 18:28:44.642] [driver] [info] [Task Finished] Transform to mem in 0.255907s (14015 normal rows, 0 exceptions)
[2021-03-31 18:28:44.899] [driver] [info] [Task Finished] Transform to mem in 0.256261s (14237 normal rows, 0 exceptions)
[2021-03-31 18:28:45.155] [driver] [info] [Task Finished] Transform to mem in 0.256582s (14159 normal rows, 0 exceptions)
[2021-03-31 18:28:45.412] [driver] [info] [Task Finished] Transform to mem in 0.256224s (14010 normal rows, 0 exceptions)
[2021-03-31 18:28:45.669] [driver] [info] [Task Finished] Transform to mem in 0.257090s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:45.926] [driver] [info] [Task Finished] Transform to mem in 0.257449s (14178 normal rows, 0 exceptions)
[2021-03-31 18:28:46.182] [driver] [info] [Task Finished] Transform to mem in 0.256133s (13970 normal rows, 0 exceptions)
[2021-03-31 18:28:46.439] [driver] [info] [Task Finished] Transform to mem in 0.256680s (14253 normal rows, 0 exceptions)
[2021-03-31 18:28:46.695] [driver] [info] [Task Finished] Transform to mem in 0.256249s (14236 normal rows, 0 exceptions)
[2021-03-31 18:28:46.954] [driver] [info] [Task Finished] Transform to mem in 0.259012s (13901 normal rows, 0 exceptions)
[2021-03-31 18:28:47.214] [driver] [info] [Task Finished] Transform to mem in 0.260249s (14267 normal rows, 0 exceptions)
[2021-03-31 18:28:47.471] [driver] [info] [Task Finished] Transform to mem in 0.256394s (14248 normal rows, 0 exceptions)
[2021-03-31 18:28:47.726] [driver] [info] [Task Finished] Transform to mem in 0.255572s (13953 normal rows, 0 exceptions)
[2021-03-31 18:28:47.983] [driver] [info] [Task Finished] Transform to mem in 0.256958s (14207 normal rows, 0 exceptions)
[2021-03-31 18:28:48.240] [driver] [info] [Task Finished] Transform to mem in 0.256400s (14245 normal rows, 0 exceptions)
[2021-03-31 18:28:48.496] [driver] [info] [Task Finished] Transform to mem in 0.256107s (13999 normal rows, 0 exceptions)
[2021-03-31 18:28:48.753] [driver] [info] [Task Finished] Transform to mem in 0.256553s (14158 normal rows, 0 exceptions)
[2021-03-31 18:28:49.009] [driver] [info] [Task Finished] Transform to mem in 0.256785s (14242 normal rows, 0 exceptions)
[2021-03-31 18:28:49.266] [driver] [info] [Task Finished] Transform to mem in 0.256363s (14020 normal rows, 0 exceptions)
[2021-03-31 18:28:49.523] [driver] [info] [Task Finished] Transform to mem in 0.256918s (14173 normal rows, 0 exceptions)
[2021-03-31 18:28:49.779] [driver] [info] [Task Finished] Transform to mem in 0.256580s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:50.035] [driver] [info] [Task Finished] Transform to mem in 0.256224s (14054 normal rows, 0 exceptions)
[2021-03-31 18:28:50.292] [driver] [info] [Task Finished] Transform to mem in 0.256210s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:50.551] [driver] [info] [Task Finished] Transform to mem in 0.259695s (14238 normal rows, 0 exceptions)
[2021-03-31 18:28:50.809] [driver] [info] [Task Finished] Transform to mem in 0.257733s (14081 normal rows, 0 exceptions)
[2021-03-31 18:28:51.067] [driver] [info] [Task Finished] Transform to mem in 0.258190s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:51.324] [driver] [info] [Task Finished] Transform to mem in 0.256829s (14221 normal rows, 0 exceptions)
[2021-03-31 18:28:51.581] [driver] [info] [Task Finished] Transform to mem in 0.256297s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:51.837] [driver] [info] [Task Finished] Transform to mem in 0.256524s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:52.094] [driver] [info] [Task Finished] Transform to mem in 0.257129s (14212 normal rows, 0 exceptions)
[2021-03-31 18:28:52.351] [driver] [info] [Task Finished] Transform to mem in 0.256619s (14108 normal rows, 0 exceptions)
[2021-03-31 18:28:52.608] [driver] [info] [Task Finished] Transform to mem in 0.256641s (14100 normal rows, 0 exceptions)
[2021-03-31 18:28:52.864] [driver] [info] [Task Finished] Transform to mem in 0.256848s (14217 normal rows, 0 exceptions)
[2021-03-31 18:28:53.121] [driver] [info] [Task Finished] Transform to mem in 0.256199s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:53.377] [driver] [info] [Task Finished] Transform to mem in 0.256523s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:53.634] [driver] [info] [Task Finished] Transform to mem in 0.256740s (14223 normal rows, 0 exceptions)
[2021-03-31 18:28:53.890] [driver] [info] [Task Finished] Transform to mem in 0.255805s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:54.146] [driver] [info] [Task Finished] Transform to mem in 0.256729s (14086 normal rows, 0 exceptions)
[2021-03-31 18:28:54.406] [driver] [info] [Task Finished] Transform to mem in 0.259597s (14232 normal rows, 0 exceptions)
[2021-03-31 18:28:54.666] [driver] [info] [Task Finished] Transform to mem in 0.259880s (14170 normal rows, 0 exceptions)
[2021-03-31 18:28:54.922] [driver] [info] [Task Finished] Transform to mem in 0.256300s (14002 normal rows, 0 exceptions)
[2021-03-31 18:28:55.179] [driver] [info] [Task Finished] Transform to mem in 0.256640s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:55.435] [driver] [info] [Task Finished] Transform to mem in 0.256417s (14206 normal rows, 0 exceptions)
[2021-03-31 18:28:55.692] [driver] [info] [Task Finished] Transform to mem in 0.256157s (13961 normal rows, 0 exceptions)
[2021-03-31 18:28:55.948] [driver] [info] [Task Finished] Transform to mem in 0.256653s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:56.205] [driver] [info] [Task Finished] Transform to mem in 0.256551s (14233 normal rows, 0 exceptions)
[2021-03-31 18:28:56.504] [driver] [info] [Task Finished] Transform to mem in 0.299523s (16320 normal rows, 0 exceptions)
[2021-03-31 18:28:56.504] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 41.1797s
[2021-03-31 18:28:56.504] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.177s, 54108450 input rows, time to process 1 row via fast path: 0.000761008ms

[2021-03-31 18:28:56.542] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:28:56.946] [driver] [info] [Task Finished] write to file in 0.404343s (2264570 rows, 344.74 MB)
[2021-03-31 18:28:56.946] [global] [info] writing output took 0.441695s
[2021-03-31 18:28:56.946] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.441779s
[2021-03-31 18:28:56.946] [global] [info] [Transform Stage] Stage 0 took 41.8265s
[2021-03-31 18:28:56.946] [global] [info] Query Execution took 41.8834s. (planning: 0.0568158s, execution: 41.8266s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.004995584487915039
Tuplex job time: 42.150646686553955 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0195681,"llvm_optimization_time_s":0.136185,"llvm_compilation_time_s":0.0613808,"total_compilation_time_s":0.205033,"sampling_time_s":0.112983,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.177,"fast_path_time_s":41.1797,"fast_path_per_row_time_ns":761.008,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004995584487915039, "jobTime": 42.150646686553955}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
