[2021-03-31 18:22:36.252] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:22:36.252] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:22:36.252] [local ee] [info] initializing LLVM backend
[2021-03-31 18:22:36.252] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:22:36.252] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:22:36.253] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:22:36.253] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:22:36.253] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:22:36.254] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:22:36.390] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.406] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.413] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.440] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.459] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.472] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.503] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:22:36.519] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:22:36.521] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:22:36.524] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:22:36.526] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:22:36.541] [logical planner] [info] logical optimization took 0.019594ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:22:36.541] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:22:36.542] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:22:36.543] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:22:36.543] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:22:36.544] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:22:36.544] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:22:36.545] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:22:36.546] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:22:36.547] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:22:36.548] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:22:36.548] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:22:36.550] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:22:36.550] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:22:36.551] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:22:36.576] [global] [info] Stage0 depends on: 
[2021-03-31 18:22:36.720] [global] [info] Optimization via LLVM passes took 0.136419 ms
[2021-03-31 18:22:36.782] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:22:36.782] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.205578s
[2021-03-31 18:22:36.782] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:22:37.038] [driver] [info] [Task Finished] Transform to mem in 0.255768s (14226 normal rows, 0 exceptions)
[2021-03-31 18:22:37.291] [driver] [info] [Task Finished] Transform to mem in 0.253278s (14206 normal rows, 0 exceptions)
[2021-03-31 18:22:37.545] [driver] [info] [Task Finished] Transform to mem in 0.254045s (13970 normal rows, 0 exceptions)
[2021-03-31 18:22:37.799] [driver] [info] [Task Finished] Transform to mem in 0.253619s (14253 normal rows, 0 exceptions)
[2021-03-31 18:22:38.052] [driver] [info] [Task Finished] Transform to mem in 0.253480s (14217 normal rows, 0 exceptions)
[2021-03-31 18:22:38.305] [driver] [info] [Task Finished] Transform to mem in 0.252973s (13920 normal rows, 0 exceptions)
[2021-03-31 18:22:38.559] [driver] [info] [Task Finished] Transform to mem in 0.253454s (14267 normal rows, 0 exceptions)
[2021-03-31 18:22:38.815] [driver] [info] [Task Finished] Transform to mem in 0.256001s (14248 normal rows, 0 exceptions)
[2021-03-31 18:22:39.068] [driver] [info] [Task Finished] Transform to mem in 0.252971s (13953 normal rows, 0 exceptions)
[2021-03-31 18:22:39.321] [driver] [info] [Task Finished] Transform to mem in 0.253595s (14207 normal rows, 0 exceptions)
[2021-03-31 18:22:39.574] [driver] [info] [Task Finished] Transform to mem in 0.253139s (14237 normal rows, 0 exceptions)
[2021-03-31 18:22:39.827] [driver] [info] [Task Finished] Transform to mem in 0.252811s (13988 normal rows, 0 exceptions)
[2021-03-31 18:22:40.081] [driver] [info] [Task Finished] Transform to mem in 0.253410s (14177 normal rows, 0 exceptions)
[2021-03-31 18:22:40.334] [driver] [info] [Task Finished] Transform to mem in 0.253389s (14242 normal rows, 0 exceptions)
[2021-03-31 18:22:40.587] [driver] [info] [Task Finished] Transform to mem in 0.252785s (14020 normal rows, 0 exceptions)
[2021-03-31 18:22:40.840] [driver] [info] [Task Finished] Transform to mem in 0.253456s (14162 normal rows, 0 exceptions)
[2021-03-31 18:22:41.094] [driver] [info] [Task Finished] Transform to mem in 0.253297s (14241 normal rows, 0 exceptions)
[2021-03-31 18:22:41.348] [driver] [info] [Task Finished] Transform to mem in 0.253825s (14059 normal rows, 0 exceptions)
[2021-03-31 18:22:41.601] [driver] [info] [Task Finished] Transform to mem in 0.253350s (14114 normal rows, 0 exceptions)
[2021-03-31 18:22:41.854] [driver] [info] [Task Finished] Transform to mem in 0.252970s (14238 normal rows, 0 exceptions)
[2021-03-31 18:22:42.107] [driver] [info] [Task Finished] Transform to mem in 0.252787s (14081 normal rows, 0 exceptions)
[2021-03-31 18:22:42.363] [driver] [info] [Task Finished] Transform to mem in 0.256375s (14095 normal rows, 0 exceptions)
[2021-03-31 18:22:42.619] [driver] [info] [Task Finished] Transform to mem in 0.256275s (14221 normal rows, 0 exceptions)
[2021-03-31 18:22:42.872] [driver] [info] [Task Finished] Transform to mem in 0.252924s (14086 normal rows, 0 exceptions)
[2021-03-31 18:22:43.125] [driver] [info] [Task Finished] Transform to mem in 0.253008s (14097 normal rows, 0 exceptions)
[2021-03-31 18:22:43.378] [driver] [info] [Task Finished] Transform to mem in 0.253017s (14212 normal rows, 0 exceptions)
[2021-03-31 18:22:43.631] [driver] [info] [Task Finished] Transform to mem in 0.253101s (14108 normal rows, 0 exceptions)
[2021-03-31 18:22:43.884] [driver] [info] [Task Finished] Transform to mem in 0.252882s (14100 normal rows, 0 exceptions)
[2021-03-31 18:22:44.138] [driver] [info] [Task Finished] Transform to mem in 0.253206s (14212 normal rows, 0 exceptions)
[2021-03-31 18:22:44.391] [driver] [info] [Task Finished] Transform to mem in 0.253139s (14099 normal rows, 0 exceptions)
[2021-03-31 18:22:44.644] [driver] [info] [Task Finished] Transform to mem in 0.252923s (14094 normal rows, 0 exceptions)
[2021-03-31 18:22:44.897] [driver] [info] [Task Finished] Transform to mem in 0.253268s (14223 normal rows, 0 exceptions)
[2021-03-31 18:22:45.150] [driver] [info] [Task Finished] Transform to mem in 0.252750s (14089 normal rows, 0 exceptions)
[2021-03-31 18:22:45.403] [driver] [info] [Task Finished] Transform to mem in 0.253017s (14083 normal rows, 0 exceptions)
[2021-03-31 18:22:45.656] [driver] [info] [Task Finished] Transform to mem in 0.253338s (14235 normal rows, 0 exceptions)
[2021-03-31 18:22:45.912] [driver] [info] [Task Finished] Transform to mem in 0.256140s (14170 normal rows, 0 exceptions)
[2021-03-31 18:22:46.167] [driver] [info] [Task Finished] Transform to mem in 0.255190s (14002 normal rows, 0 exceptions)
[2021-03-31 18:22:46.422] [driver] [info] [Task Finished] Transform to mem in 0.254519s (14226 normal rows, 0 exceptions)
[2021-03-31 18:22:46.675] [driver] [info] [Task Finished] Transform to mem in 0.252798s (14206 normal rows, 0 exceptions)
[2021-03-31 18:22:46.928] [driver] [info] [Task Finished] Transform to mem in 0.252770s (13961 normal rows, 0 exceptions)
[2021-03-31 18:22:47.181] [driver] [info] [Task Finished] Transform to mem in 0.253549s (14246 normal rows, 0 exceptions)
[2021-03-31 18:22:47.436] [driver] [info] [Task Finished] Transform to mem in 0.254632s (14233 normal rows, 0 exceptions)
[2021-03-31 18:22:47.688] [driver] [info] [Task Finished] Transform to mem in 0.252602s (13915 normal rows, 0 exceptions)
[2021-03-31 18:22:47.942] [driver] [info] [Task Finished] Transform to mem in 0.253608s (14253 normal rows, 0 exceptions)
[2021-03-31 18:22:48.195] [driver] [info] [Task Finished] Transform to mem in 0.253275s (14267 normal rows, 0 exceptions)
[2021-03-31 18:22:48.448] [driver] [info] [Task Finished] Transform to mem in 0.252514s (13892 normal rows, 0 exceptions)
[2021-03-31 18:22:48.702] [driver] [info] [Task Finished] Transform to mem in 0.253674s (14245 normal rows, 0 exceptions)
[2021-03-31 18:22:48.955] [driver] [info] [Task Finished] Transform to mem in 0.253129s (14260 normal rows, 0 exceptions)
[2021-03-31 18:22:49.207] [driver] [info] [Task Finished] Transform to mem in 0.252212s (13978 normal rows, 0 exceptions)
[2021-03-31 18:22:49.461] [driver] [info] [Task Finished] Transform to mem in 0.253601s (14181 normal rows, 0 exceptions)
[2021-03-31 18:22:49.717] [driver] [info] [Task Finished] Transform to mem in 0.256261s (14234 normal rows, 0 exceptions)
[2021-03-31 18:22:49.973] [driver] [info] [Task Finished] Transform to mem in 0.256048s (14015 normal rows, 0 exceptions)
[2021-03-31 18:22:50.226] [driver] [info] [Task Finished] Transform to mem in 0.253175s (14142 normal rows, 0 exceptions)
[2021-03-31 18:22:50.479] [driver] [info] [Task Finished] Transform to mem in 0.253291s (14280 normal rows, 0 exceptions)
[2021-03-31 18:22:50.736] [driver] [info] [Task Finished] Transform to mem in 0.256114s (13982 normal rows, 0 exceptions)
[2021-03-31 18:22:50.989] [driver] [info] [Task Finished] Transform to mem in 0.253347s (14173 normal rows, 0 exceptions)
[2021-03-31 18:22:51.242] [driver] [info] [Task Finished] Transform to mem in 0.253189s (14235 normal rows, 0 exceptions)
[2021-03-31 18:22:51.495] [driver] [info] [Task Finished] Transform to mem in 0.252856s (14054 normal rows, 0 exceptions)
[2021-03-31 18:22:51.748] [driver] [info] [Task Finished] Transform to mem in 0.253268s (14129 normal rows, 0 exceptions)
[2021-03-31 18:22:52.002] [driver] [info] [Task Finished] Transform to mem in 0.253318s (14223 normal rows, 0 exceptions)
[2021-03-31 18:22:52.255] [driver] [info] [Task Finished] Transform to mem in 0.252939s (14081 normal rows, 0 exceptions)
[2021-03-31 18:22:52.508] [driver] [info] [Task Finished] Transform to mem in 0.253052s (14108 normal rows, 0 exceptions)
[2021-03-31 18:22:52.761] [driver] [info] [Task Finished] Transform to mem in 0.253269s (14208 normal rows, 0 exceptions)
[2021-03-31 18:22:53.014] [driver] [info] [Task Finished] Transform to mem in 0.252812s (14089 normal rows, 0 exceptions)
[2021-03-31 18:22:53.270] [driver] [info] [Task Finished] Transform to mem in 0.256047s (14094 normal rows, 0 exceptions)
[2021-03-31 18:22:53.525] [driver] [info] [Task Finished] Transform to mem in 0.255084s (14232 normal rows, 0 exceptions)
[2021-03-31 18:22:53.779] [driver] [info] [Task Finished] Transform to mem in 0.254032s (14088 normal rows, 0 exceptions)
[2021-03-31 18:22:54.032] [driver] [info] [Task Finished] Transform to mem in 0.253252s (14106 normal rows, 0 exceptions)
[2021-03-31 18:22:54.287] [driver] [info] [Task Finished] Transform to mem in 0.254752s (14211 normal rows, 0 exceptions)
[2021-03-31 18:22:54.542] [driver] [info] [Task Finished] Transform to mem in 0.254798s (14113 normal rows, 0 exceptions)
[2021-03-31 18:22:54.795] [driver] [info] [Task Finished] Transform to mem in 0.252922s (14075 normal rows, 0 exceptions)
[2021-03-31 18:22:55.048] [driver] [info] [Task Finished] Transform to mem in 0.253402s (14223 normal rows, 0 exceptions)
[2021-03-31 18:22:55.301] [driver] [info] [Task Finished] Transform to mem in 0.252913s (14095 normal rows, 0 exceptions)
[2021-03-31 18:22:55.554] [driver] [info] [Task Finished] Transform to mem in 0.252803s (14086 normal rows, 0 exceptions)
[2021-03-31 18:22:55.807] [driver] [info] [Task Finished] Transform to mem in 0.253334s (14226 normal rows, 0 exceptions)
[2021-03-31 18:22:56.060] [driver] [info] [Task Finished] Transform to mem in 0.252907s (14170 normal rows, 0 exceptions)
[2021-03-31 18:22:56.313] [driver] [info] [Task Finished] Transform to mem in 0.252783s (14002 normal rows, 0 exceptions)
[2021-03-31 18:22:56.566] [driver] [info] [Task Finished] Transform to mem in 0.253107s (14226 normal rows, 0 exceptions)
[2021-03-31 18:22:56.819] [driver] [info] [Task Finished] Transform to mem in 0.252920s (14206 normal rows, 0 exceptions)
[2021-03-31 18:22:57.075] [driver] [info] [Task Finished] Transform to mem in 0.255708s (13970 normal rows, 0 exceptions)
[2021-03-31 18:22:57.332] [driver] [info] [Task Finished] Transform to mem in 0.256787s (14237 normal rows, 0 exceptions)
[2021-03-31 18:22:57.584] [driver] [info] [Task Finished] Transform to mem in 0.252919s (14233 normal rows, 0 exceptions)
[2021-03-31 18:22:57.837] [driver] [info] [Task Finished] Transform to mem in 0.252369s (13920 normal rows, 0 exceptions)
[2021-03-31 18:22:58.094] [driver] [info] [Task Finished] Transform to mem in 0.257138s (14267 normal rows, 0 exceptions)
[2021-03-31 18:22:58.347] [driver] [info] [Task Finished] Transform to mem in 0.253275s (14248 normal rows, 0 exceptions)
[2021-03-31 18:22:58.600] [driver] [info] [Task Finished] Transform to mem in 0.252432s (13912 normal rows, 0 exceptions)
[2021-03-31 18:22:58.853] [driver] [info] [Task Finished] Transform to mem in 0.253717s (14248 normal rows, 0 exceptions)
[2021-03-31 18:22:59.107] [driver] [info] [Task Finished] Transform to mem in 0.253127s (14237 normal rows, 0 exceptions)
[2021-03-31 18:22:59.359] [driver] [info] [Task Finished] Transform to mem in 0.252373s (13988 normal rows, 0 exceptions)
[2021-03-31 18:22:59.613] [driver] [info] [Task Finished] Transform to mem in 0.253570s (14171 normal rows, 0 exceptions)
[2021-03-31 18:22:59.866] [driver] [info] [Task Finished] Transform to mem in 0.253026s (14234 normal rows, 0 exceptions)
[2021-03-31 18:23:00.118] [driver] [info] [Task Finished] Transform to mem in 0.252584s (14015 normal rows, 0 exceptions)
[2021-03-31 18:23:00.372] [driver] [info] [Task Finished] Transform to mem in 0.253387s (14176 normal rows, 0 exceptions)
[2021-03-31 18:23:00.628] [driver] [info] [Task Finished] Transform to mem in 0.256278s (14246 normal rows, 0 exceptions)
[2021-03-31 18:23:00.882] [driver] [info] [Task Finished] Transform to mem in 0.254513s (14059 normal rows, 0 exceptions)
[2021-03-31 18:23:01.137] [driver] [info] [Task Finished] Transform to mem in 0.254763s (14114 normal rows, 0 exceptions)
[2021-03-31 18:23:01.390] [driver] [info] [Task Finished] Transform to mem in 0.252929s (14238 normal rows, 0 exceptions)
[2021-03-31 18:23:01.645] [driver] [info] [Task Finished] Transform to mem in 0.254387s (14081 normal rows, 0 exceptions)
[2021-03-31 18:23:01.900] [driver] [info] [Task Finished] Transform to mem in 0.255254s (14095 normal rows, 0 exceptions)
[2021-03-31 18:23:02.153] [driver] [info] [Task Finished] Transform to mem in 0.253551s (14221 normal rows, 0 exceptions)
[2021-03-31 18:23:02.406] [driver] [info] [Task Finished] Transform to mem in 0.252880s (14086 normal rows, 0 exceptions)
[2021-03-31 18:23:02.659] [driver] [info] [Task Finished] Transform to mem in 0.253077s (14091 normal rows, 0 exceptions)
[2021-03-31 18:23:02.913] [driver] [info] [Task Finished] Transform to mem in 0.254059s (14218 normal rows, 0 exceptions)
[2021-03-31 18:23:03.166] [driver] [info] [Task Finished] Transform to mem in 0.252838s (14095 normal rows, 0 exceptions)
[2021-03-31 18:23:03.419] [driver] [info] [Task Finished] Transform to mem in 0.253154s (14113 normal rows, 0 exceptions)
[2021-03-31 18:23:03.673] [driver] [info] [Task Finished] Transform to mem in 0.253363s (14197 normal rows, 0 exceptions)
[2021-03-31 18:23:03.926] [driver] [info] [Task Finished] Transform to mem in 0.252705s (14114 normal rows, 0 exceptions)
[2021-03-31 18:23:04.178] [driver] [info] [Task Finished] Transform to mem in 0.252899s (14094 normal rows, 0 exceptions)
[2021-03-31 18:23:04.435] [driver] [info] [Task Finished] Transform to mem in 0.256436s (14197 normal rows, 0 exceptions)
[2021-03-31 18:23:04.691] [driver] [info] [Task Finished] Transform to mem in 0.256151s (14115 normal rows, 0 exceptions)
[2021-03-31 18:23:04.944] [driver] [info] [Task Finished] Transform to mem in 0.252908s (14083 normal rows, 0 exceptions)
[2021-03-31 18:23:05.197] [driver] [info] [Task Finished] Transform to mem in 0.253332s (14235 normal rows, 0 exceptions)
[2021-03-31 18:23:05.454] [driver] [info] [Task Finished] Transform to mem in 0.256394s (14144 normal rows, 0 exceptions)
[2021-03-31 18:23:05.707] [driver] [info] [Task Finished] Transform to mem in 0.253002s (14015 normal rows, 0 exceptions)
[2021-03-31 18:23:05.960] [driver] [info] [Task Finished] Transform to mem in 0.253552s (14237 normal rows, 0 exceptions)
[2021-03-31 18:23:06.214] [driver] [info] [Task Finished] Transform to mem in 0.253164s (14159 normal rows, 0 exceptions)
[2021-03-31 18:23:06.466] [driver] [info] [Task Finished] Transform to mem in 0.252878s (14010 normal rows, 0 exceptions)
[2021-03-31 18:23:06.720] [driver] [info] [Task Finished] Transform to mem in 0.253616s (14246 normal rows, 0 exceptions)
[2021-03-31 18:23:06.973] [driver] [info] [Task Finished] Transform to mem in 0.253124s (14178 normal rows, 0 exceptions)
[2021-03-31 18:23:07.226] [driver] [info] [Task Finished] Transform to mem in 0.252744s (13970 normal rows, 0 exceptions)
[2021-03-31 18:23:07.479] [driver] [info] [Task Finished] Transform to mem in 0.253484s (14253 normal rows, 0 exceptions)
[2021-03-31 18:23:07.733] [driver] [info] [Task Finished] Transform to mem in 0.253127s (14236 normal rows, 0 exceptions)
[2021-03-31 18:23:07.988] [driver] [info] [Task Finished] Transform to mem in 0.255418s (13901 normal rows, 0 exceptions)
[2021-03-31 18:23:08.243] [driver] [info] [Task Finished] Transform to mem in 0.255388s (14267 normal rows, 0 exceptions)
[2021-03-31 18:23:08.498] [driver] [info] [Task Finished] Transform to mem in 0.254380s (14248 normal rows, 0 exceptions)
[2021-03-31 18:23:08.750] [driver] [info] [Task Finished] Transform to mem in 0.252397s (13953 normal rows, 0 exceptions)
[2021-03-31 18:23:09.004] [driver] [info] [Task Finished] Transform to mem in 0.254131s (14207 normal rows, 0 exceptions)
[2021-03-31 18:23:09.258] [driver] [info] [Task Finished] Transform to mem in 0.253164s (14245 normal rows, 0 exceptions)
[2021-03-31 18:23:09.510] [driver] [info] [Task Finished] Transform to mem in 0.252692s (13999 normal rows, 0 exceptions)
[2021-03-31 18:23:09.764] [driver] [info] [Task Finished] Transform to mem in 0.253457s (14158 normal rows, 0 exceptions)
[2021-03-31 18:23:10.017] [driver] [info] [Task Finished] Transform to mem in 0.253326s (14242 normal rows, 0 exceptions)
[2021-03-31 18:23:10.270] [driver] [info] [Task Finished] Transform to mem in 0.252877s (14020 normal rows, 0 exceptions)
[2021-03-31 18:23:10.523] [driver] [info] [Task Finished] Transform to mem in 0.253289s (14173 normal rows, 0 exceptions)
[2021-03-31 18:23:10.777] [driver] [info] [Task Finished] Transform to mem in 0.253240s (14235 normal rows, 0 exceptions)
[2021-03-31 18:23:11.029] [driver] [info] [Task Finished] Transform to mem in 0.252850s (14054 normal rows, 0 exceptions)
[2021-03-31 18:23:11.283] [driver] [info] [Task Finished] Transform to mem in 0.253223s (14114 normal rows, 0 exceptions)
[2021-03-31 18:23:11.536] [driver] [info] [Task Finished] Transform to mem in 0.253097s (14238 normal rows, 0 exceptions)
[2021-03-31 18:23:11.792] [driver] [info] [Task Finished] Transform to mem in 0.256114s (14081 normal rows, 0 exceptions)
[2021-03-31 18:23:12.048] [driver] [info] [Task Finished] Transform to mem in 0.256085s (14095 normal rows, 0 exceptions)
[2021-03-31 18:23:12.301] [driver] [info] [Task Finished] Transform to mem in 0.253346s (14221 normal rows, 0 exceptions)
[2021-03-31 18:23:12.555] [driver] [info] [Task Finished] Transform to mem in 0.253370s (14089 normal rows, 0 exceptions)
[2021-03-31 18:23:12.808] [driver] [info] [Task Finished] Transform to mem in 0.253143s (14094 normal rows, 0 exceptions)
[2021-03-31 18:23:13.061] [driver] [info] [Task Finished] Transform to mem in 0.253355s (14212 normal rows, 0 exceptions)
[2021-03-31 18:23:13.314] [driver] [info] [Task Finished] Transform to mem in 0.252788s (14108 normal rows, 0 exceptions)
[2021-03-31 18:23:13.567] [driver] [info] [Task Finished] Transform to mem in 0.252947s (14100 normal rows, 0 exceptions)
[2021-03-31 18:23:13.821] [driver] [info] [Task Finished] Transform to mem in 0.253518s (14217 normal rows, 0 exceptions)
[2021-03-31 18:23:14.073] [driver] [info] [Task Finished] Transform to mem in 0.252883s (14094 normal rows, 0 exceptions)
[2021-03-31 18:23:14.327] [driver] [info] [Task Finished] Transform to mem in 0.253281s (14094 normal rows, 0 exceptions)
[2021-03-31 18:23:14.580] [driver] [info] [Task Finished] Transform to mem in 0.253292s (14223 normal rows, 0 exceptions)
[2021-03-31 18:23:14.833] [driver] [info] [Task Finished] Transform to mem in 0.252871s (14089 normal rows, 0 exceptions)
[2021-03-31 18:23:15.086] [driver] [info] [Task Finished] Transform to mem in 0.252688s (14086 normal rows, 0 exceptions)
[2021-03-31 18:23:15.342] [driver] [info] [Task Finished] Transform to mem in 0.256655s (14232 normal rows, 0 exceptions)
[2021-03-31 18:23:15.597] [driver] [info] [Task Finished] Transform to mem in 0.254202s (14170 normal rows, 0 exceptions)
[2021-03-31 18:23:15.851] [driver] [info] [Task Finished] Transform to mem in 0.254678s (14002 normal rows, 0 exceptions)
[2021-03-31 18:23:16.105] [driver] [info] [Task Finished] Transform to mem in 0.253432s (14226 normal rows, 0 exceptions)
[2021-03-31 18:23:16.358] [driver] [info] [Task Finished] Transform to mem in 0.252991s (14206 normal rows, 0 exceptions)
[2021-03-31 18:23:16.610] [driver] [info] [Task Finished] Transform to mem in 0.252597s (13961 normal rows, 0 exceptions)
[2021-03-31 18:23:16.864] [driver] [info] [Task Finished] Transform to mem in 0.253549s (14246 normal rows, 0 exceptions)
[2021-03-31 18:23:17.117] [driver] [info] [Task Finished] Transform to mem in 0.252986s (14233 normal rows, 0 exceptions)
[2021-03-31 18:23:17.412] [driver] [info] [Task Finished] Transform to mem in 0.295299s (16320 normal rows, 0 exceptions)
[2021-03-31 18:23:17.412] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 40.6305s
[2021-03-31 18:23:17.412] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.6278s, 54108450 input rows, time to process 1 row via fast path: 0.000750859ms

[2021-03-31 18:23:17.449] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:23:17.876] [driver] [info] [Task Finished] write to file in 0.426064s (2264570 rows, 344.74 MB)
[2021-03-31 18:23:17.876] [global] [info] writing output took 0.463266s
[2021-03-31 18:23:17.876] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.463349s
[2021-03-31 18:23:17.876] [global] [info] [Transform Stage] Stage 0 took 41.2995s
[2021-03-31 18:23:17.876] [global] [info] Query Execution took 41.3563s. (planning: 0.0568314s, execution: 41.2995s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.0050809383392333984
Tuplex job time: 41.623549938201904 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0195936,"llvm_optimization_time_s":0.136419,"llvm_compilation_time_s":0.0616047,"total_compilation_time_s":0.205576,"sampling_time_s":0.113118,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.6278,"fast_path_time_s":40.6305,"fast_path_per_row_time_ns":750.859,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.0050809383392333984, "jobTime": 41.623549938201904}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
