[2021-03-31 18:27:32.883] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:27:32.884] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:27:32.884] [local ee] [info] initializing LLVM backend
[2021-03-31 18:27:32.884] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:27:32.884] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:27:32.884] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:27:32.885] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:27:32.885] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:27:32.885] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:27:33.017] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.032] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.040] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.066] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.086] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.099] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.129] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:27:33.145] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:27:33.148] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:27:33.150] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:27:33.153] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:27:33.168] [logical planner] [info] logical optimization took 0.019558ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:27:33.168] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:27:33.168] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:27:33.169] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:27:33.170] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:27:33.170] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:27:33.171] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:27:33.172] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:27:33.172] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:27:33.173] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:27:33.174] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:27:33.175] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:27:33.176] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:27:33.177] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:27:33.177] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:27:33.202] [global] [info] Stage0 depends on: 
[2021-03-31 18:27:33.346] [global] [info] Optimization via LLVM passes took 0.136421 ms
[2021-03-31 18:27:33.408] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:27:33.408] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.205407s
[2021-03-31 18:27:33.408] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:27:33.667] [driver] [info] [Task Finished] Transform to mem in 0.258459s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:33.920] [driver] [info] [Task Finished] Transform to mem in 0.252990s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:34.173] [driver] [info] [Task Finished] Transform to mem in 0.253026s (13970 normal rows, 0 exceptions)
[2021-03-31 18:27:34.426] [driver] [info] [Task Finished] Transform to mem in 0.253268s (14253 normal rows, 0 exceptions)
[2021-03-31 18:27:34.679] [driver] [info] [Task Finished] Transform to mem in 0.252715s (14217 normal rows, 0 exceptions)
[2021-03-31 18:27:34.932] [driver] [info] [Task Finished] Transform to mem in 0.253747s (13920 normal rows, 0 exceptions)
[2021-03-31 18:27:35.186] [driver] [info] [Task Finished] Transform to mem in 0.253159s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:35.439] [driver] [info] [Task Finished] Transform to mem in 0.253048s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:35.691] [driver] [info] [Task Finished] Transform to mem in 0.252430s (13953 normal rows, 0 exceptions)
[2021-03-31 18:27:35.944] [driver] [info] [Task Finished] Transform to mem in 0.253098s (14207 normal rows, 0 exceptions)
[2021-03-31 18:27:36.197] [driver] [info] [Task Finished] Transform to mem in 0.252489s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:36.449] [driver] [info] [Task Finished] Transform to mem in 0.252146s (13988 normal rows, 0 exceptions)
[2021-03-31 18:27:36.702] [driver] [info] [Task Finished] Transform to mem in 0.253285s (14177 normal rows, 0 exceptions)
[2021-03-31 18:27:36.958] [driver] [info] [Task Finished] Transform to mem in 0.255965s (14242 normal rows, 0 exceptions)
[2021-03-31 18:27:37.213] [driver] [info] [Task Finished] Transform to mem in 0.254406s (14020 normal rows, 0 exceptions)
[2021-03-31 18:27:37.467] [driver] [info] [Task Finished] Transform to mem in 0.254581s (14162 normal rows, 0 exceptions)
[2021-03-31 18:27:37.720] [driver] [info] [Task Finished] Transform to mem in 0.252386s (14241 normal rows, 0 exceptions)
[2021-03-31 18:27:37.972] [driver] [info] [Task Finished] Transform to mem in 0.252922s (14059 normal rows, 0 exceptions)
[2021-03-31 18:27:38.225] [driver] [info] [Task Finished] Transform to mem in 0.252906s (14114 normal rows, 0 exceptions)
[2021-03-31 18:27:38.478] [driver] [info] [Task Finished] Transform to mem in 0.252767s (14238 normal rows, 0 exceptions)
[2021-03-31 18:27:38.731] [driver] [info] [Task Finished] Transform to mem in 0.252888s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:38.984] [driver] [info] [Task Finished] Transform to mem in 0.252562s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:39.236] [driver] [info] [Task Finished] Transform to mem in 0.252679s (14221 normal rows, 0 exceptions)
[2021-03-31 18:27:39.489] [driver] [info] [Task Finished] Transform to mem in 0.252428s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:39.742] [driver] [info] [Task Finished] Transform to mem in 0.253034s (14097 normal rows, 0 exceptions)
[2021-03-31 18:27:39.994] [driver] [info] [Task Finished] Transform to mem in 0.252444s (14212 normal rows, 0 exceptions)
[2021-03-31 18:27:40.247] [driver] [info] [Task Finished] Transform to mem in 0.252250s (14108 normal rows, 0 exceptions)
[2021-03-31 18:27:40.502] [driver] [info] [Task Finished] Transform to mem in 0.255755s (14100 normal rows, 0 exceptions)
[2021-03-31 18:27:40.755] [driver] [info] [Task Finished] Transform to mem in 0.252378s (14212 normal rows, 0 exceptions)
[2021-03-31 18:27:41.011] [driver] [info] [Task Finished] Transform to mem in 0.255870s (14099 normal rows, 0 exceptions)
[2021-03-31 18:27:41.263] [driver] [info] [Task Finished] Transform to mem in 0.252739s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:41.516] [driver] [info] [Task Finished] Transform to mem in 0.252808s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:41.769] [driver] [info] [Task Finished] Transform to mem in 0.252384s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:42.021] [driver] [info] [Task Finished] Transform to mem in 0.252729s (14083 normal rows, 0 exceptions)
[2021-03-31 18:27:42.274] [driver] [info] [Task Finished] Transform to mem in 0.253026s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:42.527] [driver] [info] [Task Finished] Transform to mem in 0.252801s (14170 normal rows, 0 exceptions)
[2021-03-31 18:27:42.780] [driver] [info] [Task Finished] Transform to mem in 0.252625s (14002 normal rows, 0 exceptions)
[2021-03-31 18:27:43.033] [driver] [info] [Task Finished] Transform to mem in 0.253523s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:43.286] [driver] [info] [Task Finished] Transform to mem in 0.252766s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:43.539] [driver] [info] [Task Finished] Transform to mem in 0.252453s (13961 normal rows, 0 exceptions)
[2021-03-31 18:27:43.792] [driver] [info] [Task Finished] Transform to mem in 0.253384s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:44.045] [driver] [info] [Task Finished] Transform to mem in 0.252571s (14233 normal rows, 0 exceptions)
[2021-03-31 18:27:44.300] [driver] [info] [Task Finished] Transform to mem in 0.255436s (13915 normal rows, 0 exceptions)
[2021-03-31 18:27:44.553] [driver] [info] [Task Finished] Transform to mem in 0.253248s (14253 normal rows, 0 exceptions)
[2021-03-31 18:27:44.809] [driver] [info] [Task Finished] Transform to mem in 0.256033s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:45.062] [driver] [info] [Task Finished] Transform to mem in 0.252107s (13892 normal rows, 0 exceptions)
[2021-03-31 18:27:45.314] [driver] [info] [Task Finished] Transform to mem in 0.252945s (14245 normal rows, 0 exceptions)
[2021-03-31 18:27:45.568] [driver] [info] [Task Finished] Transform to mem in 0.253083s (14260 normal rows, 0 exceptions)
[2021-03-31 18:27:45.820] [driver] [info] [Task Finished] Transform to mem in 0.251996s (13978 normal rows, 0 exceptions)
[2021-03-31 18:27:46.073] [driver] [info] [Task Finished] Transform to mem in 0.252947s (14181 normal rows, 0 exceptions)
[2021-03-31 18:27:46.326] [driver] [info] [Task Finished] Transform to mem in 0.252983s (14234 normal rows, 0 exceptions)
[2021-03-31 18:27:46.578] [driver] [info] [Task Finished] Transform to mem in 0.252470s (14015 normal rows, 0 exceptions)
[2021-03-31 18:27:46.831] [driver] [info] [Task Finished] Transform to mem in 0.253423s (14142 normal rows, 0 exceptions)
[2021-03-31 18:27:47.085] [driver] [info] [Task Finished] Transform to mem in 0.253149s (14280 normal rows, 0 exceptions)
[2021-03-31 18:27:47.337] [driver] [info] [Task Finished] Transform to mem in 0.252093s (13982 normal rows, 0 exceptions)
[2021-03-31 18:27:47.589] [driver] [info] [Task Finished] Transform to mem in 0.252589s (14173 normal rows, 0 exceptions)
[2021-03-31 18:27:47.845] [driver] [info] [Task Finished] Transform to mem in 0.255909s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:48.098] [driver] [info] [Task Finished] Transform to mem in 0.252507s (14054 normal rows, 0 exceptions)
[2021-03-31 18:27:48.354] [driver] [info] [Task Finished] Transform to mem in 0.256314s (14129 normal rows, 0 exceptions)
[2021-03-31 18:27:48.607] [driver] [info] [Task Finished] Transform to mem in 0.252489s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:48.859] [driver] [info] [Task Finished] Transform to mem in 0.252758s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:49.113] [driver] [info] [Task Finished] Transform to mem in 0.253124s (14108 normal rows, 0 exceptions)
[2021-03-31 18:27:49.366] [driver] [info] [Task Finished] Transform to mem in 0.253048s (14208 normal rows, 0 exceptions)
[2021-03-31 18:27:49.618] [driver] [info] [Task Finished] Transform to mem in 0.252844s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:49.871] [driver] [info] [Task Finished] Transform to mem in 0.252543s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:50.124] [driver] [info] [Task Finished] Transform to mem in 0.253428s (14232 normal rows, 0 exceptions)
[2021-03-31 18:27:50.377] [driver] [info] [Task Finished] Transform to mem in 0.252732s (14088 normal rows, 0 exceptions)
[2021-03-31 18:27:50.630] [driver] [info] [Task Finished] Transform to mem in 0.252621s (14106 normal rows, 0 exceptions)
[2021-03-31 18:27:50.883] [driver] [info] [Task Finished] Transform to mem in 0.253248s (14211 normal rows, 0 exceptions)
[2021-03-31 18:27:51.136] [driver] [info] [Task Finished] Transform to mem in 0.252483s (14113 normal rows, 0 exceptions)
[2021-03-31 18:27:51.388] [driver] [info] [Task Finished] Transform to mem in 0.252309s (14075 normal rows, 0 exceptions)
[2021-03-31 18:27:51.644] [driver] [info] [Task Finished] Transform to mem in 0.256513s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:51.896] [driver] [info] [Task Finished] Transform to mem in 0.251891s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:52.152] [driver] [info] [Task Finished] Transform to mem in 0.256010s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:52.406] [driver] [info] [Task Finished] Transform to mem in 0.253176s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:52.658] [driver] [info] [Task Finished] Transform to mem in 0.252700s (14170 normal rows, 0 exceptions)
[2021-03-31 18:27:52.910] [driver] [info] [Task Finished] Transform to mem in 0.252190s (14002 normal rows, 0 exceptions)
[2021-03-31 18:27:53.164] [driver] [info] [Task Finished] Transform to mem in 0.253060s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:53.416] [driver] [info] [Task Finished] Transform to mem in 0.252734s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:53.669] [driver] [info] [Task Finished] Transform to mem in 0.252457s (13970 normal rows, 0 exceptions)
[2021-03-31 18:27:53.922] [driver] [info] [Task Finished] Transform to mem in 0.253294s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:54.175] [driver] [info] [Task Finished] Transform to mem in 0.252927s (14233 normal rows, 0 exceptions)
[2021-03-31 18:27:54.427] [driver] [info] [Task Finished] Transform to mem in 0.252312s (13920 normal rows, 0 exceptions)
[2021-03-31 18:27:54.681] [driver] [info] [Task Finished] Transform to mem in 0.253354s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:54.934] [driver] [info] [Task Finished] Transform to mem in 0.252953s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:55.189] [driver] [info] [Task Finished] Transform to mem in 0.255264s (13912 normal rows, 0 exceptions)
[2021-03-31 18:27:55.442] [driver] [info] [Task Finished] Transform to mem in 0.253520s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:55.698] [driver] [info] [Task Finished] Transform to mem in 0.255603s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:55.950] [driver] [info] [Task Finished] Transform to mem in 0.252310s (13988 normal rows, 0 exceptions)
[2021-03-31 18:27:56.204] [driver] [info] [Task Finished] Transform to mem in 0.253269s (14171 normal rows, 0 exceptions)
[2021-03-31 18:27:56.457] [driver] [info] [Task Finished] Transform to mem in 0.252877s (14234 normal rows, 0 exceptions)
[2021-03-31 18:27:56.709] [driver] [info] [Task Finished] Transform to mem in 0.252581s (14015 normal rows, 0 exceptions)
[2021-03-31 18:27:56.962] [driver] [info] [Task Finished] Transform to mem in 0.253062s (14176 normal rows, 0 exceptions)
[2021-03-31 18:27:57.215] [driver] [info] [Task Finished] Transform to mem in 0.253043s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:57.468] [driver] [info] [Task Finished] Transform to mem in 0.252596s (14059 normal rows, 0 exceptions)
[2021-03-31 18:27:57.721] [driver] [info] [Task Finished] Transform to mem in 0.253107s (14114 normal rows, 0 exceptions)
[2021-03-31 18:27:57.974] [driver] [info] [Task Finished] Transform to mem in 0.253137s (14238 normal rows, 0 exceptions)
[2021-03-31 18:27:58.227] [driver] [info] [Task Finished] Transform to mem in 0.252773s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:58.479] [driver] [info] [Task Finished] Transform to mem in 0.252496s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:58.732] [driver] [info] [Task Finished] Transform to mem in 0.252918s (14221 normal rows, 0 exceptions)
[2021-03-31 18:27:58.988] [driver] [info] [Task Finished] Transform to mem in 0.255436s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:59.240] [driver] [info] [Task Finished] Transform to mem in 0.252338s (14091 normal rows, 0 exceptions)
[2021-03-31 18:27:59.496] [driver] [info] [Task Finished] Transform to mem in 0.256117s (14218 normal rows, 0 exceptions)
[2021-03-31 18:27:59.749] [driver] [info] [Task Finished] Transform to mem in 0.252212s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:00.002] [driver] [info] [Task Finished] Transform to mem in 0.253138s (14113 normal rows, 0 exceptions)
[2021-03-31 18:28:00.255] [driver] [info] [Task Finished] Transform to mem in 0.252922s (14197 normal rows, 0 exceptions)
[2021-03-31 18:28:00.508] [driver] [info] [Task Finished] Transform to mem in 0.252844s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:00.761] [driver] [info] [Task Finished] Transform to mem in 0.253024s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:01.014] [driver] [info] [Task Finished] Transform to mem in 0.252950s (14197 normal rows, 0 exceptions)
[2021-03-31 18:28:01.266] [driver] [info] [Task Finished] Transform to mem in 0.252847s (14115 normal rows, 0 exceptions)
[2021-03-31 18:28:01.519] [driver] [info] [Task Finished] Transform to mem in 0.252669s (14083 normal rows, 0 exceptions)
[2021-03-31 18:28:01.772] [driver] [info] [Task Finished] Transform to mem in 0.253276s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:02.025] [driver] [info] [Task Finished] Transform to mem in 0.252568s (14144 normal rows, 0 exceptions)
[2021-03-31 18:28:02.277] [driver] [info] [Task Finished] Transform to mem in 0.252485s (14015 normal rows, 0 exceptions)
[2021-03-31 18:28:02.531] [driver] [info] [Task Finished] Transform to mem in 0.253199s (14237 normal rows, 0 exceptions)
[2021-03-31 18:28:02.783] [driver] [info] [Task Finished] Transform to mem in 0.252080s (14159 normal rows, 0 exceptions)
[2021-03-31 18:28:03.039] [driver] [info] [Task Finished] Transform to mem in 0.255803s (14010 normal rows, 0 exceptions)
[2021-03-31 18:28:03.291] [driver] [info] [Task Finished] Transform to mem in 0.252764s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:03.544] [driver] [info] [Task Finished] Transform to mem in 0.252283s (14178 normal rows, 0 exceptions)
[2021-03-31 18:28:03.796] [driver] [info] [Task Finished] Transform to mem in 0.252427s (13970 normal rows, 0 exceptions)
[2021-03-31 18:28:04.049] [driver] [info] [Task Finished] Transform to mem in 0.253306s (14253 normal rows, 0 exceptions)
[2021-03-31 18:28:04.302] [driver] [info] [Task Finished] Transform to mem in 0.253002s (14236 normal rows, 0 exceptions)
[2021-03-31 18:28:04.554] [driver] [info] [Task Finished] Transform to mem in 0.252052s (13901 normal rows, 0 exceptions)
[2021-03-31 18:28:04.808] [driver] [info] [Task Finished] Transform to mem in 0.253294s (14267 normal rows, 0 exceptions)
[2021-03-31 18:28:05.061] [driver] [info] [Task Finished] Transform to mem in 0.253068s (14248 normal rows, 0 exceptions)
[2021-03-31 18:28:05.313] [driver] [info] [Task Finished] Transform to mem in 0.252372s (13953 normal rows, 0 exceptions)
[2021-03-31 18:28:05.566] [driver] [info] [Task Finished] Transform to mem in 0.252873s (14207 normal rows, 0 exceptions)
[2021-03-31 18:28:05.819] [driver] [info] [Task Finished] Transform to mem in 0.253042s (14245 normal rows, 0 exceptions)
[2021-03-31 18:28:06.072] [driver] [info] [Task Finished] Transform to mem in 0.252667s (13999 normal rows, 0 exceptions)
[2021-03-31 18:28:06.325] [driver] [info] [Task Finished] Transform to mem in 0.252910s (14158 normal rows, 0 exceptions)
[2021-03-31 18:28:06.578] [driver] [info] [Task Finished] Transform to mem in 0.252822s (14242 normal rows, 0 exceptions)
[2021-03-31 18:28:06.834] [driver] [info] [Task Finished] Transform to mem in 0.255853s (14020 normal rows, 0 exceptions)
[2021-03-31 18:28:07.087] [driver] [info] [Task Finished] Transform to mem in 0.253066s (14173 normal rows, 0 exceptions)
[2021-03-31 18:28:07.340] [driver] [info] [Task Finished] Transform to mem in 0.252949s (14235 normal rows, 0 exceptions)
[2021-03-31 18:28:07.592] [driver] [info] [Task Finished] Transform to mem in 0.252849s (14054 normal rows, 0 exceptions)
[2021-03-31 18:28:07.845] [driver] [info] [Task Finished] Transform to mem in 0.252989s (14114 normal rows, 0 exceptions)
[2021-03-31 18:28:08.098] [driver] [info] [Task Finished] Transform to mem in 0.252664s (14238 normal rows, 0 exceptions)
[2021-03-31 18:28:08.351] [driver] [info] [Task Finished] Transform to mem in 0.252846s (14081 normal rows, 0 exceptions)
[2021-03-31 18:28:08.604] [driver] [info] [Task Finished] Transform to mem in 0.252810s (14095 normal rows, 0 exceptions)
[2021-03-31 18:28:08.857] [driver] [info] [Task Finished] Transform to mem in 0.253003s (14221 normal rows, 0 exceptions)
[2021-03-31 18:28:09.110] [driver] [info] [Task Finished] Transform to mem in 0.252765s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:09.363] [driver] [info] [Task Finished] Transform to mem in 0.252891s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:09.615] [driver] [info] [Task Finished] Transform to mem in 0.252461s (14212 normal rows, 0 exceptions)
[2021-03-31 18:28:09.867] [driver] [info] [Task Finished] Transform to mem in 0.252004s (14108 normal rows, 0 exceptions)
[2021-03-31 18:28:10.120] [driver] [info] [Task Finished] Transform to mem in 0.252559s (14100 normal rows, 0 exceptions)
[2021-03-31 18:28:10.376] [driver] [info] [Task Finished] Transform to mem in 0.256630s (14217 normal rows, 0 exceptions)
[2021-03-31 18:28:10.629] [driver] [info] [Task Finished] Transform to mem in 0.252322s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:10.882] [driver] [info] [Task Finished] Transform to mem in 0.253261s (14094 normal rows, 0 exceptions)
[2021-03-31 18:28:11.135] [driver] [info] [Task Finished] Transform to mem in 0.252896s (14223 normal rows, 0 exceptions)
[2021-03-31 18:28:11.387] [driver] [info] [Task Finished] Transform to mem in 0.252568s (14089 normal rows, 0 exceptions)
[2021-03-31 18:28:11.640] [driver] [info] [Task Finished] Transform to mem in 0.252613s (14086 normal rows, 0 exceptions)
[2021-03-31 18:28:11.893] [driver] [info] [Task Finished] Transform to mem in 0.253073s (14232 normal rows, 0 exceptions)
[2021-03-31 18:28:12.146] [driver] [info] [Task Finished] Transform to mem in 0.253079s (14170 normal rows, 0 exceptions)
[2021-03-31 18:28:12.398] [driver] [info] [Task Finished] Transform to mem in 0.252320s (14002 normal rows, 0 exceptions)
[2021-03-31 18:28:12.652] [driver] [info] [Task Finished] Transform to mem in 0.253472s (14226 normal rows, 0 exceptions)
[2021-03-31 18:28:12.905] [driver] [info] [Task Finished] Transform to mem in 0.252858s (14206 normal rows, 0 exceptions)
[2021-03-31 18:28:13.157] [driver] [info] [Task Finished] Transform to mem in 0.251864s (13961 normal rows, 0 exceptions)
[2021-03-31 18:28:13.410] [driver] [info] [Task Finished] Transform to mem in 0.252864s (14246 normal rows, 0 exceptions)
[2021-03-31 18:28:13.662] [driver] [info] [Task Finished] Transform to mem in 0.252838s (14233 normal rows, 0 exceptions)
[2021-03-31 18:28:13.960] [driver] [info] [Task Finished] Transform to mem in 0.297913s (16320 normal rows, 0 exceptions)
[2021-03-31 18:28:13.960] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 40.5526s
[2021-03-31 18:28:13.960] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.5499s, 54108450 input rows, time to process 1 row via fast path: 0.00074942ms

[2021-03-31 18:28:13.998] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:28:14.415] [driver] [info] [Task Finished] write to file in 0.416381s (2264570 rows, 344.74 MB)
[2021-03-31 18:28:14.415] [global] [info] writing output took 0.454188s
[2021-03-31 18:28:14.415] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.454278s
[2021-03-31 18:28:14.415] [global] [info] [Transform Stage] Stage 0 took 41.2123s
[2021-03-31 18:28:14.415] [global] [info] Query Execution took 41.2689s. (planning: 0.0565853s, execution: 41.2123s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.005040884017944336
Tuplex job time: 41.53134608268738 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0195576,"llvm_optimization_time_s":0.136421,"llvm_compilation_time_s":0.0616346,"total_compilation_time_s":0.205405,"sampling_time_s":0.108152,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.5499,"fast_path_time_s":40.5526,"fast_path_per_row_time_ns":749.42,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.005040884017944336, "jobTime": 41.53134608268738}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
