[2021-03-23 12:34:37.452] [global] [error] could not find backend, defaulting to local
[2021-03-23 12:34:37.453] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-23 12:34:37.453] [local ee] [info] initializing LLVM backend
[2021-03-23 12:34:37.453] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-23 12:34:37.453] [LLVM] [info] compiling code for skylake-avx512
[2021-03-23 12:34:37.454] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-23 12:34:37.454] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-23 12:34:37.454] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-23 12:34:37.454] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-23 12:34:37.589] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.604] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.612] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.638] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.658] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.671] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.702] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:34:37.718] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:34:37.721] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:34:37.723] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:34:37.726] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:34:37.741] [logical planner] [info] logical optimization took 0.019927ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-23 12:34:37.741] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:34:37.742] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-23 12:34:37.743] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-23 12:34:37.743] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-23 12:34:37.744] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-23 12:34:37.744] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-23 12:34:37.745] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-23 12:34:37.746] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-23 12:34:37.747] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-23 12:34:37.748] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-23 12:34:37.748] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-23 12:34:37.750] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:34:37.750] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:34:37.751] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:34:37.776] [global] [info] Stage0 depends on: 
[2021-03-23 12:34:37.921] [global] [info] Optimization via LLVM passes took 0.137506 ms
[2021-03-23 12:34:37.983] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-23 12:34:37.983] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.20664s
[2021-03-23 12:34:37.983] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-23 12:34:38.244] [driver] [info] [Task Finished] Transform to mem in 0.260645s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:38.501] [driver] [info] [Task Finished] Transform to mem in 0.257099s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:38.758] [driver] [info] [Task Finished] Transform to mem in 0.256626s (13970 normal rows, 0 exceptions)
[2021-03-23 12:34:39.016] [driver] [info] [Task Finished] Transform to mem in 0.258261s (14253 normal rows, 0 exceptions)
[2021-03-23 12:34:39.274] [driver] [info] [Task Finished] Transform to mem in 0.257451s (14217 normal rows, 0 exceptions)
[2021-03-23 12:34:39.530] [driver] [info] [Task Finished] Transform to mem in 0.256801s (13920 normal rows, 0 exceptions)
[2021-03-23 12:34:39.789] [driver] [info] [Task Finished] Transform to mem in 0.258393s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:40.046] [driver] [info] [Task Finished] Transform to mem in 0.257506s (14248 normal rows, 0 exceptions)
[2021-03-23 12:34:40.303] [driver] [info] [Task Finished] Transform to mem in 0.256999s (13953 normal rows, 0 exceptions)
[2021-03-23 12:34:40.561] [driver] [info] [Task Finished] Transform to mem in 0.257982s (14207 normal rows, 0 exceptions)
[2021-03-23 12:34:40.818] [driver] [info] [Task Finished] Transform to mem in 0.256884s (14237 normal rows, 0 exceptions)
[2021-03-23 12:34:41.075] [driver] [info] [Task Finished] Transform to mem in 0.256320s (13988 normal rows, 0 exceptions)
[2021-03-23 12:34:41.332] [driver] [info] [Task Finished] Transform to mem in 0.257054s (14177 normal rows, 0 exceptions)
[2021-03-23 12:34:41.589] [driver] [info] [Task Finished] Transform to mem in 0.257084s (14242 normal rows, 0 exceptions)
[2021-03-23 12:34:41.845] [driver] [info] [Task Finished] Transform to mem in 0.256221s (14020 normal rows, 0 exceptions)
[2021-03-23 12:34:42.102] [driver] [info] [Task Finished] Transform to mem in 0.257176s (14162 normal rows, 0 exceptions)
[2021-03-23 12:34:42.359] [driver] [info] [Task Finished] Transform to mem in 0.256964s (14241 normal rows, 0 exceptions)
[2021-03-23 12:34:42.616] [driver] [info] [Task Finished] Transform to mem in 0.256920s (14059 normal rows, 0 exceptions)
[2021-03-23 12:34:42.873] [driver] [info] [Task Finished] Transform to mem in 0.257217s (14114 normal rows, 0 exceptions)
[2021-03-23 12:34:43.130] [driver] [info] [Task Finished] Transform to mem in 0.257182s (14238 normal rows, 0 exceptions)
[2021-03-23 12:34:43.387] [driver] [info] [Task Finished] Transform to mem in 0.256808s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:43.644] [driver] [info] [Task Finished] Transform to mem in 0.257025s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:43.902] [driver] [info] [Task Finished] Transform to mem in 0.257444s (14221 normal rows, 0 exceptions)
[2021-03-23 12:34:44.159] [driver] [info] [Task Finished] Transform to mem in 0.256773s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:44.416] [driver] [info] [Task Finished] Transform to mem in 0.256927s (14097 normal rows, 0 exceptions)
[2021-03-23 12:34:44.673] [driver] [info] [Task Finished] Transform to mem in 0.257250s (14212 normal rows, 0 exceptions)
[2021-03-23 12:34:44.930] [driver] [info] [Task Finished] Transform to mem in 0.256863s (14108 normal rows, 0 exceptions)
[2021-03-23 12:34:45.187] [driver] [info] [Task Finished] Transform to mem in 0.257263s (14100 normal rows, 0 exceptions)
[2021-03-23 12:34:45.444] [driver] [info] [Task Finished] Transform to mem in 0.256976s (14212 normal rows, 0 exceptions)
[2021-03-23 12:34:45.700] [driver] [info] [Task Finished] Transform to mem in 0.256519s (14099 normal rows, 0 exceptions)
[2021-03-23 12:34:45.957] [driver] [info] [Task Finished] Transform to mem in 0.256957s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:46.215] [driver] [info] [Task Finished] Transform to mem in 0.257373s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:46.472] [driver] [info] [Task Finished] Transform to mem in 0.257142s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:46.729] [driver] [info] [Task Finished] Transform to mem in 0.257038s (14083 normal rows, 0 exceptions)
[2021-03-23 12:34:46.987] [driver] [info] [Task Finished] Transform to mem in 0.257610s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:47.244] [driver] [info] [Task Finished] Transform to mem in 0.257250s (14170 normal rows, 0 exceptions)
[2021-03-23 12:34:47.501] [driver] [info] [Task Finished] Transform to mem in 0.256828s (14002 normal rows, 0 exceptions)
[2021-03-23 12:34:47.759] [driver] [info] [Task Finished] Transform to mem in 0.257906s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:48.016] [driver] [info] [Task Finished] Transform to mem in 0.257321s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:48.273] [driver] [info] [Task Finished] Transform to mem in 0.256915s (13961 normal rows, 0 exceptions)
[2021-03-23 12:34:48.531] [driver] [info] [Task Finished] Transform to mem in 0.257566s (14246 normal rows, 0 exceptions)
[2021-03-23 12:34:48.787] [driver] [info] [Task Finished] Transform to mem in 0.256898s (14233 normal rows, 0 exceptions)
[2021-03-23 12:34:49.044] [driver] [info] [Task Finished] Transform to mem in 0.256794s (13915 normal rows, 0 exceptions)
[2021-03-23 12:34:49.302] [driver] [info] [Task Finished] Transform to mem in 0.257748s (14253 normal rows, 0 exceptions)
[2021-03-23 12:34:49.559] [driver] [info] [Task Finished] Transform to mem in 0.256947s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:49.816] [driver] [info] [Task Finished] Transform to mem in 0.257150s (13892 normal rows, 0 exceptions)
[2021-03-23 12:34:50.074] [driver] [info] [Task Finished] Transform to mem in 0.257899s (14245 normal rows, 0 exceptions)
[2021-03-23 12:34:50.332] [driver] [info] [Task Finished] Transform to mem in 0.257895s (14260 normal rows, 0 exceptions)
[2021-03-23 12:34:50.589] [driver] [info] [Task Finished] Transform to mem in 0.256853s (13978 normal rows, 0 exceptions)
[2021-03-23 12:34:50.847] [driver] [info] [Task Finished] Transform to mem in 0.257687s (14181 normal rows, 0 exceptions)
[2021-03-23 12:34:51.104] [driver] [info] [Task Finished] Transform to mem in 0.257133s (14234 normal rows, 0 exceptions)
[2021-03-23 12:34:51.360] [driver] [info] [Task Finished] Transform to mem in 0.256441s (14015 normal rows, 0 exceptions)
[2021-03-23 12:34:51.618] [driver] [info] [Task Finished] Transform to mem in 0.257415s (14142 normal rows, 0 exceptions)
[2021-03-23 12:34:51.875] [driver] [info] [Task Finished] Transform to mem in 0.257521s (14280 normal rows, 0 exceptions)
[2021-03-23 12:34:52.132] [driver] [info] [Task Finished] Transform to mem in 0.257221s (13982 normal rows, 0 exceptions)
[2021-03-23 12:34:52.390] [driver] [info] [Task Finished] Transform to mem in 0.257158s (14173 normal rows, 0 exceptions)
[2021-03-23 12:34:52.647] [driver] [info] [Task Finished] Transform to mem in 0.257204s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:52.903] [driver] [info] [Task Finished] Transform to mem in 0.256642s (14054 normal rows, 0 exceptions)
[2021-03-23 12:34:53.160] [driver] [info] [Task Finished] Transform to mem in 0.256807s (14129 normal rows, 0 exceptions)
[2021-03-23 12:34:53.417] [driver] [info] [Task Finished] Transform to mem in 0.256835s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:53.674] [driver] [info] [Task Finished] Transform to mem in 0.256940s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:53.931] [driver] [info] [Task Finished] Transform to mem in 0.256873s (14108 normal rows, 0 exceptions)
[2021-03-23 12:34:54.188] [driver] [info] [Task Finished] Transform to mem in 0.257417s (14208 normal rows, 0 exceptions)
[2021-03-23 12:34:54.445] [driver] [info] [Task Finished] Transform to mem in 0.256803s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:54.703] [driver] [info] [Task Finished] Transform to mem in 0.257346s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:54.960] [driver] [info] [Task Finished] Transform to mem in 0.257054s (14232 normal rows, 0 exceptions)
[2021-03-23 12:34:55.217] [driver] [info] [Task Finished] Transform to mem in 0.256891s (14088 normal rows, 0 exceptions)
[2021-03-23 12:34:55.473] [driver] [info] [Task Finished] Transform to mem in 0.256859s (14106 normal rows, 0 exceptions)
[2021-03-23 12:34:55.731] [driver] [info] [Task Finished] Transform to mem in 0.257060s (14211 normal rows, 0 exceptions)
[2021-03-23 12:34:55.987] [driver] [info] [Task Finished] Transform to mem in 0.256731s (14113 normal rows, 0 exceptions)
[2021-03-23 12:34:56.244] [driver] [info] [Task Finished] Transform to mem in 0.257053s (14075 normal rows, 0 exceptions)
[2021-03-23 12:34:56.502] [driver] [info] [Task Finished] Transform to mem in 0.257727s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:56.759] [driver] [info] [Task Finished] Transform to mem in 0.256865s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:57.016] [driver] [info] [Task Finished] Transform to mem in 0.257114s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:57.273] [driver] [info] [Task Finished] Transform to mem in 0.257102s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:57.531] [driver] [info] [Task Finished] Transform to mem in 0.257774s (14170 normal rows, 0 exceptions)
[2021-03-23 12:34:57.788] [driver] [info] [Task Finished] Transform to mem in 0.256844s (14002 normal rows, 0 exceptions)
[2021-03-23 12:34:58.045] [driver] [info] [Task Finished] Transform to mem in 0.257278s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:58.302] [driver] [info] [Task Finished] Transform to mem in 0.257080s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:58.558] [driver] [info] [Task Finished] Transform to mem in 0.256063s (13970 normal rows, 0 exceptions)
[2021-03-23 12:34:58.817] [driver] [info] [Task Finished] Transform to mem in 0.258252s (14237 normal rows, 0 exceptions)
[2021-03-23 12:34:59.074] [driver] [info] [Task Finished] Transform to mem in 0.257495s (14233 normal rows, 0 exceptions)
[2021-03-23 12:34:59.330] [driver] [info] [Task Finished] Transform to mem in 0.255891s (13920 normal rows, 0 exceptions)
[2021-03-23 12:34:59.588] [driver] [info] [Task Finished] Transform to mem in 0.257888s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:59.845] [driver] [info] [Task Finished] Transform to mem in 0.257065s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:00.101] [driver] [info] [Task Finished] Transform to mem in 0.256149s (13912 normal rows, 0 exceptions)
[2021-03-23 12:35:00.358] [driver] [info] [Task Finished] Transform to mem in 0.257201s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:00.616] [driver] [info] [Task Finished] Transform to mem in 0.257150s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:00.872] [driver] [info] [Task Finished] Transform to mem in 0.256906s (13988 normal rows, 0 exceptions)
[2021-03-23 12:35:01.130] [driver] [info] [Task Finished] Transform to mem in 0.257548s (14171 normal rows, 0 exceptions)
[2021-03-23 12:35:01.387] [driver] [info] [Task Finished] Transform to mem in 0.257055s (14234 normal rows, 0 exceptions)
[2021-03-23 12:35:01.643] [driver] [info] [Task Finished] Transform to mem in 0.256252s (14015 normal rows, 0 exceptions)
[2021-03-23 12:35:01.901] [driver] [info] [Task Finished] Transform to mem in 0.257196s (14176 normal rows, 0 exceptions)
[2021-03-23 12:35:02.158] [driver] [info] [Task Finished] Transform to mem in 0.257153s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:02.414] [driver] [info] [Task Finished] Transform to mem in 0.256614s (14059 normal rows, 0 exceptions)
[2021-03-23 12:35:02.672] [driver] [info] [Task Finished] Transform to mem in 0.257706s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:02.929] [driver] [info] [Task Finished] Transform to mem in 0.256723s (14238 normal rows, 0 exceptions)
[2021-03-23 12:35:03.186] [driver] [info] [Task Finished] Transform to mem in 0.256798s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:03.443] [driver] [info] [Task Finished] Transform to mem in 0.256949s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:03.699] [driver] [info] [Task Finished] Transform to mem in 0.256793s (14221 normal rows, 0 exceptions)
[2021-03-23 12:35:03.956] [driver] [info] [Task Finished] Transform to mem in 0.256801s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:04.213] [driver] [info] [Task Finished] Transform to mem in 0.256934s (14091 normal rows, 0 exceptions)
[2021-03-23 12:35:04.470] [driver] [info] [Task Finished] Transform to mem in 0.256771s (14218 normal rows, 0 exceptions)
[2021-03-23 12:35:04.726] [driver] [info] [Task Finished] Transform to mem in 0.256263s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:04.983] [driver] [info] [Task Finished] Transform to mem in 0.257145s (14113 normal rows, 0 exceptions)
[2021-03-23 12:35:05.241] [driver] [info] [Task Finished] Transform to mem in 0.257276s (14197 normal rows, 0 exceptions)
[2021-03-23 12:35:05.498] [driver] [info] [Task Finished] Transform to mem in 0.257057s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:05.755] [driver] [info] [Task Finished] Transform to mem in 0.257017s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:06.012] [driver] [info] [Task Finished] Transform to mem in 0.256754s (14197 normal rows, 0 exceptions)
[2021-03-23 12:35:06.268] [driver] [info] [Task Finished] Transform to mem in 0.256474s (14115 normal rows, 0 exceptions)
[2021-03-23 12:35:06.525] [driver] [info] [Task Finished] Transform to mem in 0.257376s (14083 normal rows, 0 exceptions)
[2021-03-23 12:35:06.784] [driver] [info] [Task Finished] Transform to mem in 0.258361s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:07.041] [driver] [info] [Task Finished] Transform to mem in 0.256984s (14144 normal rows, 0 exceptions)
[2021-03-23 12:35:07.299] [driver] [info] [Task Finished] Transform to mem in 0.257704s (14015 normal rows, 0 exceptions)
[2021-03-23 12:35:07.557] [driver] [info] [Task Finished] Transform to mem in 0.257983s (14237 normal rows, 0 exceptions)
[2021-03-23 12:35:07.815] [driver] [info] [Task Finished] Transform to mem in 0.258073s (14159 normal rows, 0 exceptions)
[2021-03-23 12:35:08.073] [driver] [info] [Task Finished] Transform to mem in 0.258029s (14010 normal rows, 0 exceptions)
[2021-03-23 12:35:08.330] [driver] [info] [Task Finished] Transform to mem in 0.257717s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:08.588] [driver] [info] [Task Finished] Transform to mem in 0.257731s (14178 normal rows, 0 exceptions)
[2021-03-23 12:35:08.846] [driver] [info] [Task Finished] Transform to mem in 0.257790s (13970 normal rows, 0 exceptions)
[2021-03-23 12:35:09.105] [driver] [info] [Task Finished] Transform to mem in 0.258999s (14253 normal rows, 0 exceptions)
[2021-03-23 12:35:09.363] [driver] [info] [Task Finished] Transform to mem in 0.258080s (14236 normal rows, 0 exceptions)
[2021-03-23 12:35:09.621] [driver] [info] [Task Finished] Transform to mem in 0.257612s (13901 normal rows, 0 exceptions)
[2021-03-23 12:35:09.880] [driver] [info] [Task Finished] Transform to mem in 0.258891s (14267 normal rows, 0 exceptions)
[2021-03-23 12:35:10.138] [driver] [info] [Task Finished] Transform to mem in 0.258263s (14248 normal rows, 0 exceptions)
[2021-03-23 12:35:10.396] [driver] [info] [Task Finished] Transform to mem in 0.257727s (13953 normal rows, 0 exceptions)
[2021-03-23 12:35:10.655] [driver] [info] [Task Finished] Transform to mem in 0.259149s (14207 normal rows, 0 exceptions)
[2021-03-23 12:35:10.913] [driver] [info] [Task Finished] Transform to mem in 0.258387s (14245 normal rows, 0 exceptions)
[2021-03-23 12:35:11.170] [driver] [info] [Task Finished] Transform to mem in 0.256666s (13999 normal rows, 0 exceptions)
[2021-03-23 12:35:11.428] [driver] [info] [Task Finished] Transform to mem in 0.258192s (14158 normal rows, 0 exceptions)
[2021-03-23 12:35:11.685] [driver] [info] [Task Finished] Transform to mem in 0.257028s (14242 normal rows, 0 exceptions)
[2021-03-23 12:35:11.943] [driver] [info] [Task Finished] Transform to mem in 0.257588s (14020 normal rows, 0 exceptions)
[2021-03-23 12:35:12.201] [driver] [info] [Task Finished] Transform to mem in 0.258385s (14173 normal rows, 0 exceptions)
[2021-03-23 12:35:12.459] [driver] [info] [Task Finished] Transform to mem in 0.257732s (14235 normal rows, 0 exceptions)
[2021-03-23 12:35:12.718] [driver] [info] [Task Finished] Transform to mem in 0.258813s (14054 normal rows, 0 exceptions)
[2021-03-23 12:35:12.976] [driver] [info] [Task Finished] Transform to mem in 0.258242s (14114 normal rows, 0 exceptions)
[2021-03-23 12:35:13.234] [driver] [info] [Task Finished] Transform to mem in 0.258199s (14238 normal rows, 0 exceptions)
[2021-03-23 12:35:13.492] [driver] [info] [Task Finished] Transform to mem in 0.257747s (14081 normal rows, 0 exceptions)
[2021-03-23 12:35:13.750] [driver] [info] [Task Finished] Transform to mem in 0.257421s (14095 normal rows, 0 exceptions)
[2021-03-23 12:35:14.007] [driver] [info] [Task Finished] Transform to mem in 0.257725s (14221 normal rows, 0 exceptions)
[2021-03-23 12:35:14.265] [driver] [info] [Task Finished] Transform to mem in 0.257372s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:14.523] [driver] [info] [Task Finished] Transform to mem in 0.258060s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:14.780] [driver] [info] [Task Finished] Transform to mem in 0.257695s (14212 normal rows, 0 exceptions)
[2021-03-23 12:35:15.038] [driver] [info] [Task Finished] Transform to mem in 0.257560s (14108 normal rows, 0 exceptions)
[2021-03-23 12:35:15.296] [driver] [info] [Task Finished] Transform to mem in 0.258100s (14100 normal rows, 0 exceptions)
[2021-03-23 12:35:15.555] [driver] [info] [Task Finished] Transform to mem in 0.258572s (14217 normal rows, 0 exceptions)
[2021-03-23 12:35:15.812] [driver] [info] [Task Finished] Transform to mem in 0.257559s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:16.070] [driver] [info] [Task Finished] Transform to mem in 0.257788s (14094 normal rows, 0 exceptions)
[2021-03-23 12:35:16.328] [driver] [info] [Task Finished] Transform to mem in 0.257495s (14223 normal rows, 0 exceptions)
[2021-03-23 12:35:16.585] [driver] [info] [Task Finished] Transform to mem in 0.257425s (14089 normal rows, 0 exceptions)
[2021-03-23 12:35:16.843] [driver] [info] [Task Finished] Transform to mem in 0.257719s (14086 normal rows, 0 exceptions)
[2021-03-23 12:35:17.101] [driver] [info] [Task Finished] Transform to mem in 0.258164s (14232 normal rows, 0 exceptions)
[2021-03-23 12:35:17.359] [driver] [info] [Task Finished] Transform to mem in 0.257627s (14170 normal rows, 0 exceptions)
[2021-03-23 12:35:17.616] [driver] [info] [Task Finished] Transform to mem in 0.257335s (14002 normal rows, 0 exceptions)
[2021-03-23 12:35:17.874] [driver] [info] [Task Finished] Transform to mem in 0.257965s (14226 normal rows, 0 exceptions)
[2021-03-23 12:35:18.132] [driver] [info] [Task Finished] Transform to mem in 0.258376s (14206 normal rows, 0 exceptions)
[2021-03-23 12:35:18.389] [driver] [info] [Task Finished] Transform to mem in 0.257026s (13961 normal rows, 0 exceptions)
[2021-03-23 12:35:18.648] [driver] [info] [Task Finished] Transform to mem in 0.258310s (14246 normal rows, 0 exceptions)
[2021-03-23 12:35:18.907] [driver] [info] [Task Finished] Transform to mem in 0.259415s (14233 normal rows, 0 exceptions)
[2021-03-23 12:35:19.209] [driver] [info] [Task Finished] Transform to mem in 0.301566s (16320 normal rows, 0 exceptions)
[2021-03-23 12:35:19.209] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 41.2257s
[2021-03-23 12:35:19.209] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.2227s, 54108450 input rows, time to process 1 row via fast path: 0.000761852ms

[2021-03-23 12:35:19.250] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-23 12:35:19.632] [driver] [info] [Task Finished] write to file in 0.381894s (2264570 rows, 344.74 MB)
[2021-03-23 12:35:19.632] [global] [info] writing output took 0.423238s
[2021-03-23 12:35:19.632] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.42333s
[2021-03-23 12:35:19.632] [global] [info] [Transform Stage] Stage 0 took 41.8557s
[2021-03-23 12:35:19.632] [global] [info] Query Execution took 41.9132s. (planning: 0.057523s, execution: 41.8557s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.004980564117431641
Tuplex job time: 42.17967104911804 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '4e1bb76d17c5', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '4MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0199272,"llvm_optimization_time_s":0.137506,"llvm_compilation_time_s":0.0618077,"total_compilation_time_s":0.206637,"sampling_time_s":0.110826,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.2227,"fast_path_time_s":41.2257,"fast_path_per_row_time_ns":761.852,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004980564117431641, "jobTime": 42.17967104911804}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
